From amk at amk.ca  Thu Dec  1 00:10:38 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Wed, 30 Nov 2005 18:10:38 -0500
Subject: [Python-Dev] Python bug day this Sunday
Message-ID: <20051130231038.GA24087@rogue.amk.ca>

Let's have a Python bug day this Sunday.  One goal might be
to assess bugs and patches, and make a list of ones we can work on at  
the Python core sprint at PyCon
<http://wiki.python.org/moin/PyCon2006/Sprints/PythonCore>.

Meeting on IRC: #python-dev on irc.freenode.net

Date: Sunday, December 4th

Time: roughly 9AM to 3PM Eastern (2PM to 8PM UTC).

        People on the US West Coast may want to show up from 9AM to
        3PM Pacific time (12PM to 6PM Eastern), because it'll be more
        convenient.

--amk

From martin at v.loewis.de  Thu Dec  1 00:12:48 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 01 Dec 2005 00:12:48 +0100
Subject: [Python-Dev] ast-objects branch created
Message-ID: <438E31F0.1020900@v.loewis.de>

I created

http://svn.python.org/projects/python/branches/ast-objects/

You can convert your repository to that branch with

svn switch svn+ssh://pythondev at svn.python.org/python/branches/ast-objects

in the toplevel directory. In particular, this features

http://svn.python.org/projects/python/branches/ast-objects/Parser/asdl_c.py
http://svn.python.org/projects/python/branches/ast-objects/Include/Python-ast.h
http://svn.python.org/projects/python/branches/ast-objects/Python/Python-ast.c

The status is currently this:
- asdl_c generates a type hierarchy: "Sum" productions have one type
   per constructor, inheriting from a type for the sum; plain products
   only have a type for the product.
- attributes are in the base type, accessible through o->_base.attr;
   projections of the product types are accessible directly through
   member names.
- all projections must be non-NULL. Sequences are represented through
   potentially empty lists; optional types are potentially represented
   through Py_None. bool is gone; use Py_True/Py_False. The only
   primitive type remaining is int (which only occurs in lineno)
- the types currently have only a constructor, a dealloc function,
   and an _Check macro.
- Naming is this: for cross-object-file visible symbols (functions
   and global variables), a Py_ prefix is used. Otherwise, I use
   the type name or constructor name directly. There is a #define
   for the Py_<type>_New function, so you can also write <type>(params).
   Parameter order for the types is: projections first, then
   attributes.
- For compatibility with the current code, the Sum base types
   also have the _kind enumeration (although that appears not
   to get initialized right now).

For asdl_c, I see the following things as TODOs:
- add support for traversing the types from C, through tp_members
   (read-only). Optionally add support for pickling.
- add support for garbage collection. I don't expect this to be
   necessary right now, but will be if the API is exposed, and it
   doesn't cost much.

The bigger chunk of necessary changes is in using these, starting
with ast.c.

Feel free to commit any changes to that branch that you consider
helpful. To avoid duplicated work, posting a note here might
also help.

Regards,
Martin

From martin at v.loewis.de  Thu Dec  1 00:14:12 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 01 Dec 2005 00:14:12 +0100
Subject: [Python-Dev] Memory management in the AST parser & compiler
In-Reply-To: <438D73FC.4090009@gmail.com>
References: <4379AAD7.2050506@iinet.net.au>	<ca471dc20511281214h11fd0572kea0f811a1fa45d3@mail.gmail.com>	<e8bf7a530511281247n665ea1f6qdac70180f3d89fef@mail.gmail.com>	<ca471dc20511281315s1f8caeb0wae7ce7931c5354e3@mail.gmail.com>	<e8bf7a530511281323h3a1bfe49ydbf9f0b38609e56b@mail.gmail.com>	<ca471dc20511281346p7156a1f7i4c705561e2e28da1@mail.gmail.com>	<ee2a432c0511281458n27806bd6y4358d2eeefc94a50@mail.gmail.com>	<438B9F12.3060607@v.loewis.de>	<ee2a432c0511282324m2400e968n6d4f48d531268257@mail.gmail.com>	<438C50C8.9040005@gmail.com>	<ee2a432c0511291029m5bdc4564s84457533037a7e11@mail.gmail.com>	<438CDBA5.9050207@canterbury.ac.nz>
	<438D73FC.4090009@gmail.com>
Message-ID: <438E3244.2080808@v.loewis.de>

Nick Coghlan wrote:
 > The ast C structs are already auto-generated by a Python script 
(asdl_c.py, to
 > be precise). The trick is to make that script generate full PyObjects 
rather
 > than the simple C structures that it generates now.

See the ast-object branch.

 > The second step is to then modify ast.c to use the new structures. A 
branch
 > probably wouldn't help much with initial development (this is a 
"break the
 > world, check in when stuff compiles again" kind of change, which is 
hard to
 > split amongst multiple people), but I think it would be of benefit when
 > reviewing the change before moving it back to the trunk.

Well, there would be a clear two-split right now: one could change
ast.c, and the other compile.c.

Regards,
Martin

From jeremy at alum.mit.edu  Thu Dec  1 04:36:15 2005
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 30 Nov 2005 22:36:15 -0500
Subject: [Python-Dev] Memory management in the AST parser & compiler
In-Reply-To: <ee2a432c0511301021m2e72d710r173f085b84cc2f4@mail.gmail.com>
References: <4379AAD7.2050506@iinet.net.au>
	<ee2a432c0511281458n27806bd6y4358d2eeefc94a50@mail.gmail.com>
	<438B9F12.3060607@v.loewis.de>
	<ee2a432c0511282324m2400e968n6d4f48d531268257@mail.gmail.com>
	<438C50C8.9040005@gmail.com>
	<ee2a432c0511291029m5bdc4564s84457533037a7e11@mail.gmail.com>
	<438CDBA5.9050207@canterbury.ac.nz> <438D73FC.4090009@gmail.com>
	<438D93F0.3000005@gmail.com>
	<ee2a432c0511301021m2e72d710r173f085b84cc2f4@mail.gmail.com>
Message-ID: <e8bf7a530511301936w6e246f89gc6b59afc13d4650e@mail.gmail.com>

On 11/30/05, Neal Norwitz <nnorwitz at gmail.com> wrote:
> On 11/30/05, Thomas Lee <krumms at gmail.com> wrote:
> >
> > Quick semi-related question: where are the marshal_* functions called?
> > They're all static in Python-ast.c and don't seem to be actually called
> > anywhere. Can we ditch them?
>
> I *think* they are not necessary.  My guess is that they were there
> for marshaling the AST to disk, though I'm not sure why we would want
> to do that.  It could have been there was the idea of how they would
> be marshalled to PyObjects and exported.
>
> Unless you hear otherwise from Jeremy, I would probably remove them.
>
> I can check your patch into the branch so others can get an idea and
> hopefully provide comments.

The intent was to share the AST objects between C and Python by coping
them.  I still think passing copies is better than sharing live
objects between Python and C, although the specific mechanism may be
different if the C objects are PyObjects.

Jeremy

From greg.ewing at canterbury.ac.nz  Thu Dec  1 05:05:24 2005
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 01 Dec 2005 17:05:24 +1300
Subject: [Python-Dev] Memory management in the AST parser & compiler
In-Reply-To: <e8bf7a530511301936w6e246f89gc6b59afc13d4650e@mail.gmail.com>
References: <4379AAD7.2050506@iinet.net.au>
	<ee2a432c0511281458n27806bd6y4358d2eeefc94a50@mail.gmail.com>
	<438B9F12.3060607@v.loewis.de>
	<ee2a432c0511282324m2400e968n6d4f48d531268257@mail.gmail.com>
	<438C50C8.9040005@gmail.com>
	<ee2a432c0511291029m5bdc4564s84457533037a7e11@mail.gmail.com>
	<438CDBA5.9050207@canterbury.ac.nz> <438D73FC.4090009@gmail.com>
	<438D93F0.3000005@gmail.com>
	<ee2a432c0511301021m2e72d710r173f085b84cc2f4@mail.gmail.com>
	<e8bf7a530511301936w6e246f89gc6b59afc13d4650e@mail.gmail.com>
Message-ID: <438E7684.3070001@canterbury.ac.nz>

Jeremy Hylton wrote:

> I still think passing copies is better than sharing live
> objects between Python and C,

Even if the objects are immutable?

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From jeremy at alum.mit.edu  Thu Dec  1 06:02:05 2005
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 1 Dec 2005 00:02:05 -0500
Subject: [Python-Dev] Memory management in the AST parser & compiler
In-Reply-To: <438E7684.3070001@canterbury.ac.nz>
References: <4379AAD7.2050506@iinet.net.au>
	<ee2a432c0511282324m2400e968n6d4f48d531268257@mail.gmail.com>
	<438C50C8.9040005@gmail.com>
	<ee2a432c0511291029m5bdc4564s84457533037a7e11@mail.gmail.com>
	<438CDBA5.9050207@canterbury.ac.nz> <438D73FC.4090009@gmail.com>
	<438D93F0.3000005@gmail.com>
	<ee2a432c0511301021m2e72d710r173f085b84cc2f4@mail.gmail.com>
	<e8bf7a530511301936w6e246f89gc6b59afc13d4650e@mail.gmail.com>
	<438E7684.3070001@canterbury.ac.nz>
Message-ID: <e8bf7a530511302102t52de2255n8ffd7d34a8642025@mail.gmail.com>

Sure.  If they're immutable sharing is fine, but you end up making a
copy anyway if you want to make changes, right?

Jeremy

On 11/30/05, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Jeremy Hylton wrote:
>
> > I still think passing copies is better than sharing live
> > objects between Python and C,
>
> Even if the objects are immutable?
>
> --
> Greg Ewing, Computer Science Dept, +--------------------------------------+
> University of Canterbury,          | A citizen of NewZealandCorp, a       |
> Christchurch, New Zealand          | wholly-owned subsidiary of USA Inc.  |
> greg.ewing at canterbury.ac.nz        +--------------------------------------+
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>

From nnorwitz at gmail.com  Thu Dec  1 07:40:21 2005
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Wed, 30 Nov 2005 22:40:21 -0800
Subject: [Python-Dev] ast-objects branch created
In-Reply-To: <438E31F0.1020900@v.loewis.de>
References: <438E31F0.1020900@v.loewis.de>
Message-ID: <ee2a432c0511302240h8c7cf25x4274934a6bf9c241@mail.gmail.com>

On 11/30/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> The bigger chunk of necessary changes is in using these, starting
> with ast.c.

I got a few more files to compile.  The following files (all under
Python/) need some loving care and are looking for a kind soul to
adopt them:

  ast.c, compile.c, future.c, symtable.c

Of these, future.c is by far the easiest to get compiling.

n

From bcannon at gmail.com  Thu Dec  1 08:40:43 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Wed, 30 Nov 2005 23:40:43 -0800
Subject: [Python-Dev] ast-objects branch created
In-Reply-To: <438E31F0.1020900@v.loewis.de>
References: <438E31F0.1020900@v.loewis.de>
Message-ID: <bbaeab100511302340h3e2a3c7ai5c7a4f27859bfa17@mail.gmail.com>

On 11/30/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I created
>
> http://svn.python.org/projects/python/branches/ast-objects/
>
> You can convert your repository to that branch with
>
> svn switch svn+ssh://pythondev at svn.python.org/python/branches/ast-objects
>

If you would rather do a separate checkout, do

 svn checkout svn+ssh://pythondev at svn.python.org/python/branches/ast-objects

If you want a read-only checkout, see the newly updated entry on
checking out projects in the dev FAQ at
http://www.python.org/dev/devfaq.html#how-do-i-get-a-checkout-of-the-repository-read-only-and-read-write
.

-Brett

From martin at v.loewis.de  Thu Dec  1 09:22:50 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 01 Dec 2005 09:22:50 +0100
Subject: [Python-Dev] PyAST_FromNode returning PyTypeObject* ?
Message-ID: <438EB2DA.5060309@v.loewis.de>

Neal,

Why did you suggest that PyAST_FromNode returns PyTypeObject*?
I can't see why type objects are much used in the AST, unless
I'm missing something essential.

Anyway, I started converting ast.c (two functions only), and
noticed that there is a convention to have nested variables
referring to fresh memory (e.g. inside switch statements). I
started changing these to have all variables at the toplevel.

Then, in either success or failure, you have to release all
of them. Unfortunately, sometimes in failure, an additional
function is called, which isn't called in success. So I
added a success: label.

Also, it is somewhat inconvenient that PyList_SET_ITEM
steals references. Currently, I INCREF the objects added
to the list (as the success: label will DECREF them);
alternatively, clearing the pointer to NULL might also
be appropriate. Perhaps we could have a STEAL_ITEM
macro inside ast.c:

#define STEAL_ITEM(list,index,variable) \
do{PyList_SET_ITEM(list,index,variable);variable=NULL;}while(0)

Regards,
Martin

From ncoghlan at gmail.com  Thu Dec  1 09:48:43 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 01 Dec 2005 18:48:43 +1000
Subject: [Python-Dev] Memory management in the AST parser & compiler
In-Reply-To: <438E3244.2080808@v.loewis.de>
References: <4379AAD7.2050506@iinet.net.au>	<ca471dc20511281214h11fd0572kea0f811a1fa45d3@mail.gmail.com>	<e8bf7a530511281247n665ea1f6qdac70180f3d89fef@mail.gmail.com>	<ca471dc20511281315s1f8caeb0wae7ce7931c5354e3@mail.gmail.com>	<e8bf7a530511281323h3a1bfe49ydbf9f0b38609e56b@mail.gmail.com>	<ca471dc20511281346p7156a1f7i4c705561e2e28da1@mail.gmail.com>	<ee2a432c0511281458n27806bd6y4358d2eeefc94a50@mail.gmail.com>	<438B9F12.3060607@v.loewis.de>	<ee2a432c0511282324m2400e968n6d4f48d531268257@mail.gmail.com>	<438C50C8.9040005@gmail.com>	<ee2a432c0511291029m5bdc4564s84457533037a7e11@mail.gmail.com>	<438CDBA5.9050207@canterbury.ac.nz>
	<438D73FC.4090009@gmail.com> <438E3244.2080808@v.loewis.de>
Message-ID: <438EB8EB.308@gmail.com>

Martin v. L?wis wrote:
> Nick Coghlan wrote:
>  > The ast C structs are already auto-generated by a Python script 
> (asdl_c.py, to
>  > be precise). The trick is to make that script generate full PyObjects 
> rather
>  > than the simple C structures that it generates now.
> 
> See the ast-object branch.

Thanks Martin.

>  > The second step is to then modify ast.c to use the new structures. A 
> branch
>  > probably wouldn't help much with initial development (this is a 
> "break the
>  > world, check in when stuff compiles again" kind of change, which is 
> hard to
>  > split amongst multiple people), but I think it would be of benefit when
>  > reviewing the change before moving it back to the trunk.
> 
> Well, there would be a clear two-split right now: one could change
> ast.c, and the other compile.c.

I was focusing too much on the AST production end, and managed to forget that 
compile.c and symtable.c are consumers of the AST, so they'll likely care 
about the change as well ;)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From jeremy at alum.mit.edu  Thu Dec  1 14:11:37 2005
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 1 Dec 2005 08:11:37 -0500
Subject: [Python-Dev] ast-objects branch created
In-Reply-To: <438E31F0.1020900@v.loewis.de>
References: <438E31F0.1020900@v.loewis.de>
Message-ID: <e8bf7a530512010511s12b68cf0xf9b0f04133e608d5@mail.gmail.com>

Martin,

I'm not sure what your intent for this work is, but I'd like to create
a parallel arena branch and compare the results.  I'll start work on
that tomorrow.

Jeremy

On 11/30/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I created
>
> http://svn.python.org/projects/python/branches/ast-objects/
>
> You can convert your repository to that branch with
>
> svn switch svn+ssh://pythondev at svn.python.org/python/branches/ast-objects
>
> in the toplevel directory. In particular, this features
>
> http://svn.python.org/projects/python/branches/ast-objects/Parser/asdl_c.py
> http://svn.python.org/projects/python/branches/ast-objects/Include/Python-ast.h
> http://svn.python.org/projects/python/branches/ast-objects/Python/Python-ast.c
>
> The status is currently this:
> - asdl_c generates a type hierarchy: "Sum" productions have one type
>    per constructor, inheriting from a type for the sum; plain products
>    only have a type for the product.
> - attributes are in the base type, accessible through o->_base.attr;
>    projections of the product types are accessible directly through
>    member names.
> - all projections must be non-NULL. Sequences are represented through
>    potentially empty lists; optional types are potentially represented
>    through Py_None. bool is gone; use Py_True/Py_False. The only
>    primitive type remaining is int (which only occurs in lineno)
> - the types currently have only a constructor, a dealloc function,
>    and an _Check macro.
> - Naming is this: for cross-object-file visible symbols (functions
>    and global variables), a Py_ prefix is used. Otherwise, I use
>    the type name or constructor name directly. There is a #define
>    for the Py_<type>_New function, so you can also write <type>(params).
>    Parameter order for the types is: projections first, then
>    attributes.
> - For compatibility with the current code, the Sum base types
>    also have the _kind enumeration (although that appears not
>    to get initialized right now).
>
> For asdl_c, I see the following things as TODOs:
> - add support for traversing the types from C, through tp_members
>    (read-only). Optionally add support for pickling.
> - add support for garbage collection. I don't expect this to be
>    necessary right now, but will be if the API is exposed, and it
>    doesn't cost much.
>
> The bigger chunk of necessary changes is in using these, starting
> with ast.c.
>
> Feel free to commit any changes to that branch that you consider
> helpful. To avoid duplicated work, posting a note here might
> also help.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>

From nnorwitz at gmail.com  Thu Dec  1 19:45:11 2005
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 1 Dec 2005 10:45:11 -0800
Subject: [Python-Dev] PyAST_FromNode returning PyTypeObject* ?
In-Reply-To: <438EB2DA.5060309@v.loewis.de>
References: <438EB2DA.5060309@v.loewis.de>
Message-ID: <ee2a432c0512011045w14b57eb6w172092ad8f379080@mail.gmail.com>

On 12/1/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Neal,
>
> Why did you suggest that PyAST_FromNode returns PyTypeObject*?
> I can't see why type objects are much used in the AST, unless
> I'm missing something essential.

It was late and I was trying to make progress.  Assume it was a mistake.
It doesn't seem to make much sense based on the name.  I think I was
replacing all mods with PyTypeObject, but since they are probably
lists, PyObject
would be correct.

n

From nnorwitz at gmail.com  Thu Dec  1 19:46:38 2005
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 1 Dec 2005 10:46:38 -0800
Subject: [Python-Dev] ast-objects branch created
In-Reply-To: <e8bf7a530512010511s12b68cf0xf9b0f04133e608d5@mail.gmail.com>
References: <438E31F0.1020900@v.loewis.de>
	<e8bf7a530512010511s12b68cf0xf9b0f04133e608d5@mail.gmail.com>
Message-ID: <ee2a432c0512011046i233cf363jd98c05230a95a381@mail.gmail.com>

On 12/1/05, Jeremy Hylton <jeremy at alum.mit.edu> wrote:
> Martin,
>
> I'm not sure what your intent for this work is, but I'd like to create
> a parallel arena branch and compare the results.  I'll start work on
> that tomorrow.

I think this is a good thing.  It will be much easier to compare
implementations if we have some substantial code reflecting each
technique.

n

From martin at v.loewis.de  Thu Dec  1 23:38:17 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 01 Dec 2005 23:38:17 +0100
Subject: [Python-Dev] ast-objects branch created
In-Reply-To: <e8bf7a530512010511s12b68cf0xf9b0f04133e608d5@mail.gmail.com>
References: <438E31F0.1020900@v.loewis.de>
	<e8bf7a530512010511s12b68cf0xf9b0f04133e608d5@mail.gmail.com>
Message-ID: <438F7B59.6060206@v.loewis.de>

Jeremy Hylton wrote:
> I'm not sure what your intent for this work is, but I'd like to create
> a parallel arena branch and compare the results.  I'll start work on
> that tomorrow.

I certainly want the PyObject* branch to become "life" at some time;
I think this is the way to go, and that an arena approach is
fundamentally flawed.

That said: go ahead and create a branch. This is one of the things
that subversion makes convenient, and it allows people to actually
judge the results when we are done. I'm personally not worried about
the duplicated work: if we actually carry out the experiment of
multiple alternative (or perhaps supplementing) implementations,
we have much better grounds to pick the approach for the mainline.

Regards,
Martin

From trentm at ActiveState.com  Fri Dec  2 03:55:57 2005
From: trentm at ActiveState.com (Trent Mick)
Date: Thu, 1 Dec 2005 18:55:57 -0800
Subject: [Python-Dev] Plea to distribute debugging lib
In-Reply-To: <irv86syb.fsf@python.net>
References: <uek5wdvjw.fsf@boost-consulting.com>
	<20051104202824.GA19678@discworld.dyndns.org>
	<ca471dc20511041233i25b3c33wbfa3e3ad4a356702@mail.gmail.com>
	<irv86syb.fsf@python.net>
Message-ID: <20051202025557.GA22377@ActiveState.com>

[Thomas Heller wrote]
> Anyway, AFAIK, the activestate distribution contains Python debug dlls.

[Er, a month late, but I was in flitting around Australia at the time. :)]

Yes, as a separate download.

    ftp://ftp.activestate.com/ActivePython/etc/
        ActivePython-<version>-win32-ix86-debug.zip

And those should be binary compatible with the equivalent python.org
installs as well. Note that the simple "install.py" script in those
packages bails if the Python installation isn't ActivePython, but I
could easily remove that if you think that would be useful for your
users.

Trent

-- 
Trent Mick
TrentM at ActiveState.com

From martin at v.loewis.de  Fri Dec  2 08:38:21 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 02 Dec 2005 08:38:21 +0100
Subject: [Python-Dev] Subject lines of commit email
Message-ID: <438FF9ED.9020200@v.loewis.de>

Due to popular demand, we have changed the format
of the subject lines that mailer.py will send out
on commit, to include the file names of the modified
files (not just the directory).

As the original mailer.py couldn't do that, David
Goodger contributed the necessary changes.

If people feel strongly that this should be reverted,
we will need to take a poll, as people also feel strongly
that file names in the subject line are essential.

Regards,
Martin

From dave at boost-consulting.com  Fri Dec  2 14:53:17 2005
From: dave at boost-consulting.com (David Abrahams)
Date: Fri, 02 Dec 2005 08:53:17 -0500
Subject: [Python-Dev] Plea to distribute debugging lib
In-Reply-To: <20051202025557.GA22377@ActiveState.com> (Trent Mick's message of
	"Thu, 1 Dec 2005 18:55:57 -0800")
References: <uek5wdvjw.fsf@boost-consulting.com>
	<20051104202824.GA19678@discworld.dyndns.org>
	<ca471dc20511041233i25b3c33wbfa3e3ad4a356702@mail.gmail.com>
	<irv86syb.fsf@python.net> <20051202025557.GA22377@ActiveState.com>
Message-ID: <umzjjtxki.fsf@boost-consulting.com>

Trent Mick <trentm at ActiveState.com> writes:

> [Thomas Heller wrote]
>> Anyway, AFAIK, the activestate distribution contains Python debug dlls.
>
> [Er, a month late, but I was in flitting around Australia at the time. :)]
>
> Yes, as a separate download.
>
>     ftp://ftp.activestate.com/ActivePython/etc/
>         ActivePython-<version>-win32-ix86-debug.zip
>
> And those should be binary compatible with the equivalent python.org
> installs as well. Note that the simple "install.py" script in those
> packages bails if the Python installation isn't ActivePython, but I
> could easily remove that if you think that would be useful for your
> users.

Yes, please!

Would Python.org be willing to post links to the Activestate package?
That would help, too.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com

From vinay_sajip at yahoo.co.uk  Fri Dec  2 20:40:09 2005
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 2 Dec 2005 19:40:09 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Proposed_additional_keyword_argument_in_lo?=
	=?utf-8?q?gging=09calls?=
References: <001a01c5ef77$d7682300$0200a8c0@alpha>
	<ca471dc20511281213i7aa48897qb4fd10d89fbae5dd@mail.gmail.com>
	<loom.20051129T234808-309@post.gmane.org> <64qaik1k.fsf@python.net>
Message-ID: <loom.20051202T203548-571@post.gmane.org>

Thomas Heller <theller <at> python.net> writes:
> > by the logging package itself:
> >
> > name            Name of the logger
> > levelno         Numeric logging level for the message (DEBUG, INFO,
> >                 WARNING, ERROR, CRITICAL)
> [and so on].
> 
> Shouldn't this list be documented?  Or is it?

The list of values which can be used in format strings is:

http://docs.python.org/lib/node357.html

The list of internal values is not (e.g. msg, args, exc_text) and I suppose the 
documentation could be updated to include them, but they're implementation 
details so I don't really want to encourage changing them. If you need to, it's 
easy enough to pick up the info from the source code for LogRecord.__init__(), 
which does all the setup.

Regards,

Vinay Sajip




From vinay_sajip at yahoo.co.uk  Fri Dec  2 20:45:52 2005
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 2 Dec 2005 19:45:52 +0000 (UTC)
Subject: [Python-Dev] Proposed additional keyword argument in logging
	calls
References: <fb6fbf560511300939i2e31eb16la2a3fb15bb688053@mail.gmail.com>
Message-ID: <loom.20051202T204032-706@post.gmane.org>

Jim Jewett <jimjjewett <at> gmail.com> writes:

> 
> > I couldn't think of a good reason why it should be possible to overwrite 
these
> > values with values from a user-supplied dictionary, other than to spoof log
> > entries in some way. The intention is to stop a user accidentally 
overwriting
> > one of the above attributes.
> 
> This makes sense, but is it worth the time to check on each logging call?

Perhaps not - it depends on the performance criticality of your application. If 
you don't supply a dictionary, the impact is minimal. If you need better 
performance you can roll your own LogRecord subclass and do everything inline, 
as Skip has mentioned. And if one wants this functionality, and don't check on 
every call, when would be a good time to check?

Regards,


Vinay Sajip


From vinay_sajip at yahoo.co.uk  Fri Dec  2 20:54:41 2005
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 2 Dec 2005 19:54:41 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Proposed_additional_keyword_argument_in_lo?=
	=?utf-8?q?gging=09calls?=
References: <001a01c5ef77$d7682300$0200a8c0@alpha>
	<ca471dc20511281213i7aa48897qb4fd10d89fbae5dd@mail.gmail.com>
	<loom.20051129T234808-309@post.gmane.org>
	<17292.59915.267228.293830@montanaro.dyndns.org>
Message-ID: <loom.20051202T204614-397@post.gmane.org>

> If the user doesn't need those values and can provide cheap substitutes,
> perhaps their computation can be avoided.  I did that recently by inlining
> only the parts of logging.LogRecord.__init__ in a subclass and avoided
> calling logging.LogRecord.__init__ altogether.  It generated lots of
> instance variables we never use and just slowed things down.

Well, that approach is fine if you need maximal performance. If those values 
set in LogRecord.__init__ can be computed more cheaply than they are at 
present, please give an idea as to where speedups can be applied, or even 
supply a patch ;-) If you don't need many of the values set by 
LogRecord.__init__, then your approach is fine (until someone decides they want 
something which is documented and tries to access it via a specific format 
string - which could be done, in the general case, via configuration changes).

The use case I'm supporting with this is one where some/most/all of the default 
attributes are wanted, and also additional user-defined ones in some special 
cases. With the proposed scheme the extra value is passed to LogRecord 
constuction code so it could be used by your custom LogRecord initialiser with 
the same benefit as in the general case (I think).

Regards,

Vinay Sajip


From nirs at freeshell.org  Fri Dec  2 22:56:15 2005
From: nirs at freeshell.org (Nir Soffer)
Date: Fri, 2 Dec 2005 23:56:15 +0200
Subject: [Python-Dev] os.normpath may change the meaning of the path if it
	contains symbolic links?
In-Reply-To: <loom.20051202T204614-397@post.gmane.org>
References: <001a01c5ef77$d7682300$0200a8c0@alpha>
	<ca471dc20511281213i7aa48897qb4fd10d89fbae5dd@mail.gmail.com>
	<loom.20051129T234808-309@post.gmane.org>
	<17292.59915.267228.293830@montanaro.dyndns.org>
	<loom.20051202T204614-397@post.gmane.org>
Message-ID: <75974B66-637E-11DA-B2FF-000A95B45AA0@freeshell.org>

I'm working on  
<https://sourceforge.net/tracker/index.php? 
func=detail&aid=1239890&group_id=5470&atid=305470>

My patch uses os.path.normpath to simplify the path sent to makedirs as  
first step. This eliminates the need to check for os.currdir or other  
special case, because normpath already handle all those ugly spacial  
cases (and hopefully tested). And of course eliminate possible  
pointless system calls.

For example (on Mac OS X):

 >>> os.path.normpath('a/./////b/c')
'a/b/c'
 >>> os.path.normpath('a/b/c/.')
'a/b/c'
 >>> os.path.normpath('./a/b')
'a/b'
 >>> os.path.normpath('a/b/////')
'a/b'

However, I found this alarming note in the docs:

normpath(path)
	...
	"It should be understood that this may change the meaning of the path  
if it contains symbolic links!"

The function docstring does not contain this note:

	"""Normalize path, eliminating double slashes, etc."""

And finally, there is no test for this symbolic link problem.

Anyone has a clue about this?


Best Regards,

Nir Soffer


From fredrik at pythonware.com  Fri Dec  2 23:34:34 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri, 2 Dec 2005 23:34:34 +0100
Subject: [Python-Dev] os.normpath may change the meaning of the path if
	itcontains symbolic links?
References: <001a01c5ef77$d7682300$0200a8c0@alpha><ca471dc20511281213i7aa48897qb4fd10d89fbae5dd@mail.gmail.com><loom.20051129T234808-309@post.gmane.org><17292.59915.267228.293830@montanaro.dyndns.org><loom.20051202T204614-397@post.gmane.org>
	<75974B66-637E-11DA-B2FF-000A95B45AA0@freeshell.org>
Message-ID: <dmqi5c$cav$1@sea.gmane.org>

Nir Soffer wrote:

> However, I found this alarming note in the docs:
>
> normpath(path)
> ...
> "It should be understood that this may change the meaning of the path
> if it contains symbolic links!"
>
> The function docstring does not contain this note:
>
> """Normalize path, eliminating double slashes, etc."""
>
> And finally, there is no test for this symbolic link problem.
>
> Anyone has a clue about this?

if BAR is a symbolic link, FOO/BAR/../DIR isn't necessarily the same
thing as FOO/DIR.

a simple example:

    $ ln -s /etc etc
    $ ls -ld etc
    lrwxrwxrwx    1 fredrik  fredrik         4 Dec  2 23:22 etc -> /etc
    $ etc/../usr/local/bin/python2.4
    Python 2.4.1 (#1, Sep 12 2005, 19:35:02)
    ...
    >>> import os
    >>> os.path.normpath("etc/../usr/local/bin/python2.4")
    'usr/local/bin/python2.4'
    >>> <control-D>
    $ usr/local/bin/python2.4
    -bash: usr/local/bin/python2.4: No such file or directory

</F> 




From skip at pobox.com  Sat Dec  3 00:02:39 2005
From: skip at pobox.com (skip@pobox.com)
Date: Fri, 2 Dec 2005 17:02:39 -0600
Subject: [Python-Dev]
 =?utf-8?q?Proposed_additional_keyword_argument_in_lo?=
 =?utf-8?q?gging=09calls?=
In-Reply-To: <loom.20051202T204614-397@post.gmane.org>
References: <001a01c5ef77$d7682300$0200a8c0@alpha>
	<ca471dc20511281213i7aa48897qb4fd10d89fbae5dd@mail.gmail.com>
	<loom.20051129T234808-309@post.gmane.org>
	<17292.59915.267228.293830@montanaro.dyndns.org>
	<loom.20051202T204614-397@post.gmane.org>
Message-ID: <17296.53903.484779.85106@montanaro.dyndns.org>


    >> If the user doesn't need those values and can provide cheap
    >> substitutes, perhaps their computation can be avoided.  I did that
    >> recently by inlining only the parts of logging.LogRecord.__init__ in
    >> a subclass and avoided calling logging.LogRecord.__init__ altogether.
    >> It generated lots of instance variables we never use and just slowed
    >> things down.

    Vinay> Well, that approach is fine if you need maximal performance. If
    Vinay> those values set in LogRecord.__init__ can be computed more
    Vinay> cheaply than they are at present, please give an idea as to where
    Vinay> speedups can be applied, or even supply a patch ;-) 

Note the prefix "If the user doesn't need those values".  I don't need much
of the stuff that LogRecord.__init__ computes for each LogRecord
instantiation, so None would be a fine, cheap substitute for me. <wink>

In general we do a lot of logging at work because stuff happens so fast
(we're writing automated trading applications) that you can't tell what's
happening in real-time.  Your only opportunity for analysis is post-mortem.
Consequently, we tend to log a bunch of stuff that is rarely needed, but is
critical to have around when it is needed.  In fact, the folks who write C++
apps here tossed out log4cpp altogether and wrote their own logging package
because log4cpp was too slow.  To give you an idea how much logging we do, I
wrote a CompressedRotatingFileHandler class because our apps log so much
data that we need to compress them to avoid filling up disks.

    Vinay> If you don't need many of the values set by LogRecord.__init__,
    Vinay> then your approach is fine (until someone decides they want
    Vinay> something which is documented and tries to access it via a
    Vinay> specific format string - which could be done, in the general
    Vinay> case, via configuration changes).

This is used within a single app, so I'm not too worried about that.  All we
ever call the log routines with is a caller-formatted string.  There is no
substitution required of the logging package.  Most of the stuff you provide
we simply don't need.  Like I said, we log a lot of stuff.  When our
application's main CPU user is the logging package it's time to look at the
logging package with an eye to cutting out what we don't need.

    Vinay> The use case I'm supporting with this is one where some/most/all
    Vinay> of the default attributes are wanted, 

I would argue the stuff I've thrown out isn't used by most apps.  Here's
what I inlined:

        self.name = name
        self.msg = msg
        self.args = args
        self.levelname = logging.getLevelName(level)
        self.levelno = level
        self.pathname = pathname
        self.filename = os.path.basename(pathname)
        self.module = os.path.splitext(self.filename)[0]
        self.exc_info = exc_info
        self.lineno = lineno

self.created is either set to time.time() (when we run live) or our internal
tick stream clock time (when we do historical runs).

Here's what I avoided setting:

    self.thread - most apps, ours included, are not multithreaded, so
        knowing the current thread id is superfluous

    try/except on self.filename/self.module - not sure why this is necessary
        or what the except clause is catching - I've never seen an exception

    self.relativeCreated - we only use absolute time

    self.process - we never to record the process id as each process
        generates its own log files.

Plus I of course avoided the overhead of the __init__ function call itself.

    Vinay> With the proposed scheme the extra value is passed to LogRecord
    Vinay> constuction code so it could be used by your custom LogRecord
    Vinay> initialiser with the same benefit as in the general case (I
    Vinay> think).

Sure, I'd be happy to use extra dict if it suppressed the superfluous
calculations, e.g.:

    if 'relativeCreated' in extra:
        self.relativeCreated = extra['relativeCreated']
    else:
        self.relativeCreated = (self.created - _startTime) * 1000

Maybe adding all those tests will just slow down the common case though.
Perhaps extra should be used to override *all* logRecord attributes:

    if extra:
        self.__dict__.update(extra)
    else:
        ... do the usual initialization ...

In that case it would be the caller's responsibility to make sure that extra
contains everything they need.  To insure that, a static LogRecord method
could be used to return a default extras dict:

    extras = LogRecord.get_default_extra()

    while True:
        ... compute ...
        ... maybe override some values in extras ...
        logging.info(..., extra=extras)
        ... compute some more ...

Skip

From jepler at unpythonic.net  Sat Dec  3 03:53:22 2005
From: jepler at unpythonic.net (jepler@unpythonic.net)
Date: Fri, 2 Dec 2005 20:53:22 -0600
Subject: [Python-Dev] os.normpath may change the meaning of the path if
	it contains symbolic links?
In-Reply-To: <75974B66-637E-11DA-B2FF-000A95B45AA0@freeshell.org>
References: <001a01c5ef77$d7682300$0200a8c0@alpha>
	<ca471dc20511281213i7aa48897qb4fd10d89fbae5dd@mail.gmail.com>
	<loom.20051129T234808-309@post.gmane.org>
	<17292.59915.267228.293830@montanaro.dyndns.org>
	<loom.20051202T204614-397@post.gmane.org>
	<75974B66-637E-11DA-B2FF-000A95B45AA0@freeshell.org>
Message-ID: <20051203025321.GA803@unpythonic.net>

Consider:
    $ mkdir -p d/d/d
    $ echo 1 > d/d/a
    $ echo 2 > a
    $ ln -s d/d/d x
    $ python -c 'import os; print open(os.path.normpath("x/../a")).read(),'
    2
    $ cat x/../a
    1

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20051202/ded58ff7/attachment.pgp

From aahz at pythoncraft.com  Sun Dec  4 16:44:32 2005
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 4 Dec 2005 07:44:32 -0800
Subject: [Python-Dev] SVN backup?
Message-ID: <20051204154432.GA15771@panix.com>

While cleaning up some old CDs, I discovered that I had received some
backups of the CVS repository.  Should we repeat the exercise for SVN?
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Don't listen to schmucks on USENET when making legal decisions.  Hire
yourself a competent schmuck."  --USENET schmuck (aka Robert Kern)

From martin at v.loewis.de  Sun Dec  4 18:14:12 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 04 Dec 2005 18:14:12 +0100
Subject: [Python-Dev] SVN backup?
In-Reply-To: <20051204154432.GA15771@panix.com>
References: <20051204154432.GA15771@panix.com>
Message-ID: <439323E4.7020507@v.loewis.de>

Aahz wrote:
> While cleaning up some old CDs, I discovered that I had received some
> backups of the CVS repository.  Should we repeat the exercise for SVN?

No *exactly* sure what you are asking: if you mean that you had been
producing regular backups of the CVS tarball, and ask whether you should
do that for subversion also... Certainly, if you can contribute the
resources for that!

The daily snapshot of the repository is at

http://svn.python.org/snapshots/projects-svn-tarball.tar.bz2

Regards,
Martin

From vinay_sajip at yahoo.co.uk  Sun Dec  4 23:39:32 2005
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sun, 4 Dec 2005 22:39:32 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Proposed_additional_keyword_argument_in_lo?=
	=?utf-8?q?gging=09calls?=
References: <001a01c5ef77$d7682300$0200a8c0@alpha>
	<ca471dc20511281213i7aa48897qb4fd10d89fbae5dd@mail.gmail.com>
	<loom.20051129T234808-309@post.gmane.org>
	<17292.59915.267228.293830@montanaro.dyndns.org>
	<loom.20051202T204614-397@post.gmane.org>
	<17296.53903.484779.85106@montanaro.dyndns.org>
Message-ID: <loom.20051204T233034-414@post.gmane.org>

Skip,

Thanks for the detailed post indicating what stuff you find useful, and what
stuff you don't need. It seems that your use case is fairly unusual, so I
completely understand that you have optimized how you use logging. I will
consider how to try to incorporate your feedback without breaking backwards
compatibility, but it will probably not be via the 'extra' mechanism I proposed.
For example, to avoid thread and process computation, I could introduce
module-level variables eg. log_threads and log_processes, both set to true to
maintain backwards compatibility, but settable to false to avoid expensive
thread and process computations where not needed. The 'extra' mechanism will
remain to provide additional diagnostic information where e.g. the same code is
executed by multiple threads and there is a need to distinguish the different
threads in the logging output.

Regards,

Vinay Sajip




From jeremy at alum.mit.edu  Mon Dec  5 06:37:44 2005
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 5 Dec 2005 00:37:44 -0500
Subject: [Python-Dev] ast-objects branch created
In-Reply-To: <438F7B59.6060206@v.loewis.de>
References: <438E31F0.1020900@v.loewis.de>
	<e8bf7a530512010511s12b68cf0xf9b0f04133e608d5@mail.gmail.com>
	<438F7B59.6060206@v.loewis.de>
Message-ID: <e8bf7a530512042137v1b39f275g9887116e439a6ace@mail.gmail.com>

On 12/1/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Jeremy Hylton wrote:
> > I'm not sure what your intent for this work is, but I'd like to create
> > a parallel arena branch and compare the results.  I'll start work on
> > that tomorrow.
>
> I certainly want the PyObject* branch to become "life" at some time;
> I think this is the way to go, and that an arena approach is
> fundamentally flawed.

I have implemented a version of the arena API that handles freeing
memory in ast.c.  It worked out rather like I expected, although I
still haven't thought much about how it would extend to the rest of
the compiler.  It seems like the same approach should apply, although
I think the primary concern was the complexity of memory management in
ast.c.

The way the arena approach works is to free all the AST nodes at the
end of compilation.  This approach isn't all that different than the
one it replaced.  In the trunk, there is a single call to free_mod()
at the end of compilation and it recursively frees all the
sub-objects.  One way to think about the arena changes is to replace a
set of recursive function calls based on the tree structure with a
flat list of all AST nodes created during object creation.  The real
advantage is in the error cases, where all the memory gets freed even
though all the nodes aren't attached to a single tree.

Can you expand on why you think this approach is fundamentally flawed?

> That said: go ahead and create a branch. This is one of the things
> that subversion makes convenient, and it allows people to actually
> judge the results when we are done. I'm personally not worried about
> the duplicated work: if we actually carry out the experiment of
> multiple alternative (or perhaps supplementing) implementations,
> we have much better grounds to pick the approach for the mainline.

Sure does.  It seems like the code generation from the AST description
also makes this kind of experimentation easier.

Jeremy

From martin at v.loewis.de  Mon Dec  5 07:51:09 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 05 Dec 2005 07:51:09 +0100
Subject: [Python-Dev] ast-objects branch created
In-Reply-To: <e8bf7a530512042137v1b39f275g9887116e439a6ace@mail.gmail.com>
References: <438E31F0.1020900@v.loewis.de>	
	<e8bf7a530512010511s12b68cf0xf9b0f04133e608d5@mail.gmail.com>	
	<438F7B59.6060206@v.loewis.de>
	<e8bf7a530512042137v1b39f275g9887116e439a6ace@mail.gmail.com>
Message-ID: <4393E35D.1090504@v.loewis.de>

Jeremy Hylton wrote:
> Can you expand on why you think this approach is fundamentally flawed?

I would expect that you allocate in the process memory that needs to
outlive the arena, e.g. python strings. The fundamental problem is that
the arena deallocation cannot know whether such memory exists, and what
to do with it. So two things may happen:
- you mistakenly allocate long-lived memory from the arena, and then
   discard it when you discard the arena. This gives you dangling
   pointer. The problem here is that at the allocation point, you may
   not know (yet) either whether this is going to survive the arena or
   not.
- you allocate memory outside of the arena to survive it, and then
   something goes wrong, and you deallocate the arena. Yet, the outside
   memory remains garbage.

IOW, there would be no problem if you were *completely* done when
you throw away the arena. This is not the case, though: eventually
you end up with byte code that need to persist.

> Sure does.  It seems like the code generation from the AST description
> also makes this kind of experimentation easier.

Indeed. I wish there was a way to generate ast.c from a transformation
description as well.

Regards,
Martin

From jeremy at alum.mit.edu  Mon Dec  5 14:46:40 2005
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 5 Dec 2005 08:46:40 -0500
Subject: [Python-Dev] ast-objects branch created
In-Reply-To: <4393E35D.1090504@v.loewis.de>
References: <438E31F0.1020900@v.loewis.de>
	<e8bf7a530512010511s12b68cf0xf9b0f04133e608d5@mail.gmail.com>
	<438F7B59.6060206@v.loewis.de>
	<e8bf7a530512042137v1b39f275g9887116e439a6ace@mail.gmail.com>
	<4393E35D.1090504@v.loewis.de>
Message-ID: <e8bf7a530512050546u1bccec07jd53690aae6638cd6@mail.gmail.com>

On 12/5/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Jeremy Hylton wrote:
> I would expect that you allocate in the process memory that needs to
> outlive the arena, e.g. python strings. The fundamental problem is that
> the arena deallocation cannot know whether such memory exists, and what
> to do with it.

I can see that problem occurring with an all-or-nothing solution, but
not if you have the freedom to allocate from an arena or from some
other mechanism.  If there are multiple ways to allocate memory, there
is some increased programming burden (you have to remember how each
pointer was allocated) but you gain flexibility.  The ast-arena branch
allocates most memory from an arena, but allocates identifiers on the
regular heap as PyObjects.  It does keep a list of these PyObjects so
that it can DECREF them later.

> IOW, there would be no problem if you were *completely* done when
> you throw away the arena. This is not the case, though: eventually
> you end up with byte code that need to persist.

Right.  The bytecode can't be allocated from the arena, but the AST
can.  The AST is an internal abstraction.

> > Sure does.  It seems like the code generation from the AST description
> > also makes this kind of experimentation easier.
>
> Indeed. I wish there was a way to generate ast.c from a transformation
> description as well.

I'm sure there's a way to generate a parser from the the description,
but that seemed like a bigger project than I wanted to tackle.  GIven
how long it took to finish the AST without a new parser, I think it
was a wise choice :-).

Jeremy

From skip at pobox.com  Mon Dec  5 15:43:59 2005
From: skip at pobox.com (skip@pobox.com)
Date: Mon, 5 Dec 2005 08:43:59 -0600
Subject: [Python-Dev] svn problem - can't get log info for a specific
	revision
Message-ID: <17300.21039.665687.816296@montanaro.dyndns.org>

I'm investigating some problems we're having at work with SIGUSR1 handling.
Our apps tend to see KeyboardInterrupt when sent SIGUSR1 signals.  We're
still running 2.3.4 but are in the process of installing/testing 2.4.2.

A couple "find ... | xargs egrep -i usr1" commands in the 2.3 and 2.4 trees
showed me that between the two a test was added to test_threadsignals.py,
apparently in svn revision 36760:

    % svn blame Lib/test/test_threadsignals.py
    ...
     36760        mwh
     36760        mwh def registerSignals((for_usr1, for_usr2, for_alrm)):
     36760        mwh     usr1 = signal.signal(signal.SIGUSR1, for_usr1)
     36760        mwh     usr2 = signal.signal(signal.SIGUSR2, for_usr2)
     36760        mwh     alrm = signal.signal(signal.SIGALRM, for_alrm)
     36760        mwh     return usr1, usr2, alrm
     36760        mwh
     36760        mwh
     41566 fred.drake # The signal handler. Just note that the signal occurred and
     36760        mwh # from who.
     36760        mwh def handle_signals(sig,frame):
     36792    tim_one     signal_blackboard[sig]['tripped'] += 1
     36760        mwh     signal_blackboard[sig]['tripped_by'] = thread.get_ident()
     36760        mwh
     36760        mwh # a function that will be spawned as a separate thread.
     36760        mwh def send_signals():
     36760        mwh     os.kill(process_pid, signal.SIGUSR1)
     36760        mwh     os.kill(process_pid, signal.SIGUSR2)
     36760        mwh     signalled_all.release()
     36760        mwh
    ...

Wanting to investigate that further, I checked the developer's FAQ and found
this command to view the checkin history for that rev:

    svn log --verbose -r 36760

When I run it I get this error:

    svn: REPORT request failed on '/projects/!svn/bc/36760/python/branches/release24-maint'
    svn: '/projects/!svn/bc/36760/python/branches/release24-maint' path not found

What is that error telling me?  I ran the "svn log" command in the same
directory as "svn blame" and "svn up", so it's not a basic problem
connecting to the server.  It appears to be something specific to this
command or revision.

Thx,

Skip

From foom at fuhm.net  Mon Dec  5 16:52:01 2005
From: foom at fuhm.net (James Y Knight)
Date: Mon, 5 Dec 2005 10:52:01 -0500
Subject: [Python-Dev] ast-objects branch created
In-Reply-To: <e8bf7a530512050546u1bccec07jd53690aae6638cd6@mail.gmail.com>
References: <438E31F0.1020900@v.loewis.de>
	<e8bf7a530512010511s12b68cf0xf9b0f04133e608d5@mail.gmail.com>
	<438F7B59.6060206@v.loewis.de>
	<e8bf7a530512042137v1b39f275g9887116e439a6ace@mail.gmail.com>
	<4393E35D.1090504@v.loewis.de>
	<e8bf7a530512050546u1bccec07jd53690aae6638cd6@mail.gmail.com>
Message-ID: <07E27726-7D67-48FB-80BB-1D59241EEEDB@fuhm.net>


On Dec 5, 2005, at 8:46 AM, Jeremy Hylton wrote:

> On 12/5/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
>> Jeremy Hylton wrote:
>> I would expect that you allocate in the process memory that needs to
>> outlive the arena, e.g. python strings. The fundamental problem is  
>> that
>> the arena deallocation cannot know whether such memory exists, and  
>> what
>> to do with it.
>>
>
> I can see that problem occurring with an all-or-nothing solution, but
> not if you have the freedom to allocate from an arena or from some
> other mechanism.  If there are multiple ways to allocate memory, there
> is some increased programming burden (you have to remember how each
> pointer was allocated) but you gain flexibility.  The ast-arena branch
> allocates most memory from an arena, but allocates identifiers on the
> regular heap as PyObjects.  It does keep a list of these PyObjects so
> that it can DECREF them later.

ISTM that having to remember which pointers are arena-allocated and  
which are normally-refcounted-allocated removes the major gain that  
an arena method is supposed to bring: resistance to mistakes. I'd  
find having a single way to allocate and track memory easier than  
multiple. Then you just have to follow the single set of best  
practices for memory management, and you're all set. (and with  
PyObjects, the same practices the rest of python uses, another win.)

I'd also like to parrot the concern others have had that if the AST  
nodes are not made of PyObjects, then a mirror hierarchy of PyObject- 
ified AST nodes will have to be created, which seems like quite a  
wasteful duplication. If it is required that there be a collection of  
AST python objects (which I think it is), is there really any good  
reason to make the "real" AST objects not be the _only_ AST objects?  
I've not heard one.

James



From tim.peters at gmail.com  Mon Dec  5 17:22:14 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Mon, 5 Dec 2005 11:22:14 -0500
Subject: [Python-Dev] svn problem - can't get log info for a specific
	revision
In-Reply-To: <17300.21039.665687.816296@montanaro.dyndns.org>
References: <17300.21039.665687.816296@montanaro.dyndns.org>
Message-ID: <1f7befae0512050822n2e85234fvc31a6053a180bf3d@mail.gmail.com>

[skip at pobox.com]
...
> Wanting to investigate that further, I checked the developer's FAQ and found
> this command to view the checkin history for that rev:
>
>    svn log --verbose -r 36760
>
> When I run it I get this error:
>
>    svn: REPORT request failed on '/projects/!svn/bc/36760/python/branches/release24-maint'
>    svn: '/projects/!svn/bc/36760/python/branches/release24-maint' path not found
>
> What is that error telling me?

I'm not sure, and I have no idea how you're getting strings like
"/!svn/" and "/bc/" in your output, but it works fine here if I point
it at the trunk instead:

$ svn log -v -r36760  svn+ssh://svn.python.org/python/trunk
------------------------------------------------------------------------
r36760 | mwh | 2004-08-03 10:37:14 -0400 (Tue, 03 Aug 2004) | 2 lines
Changed paths:
   A /python/trunk/Lib/test/test_threadsignals.py

Argh!  This was meant to be part of patch #960406.

------------------------------------------------------------------------

From jeremy at alum.mit.edu  Mon Dec  5 17:36:14 2005
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 5 Dec 2005 11:36:14 -0500
Subject: [Python-Dev] ast-objects branch created
In-Reply-To: <07E27726-7D67-48FB-80BB-1D59241EEEDB@fuhm.net>
References: <438E31F0.1020900@v.loewis.de>
	<e8bf7a530512010511s12b68cf0xf9b0f04133e608d5@mail.gmail.com>
	<438F7B59.6060206@v.loewis.de>
	<e8bf7a530512042137v1b39f275g9887116e439a6ace@mail.gmail.com>
	<4393E35D.1090504@v.loewis.de>
	<e8bf7a530512050546u1bccec07jd53690aae6638cd6@mail.gmail.com>
	<07E27726-7D67-48FB-80BB-1D59241EEEDB@fuhm.net>
Message-ID: <e8bf7a530512050836k3235ee72j94ecc3204a8811f3@mail.gmail.com>

On 12/5/05, James Y Knight <foom at fuhm.net> wrote:
>
> On Dec 5, 2005, at 8:46 AM, Jeremy Hylton wrote:
> > I can see that problem occurring with an all-or-nothing solution, but
> > not if you have the freedom to allocate from an arena or from some
> > other mechanism.  If there are multiple ways to allocate memory, there
> > is some increased programming burden (you have to remember how each
> > pointer was allocated) but you gain flexibility.  The ast-arena branch
> > allocates most memory from an arena, but allocates identifiers on the
> > regular heap as PyObjects.  It does keep a list of these PyObjects so
> > that it can DECREF them later.
>
> ISTM that having to remember which pointers are arena-allocated and
> which are normally-refcounted-allocated removes the major gain that
> an arena method is supposed to bring: resistance to mistakes. I'd
> find having a single way to allocate and track memory easier than
> multiple. Then you just have to follow the single set of best
> practices for memory management, and you're all set. (and with
> PyObjects, the same practices the rest of python uses, another win.)

It's a question of degree, right?  If you can find a small number of
rules that are easy to understand then you are still likely to avoid
mistakes.  For example, the current ast-arena branch uses two rules: 
All AST nodes are allocated from the arena.  All PyObjects attached to
an AST node (identifiers and constants) are associated with the arena,
i.e. they are DECREFed when it is freed.

> I'd also like to parrot the concern others have had that if the AST
> nodes are not made of PyObjects, then a mirror hierarchy of PyObject-
> ified AST nodes will have to be created, which seems like quite a
> wasteful duplication. If it is required that there be a collection of
> AST python objects (which I think it is), is there really any good
> reason to make the "real" AST objects not be the _only_ AST objects?
> I've not heard one.

The PyObject-ified AST nodes are only needed if user code requests an
AST from the compiler.  That is, if we add a new feature that exposes
AST, we would need AST objects represented in Python code.  I think
this feature would be great to add, but it doesn't seem like a primary
concern for the internal compiler implementation.  There is no need
for PyObject-ified AST objects in the internal compiler.  (I think
this fact is obvious, since the compiler exists but PyObject-ified AST
objects don't.)

The question, then, is the simplest way to provide Python code with
access to the AST objects.  I still think that a set of pure Python
classes to represent the AST nodes is a good approach.  You define a
simple serialization format for ASTs and the serialized AST can be
passed from the interpreter to user code and back.  The user code gets
a mutable tree of AST nodes that it can reserialize for compilation to
bytecode.  This strategy is exactly like the existing parser module.

One advantage of this approach is the AST objects in each language are
simpler to use.  The C AST nodes provide an easy API for C programmers
and the Python AST nodes provide an easy API for Python programmers. 
Put another way, since the AST code is all generated from a high level
description, the implementation doesn't matter at all.  What matters
is the API exposed in each programming language.  If the best API
happens to admit a shared implementation, that's great.  If it
doesn't, no loss.

Jeremy

From skip at pobox.com  Mon Dec  5 17:57:10 2005
From: skip at pobox.com (skip@pobox.com)
Date: Mon, 5 Dec 2005 10:57:10 -0600
Subject: [Python-Dev] svn problem - can't get log info for a specific
 revision
In-Reply-To: <1f7befae0512050822n2e85234fvc31a6053a180bf3d@mail.gmail.com>
References: <17300.21039.665687.816296@montanaro.dyndns.org>
	<1f7befae0512050822n2e85234fvc31a6053a180bf3d@mail.gmail.com>
Message-ID: <17300.29030.890744.137521@montanaro.dyndns.org>

    >> svn log --verbose -r 36760
    >> 
    >> When I run it I get this error:
    >> 
    >> svn: REPORT request failed on '/projects/!svn/bc/36760/python/branches/release24-maint'
    >> svn: '/projects/!svn/bc/36760/python/branches/release24-maint' path not found

    Tim> I'm not sure, and I have no idea how you're getting strings like
    Tim> "/!svn/" and "/bc/" in your output, but it works fine here if I
    Tim> point it at the trunk instead:

    Tim> $ svn log -v -r36760  svn+ssh://svn.python.org/python/trunk
    ...

Okay, I got it.  I was asking for that revision on the release24-maint
branch, but it was actually applied to the trunk.  It shows up now in the
maintenance branch, but still has to be queried for on the trunk.

Skip

From nnorwitz at gmail.com  Mon Dec  5 20:24:08 2005
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 5 Dec 2005 11:24:08 -0800
Subject: [Python-Dev] ast-objects branch created
In-Reply-To: <e8bf7a530512050836k3235ee72j94ecc3204a8811f3@mail.gmail.com>
References: <438E31F0.1020900@v.loewis.de>
	<e8bf7a530512010511s12b68cf0xf9b0f04133e608d5@mail.gmail.com>
	<438F7B59.6060206@v.loewis.de>
	<e8bf7a530512042137v1b39f275g9887116e439a6ace@mail.gmail.com>
	<4393E35D.1090504@v.loewis.de>
	<e8bf7a530512050546u1bccec07jd53690aae6638cd6@mail.gmail.com>
	<07E27726-7D67-48FB-80BB-1D59241EEEDB@fuhm.net>
	<e8bf7a530512050836k3235ee72j94ecc3204a8811f3@mail.gmail.com>
Message-ID: <ee2a432c0512051124n50e1282drf592a2ad54e206db@mail.gmail.com>

On 12/5/05, Jeremy Hylton <jeremy at alum.mit.edu> wrote:
> On 12/5/05, James Y Knight <foom at fuhm.net> wrote:
> >
> > ISTM that having to remember which pointers are arena-allocated and
> > which are normally-refcounted-allocated removes the major gain that
> > an arena method is supposed to bring: resistance to mistakes. I'd
> > find having a single way to allocate and track memory easier than
> > multiple. Then you just have to follow the single set of best
> > practices for memory management, and you're all set. (and with
> > PyObjects, the same practices the rest of python uses, another win.)
>
> It's a question of degree, right?  If you can find a small number of
> rules that are easy to understand then you are still likely to avoid
> mistakes.

This is my understanding of the two approaches from what I've seen so
far (Jeremy or Martin should correct me if I'm wrong).

With current arena impl:
 * need to call PyArena_AddPyObject() for any allocated PyObject
 * need to call PyArena_AddMallocPointer() for any malloc()ed memory
(there are current no manual calls like this, all the calls are in
generated code?)

With the PyObject imp:
 * need to init all PyObjects to NULL
 * need to Py_XDECREF() on exit
 * need to goto error if there is any failure

Both impls have a bit more details, but those are the highlights. 
>From what I've seen of both, the arena is easier to deal with even
though it is different from the rest of python.  There is only one
thing to remember.

I didn't look at the changes much, but from what I saw I think it may
be better to move the arenas off the branch and onto the head now.  It
appears to be much easier to get right since there is virtually no
error handling code in line.  It's all taken care of in a few central
places.

We can then decide between the arenas in the head vs PyObjects.

> > I'd also like to parrot the concern others have had that if the AST
> > nodes are not made of PyObjects, then a mirror hierarchy of PyObject-
> > ified AST nodes will have to be created, which seems like quite a
> > wasteful duplication. If it is required that there be a collection of
> > AST python objects (which I think it is), is there really any good
> > reason to make the "real" AST objects not be the _only_ AST objects?
> > I've not heard one.
>
> The PyObject-ified AST nodes are only needed if user code requests an
> AST from the compiler.  That is, if we add a new feature that exposes
> AST, we would need AST objects represented in Python code.  I think
> this feature would be great to add, but it doesn't seem like a primary
> concern for the internal compiler implementation.

FWIW, I agree with this approach.  I don't care that much about the
internal AST for its own sake.  I want to consume the AST and I only
care about the internals insofar as the result is correct and
maintainable.

So my view of the best approach is one that is easy to get right and
maintain.  That's why I think the arena should be moved to the head
now.  From what I saw it was much easier to get right, it removed a
bunch of code and should be more maintainable.

I will also probably work on the PyObject approach, since if that's
more maintainable I'd prefer that in the end.  I don't know which
approach is best.

I also really like Martin's idea about generating a lot more (all?) of
the manually written Python/ast.c code.  I'd prefer much less C code
to maintain.

n

From bcannon at gmail.com  Mon Dec  5 22:29:38 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Mon, 5 Dec 2005 13:29:38 -0800
Subject: [Python-Dev] ast-objects branch created
In-Reply-To: <ee2a432c0512051124n50e1282drf592a2ad54e206db@mail.gmail.com>
References: <438E31F0.1020900@v.loewis.de>
	<e8bf7a530512010511s12b68cf0xf9b0f04133e608d5@mail.gmail.com>
	<438F7B59.6060206@v.loewis.de>
	<e8bf7a530512042137v1b39f275g9887116e439a6ace@mail.gmail.com>
	<4393E35D.1090504@v.loewis.de>
	<e8bf7a530512050546u1bccec07jd53690aae6638cd6@mail.gmail.com>
	<07E27726-7D67-48FB-80BB-1D59241EEEDB@fuhm.net>
	<e8bf7a530512050836k3235ee72j94ecc3204a8811f3@mail.gmail.com>
	<ee2a432c0512051124n50e1282drf592a2ad54e206db@mail.gmail.com>
Message-ID: <bbaeab100512051329w54c0663du5ba1937b6b718ebe@mail.gmail.com>

On 12/5/05, Neal Norwitz <nnorwitz at gmail.com> wrote:
> On 12/5/05, Jeremy Hylton <jeremy at alum.mit.edu> wrote:
[SNIP]
> I didn't look at the changes much, but from what I saw I think it may
> be better to move the arenas off the branch and onto the head now.  It
> appears to be much easier to get right since there is virtually no
> error handling code in line.  It's all taken care of in a few central
> places.
>
> We can then decide between the arenas in the head vs PyObjects.
>

I am also +1 with merging the arena into the trunk.  The arena
approach compared to the existing solution is a lot easier to use. 
With almost all calls to the arena in the auto-generated constructor
code, one just has to make sure that key places have PyArena_Free() to
free the arena and that errors propagate up to those points.

But, as Neal is suggesting, this should not prevent the PyObject
version from moving forward since it could still turn out to be the
better solution.

> > > I'd also like to parrot the concern others have had that if the AST
> > > nodes are not made of PyObjects, then a mirror hierarchy of PyObject-
> > > ified AST nodes will have to be created, which seems like quite a
> > > wasteful duplication. If it is required that there be a collection of
> > > AST python objects (which I think it is), is there really any good
> > > reason to make the "real" AST objects not be the _only_ AST objects?
> > > I've not heard one.
> >
> > The PyObject-ified AST nodes are only needed if user code requests an
> > AST from the compiler.  That is, if we add a new feature that exposes
> > AST, we would need AST objects represented in Python code.  I think
> > this feature would be great to add, but it doesn't seem like a primary
> > concern for the internal compiler implementation.
>
> FWIW, I agree with this approach.  I don't care that much about the
> internal AST for its own sake.  I want to consume the AST and I only
> care about the internals insofar as the result is correct and
> maintainable.
>

It really comes down to how people expect to use the exposure of the
AST.  If we try to make sure there is no horrible overhead in getting
the AST to Python code and then to the bytecode compiler then it can
be used for optimizations (e.g., the existing peepholer could be
rewritten in Python and just a default transformation that the AST is
passed through).

But if we don't want to make sure that AST access is used for
optimization transformation but more for non-performance critical uses
(e.g., error checking ala PyChecker or refactoring tools) then the
simplest, easiest to maintain solution should win out.

Personally I want the former abilities for academic experimentation
reasons.  I don't think that a bunch of optimizations are suddenly
going to appear out of nowhere for Python code, but I still would like
to be able to experiment with some without having to worry about a
performance penalty for doing so.

Granted, though, if we byte-compiled scripts passed in on the
command-line we would definitely help minimize the performance impact.
 Interpreter input might be a little slower, but then again since it
will be such bite-sized chunks of AST a couple more Python calls
shouldn't be that significant.

Plus I don't know if serialization will be that much slower than
passing the AST itself out since doing a full transformation on an AST
might be extremely more costly than just getting the AST to the Python
code in the first place.

> So my view of the best approach is one that is easy to get right and
> maintain.  That's why I think the arena should be moved to the head
> now.  From what I saw it was much easier to get right, it removed a
> bunch of code and should be more maintainable.
>
> I will also probably work on the PyObject approach, since if that's
> more maintainable I'd prefer that in the end.  I don't know which
> approach is best.
>
> I also really like Martin's idea about generating a lot more (all?) of
> the manually written Python/ast.c code.  I'd prefer much less C code
> to maintain.
>

A new sprint topic for PyCon for Guido to give us a month deadline on
after we have worked on it for three years!  =)

-Brett

From martin at v.loewis.de  Mon Dec  5 23:23:38 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 05 Dec 2005 23:23:38 +0100
Subject: [Python-Dev] svn problem - can't get log info for a specific
 revision
In-Reply-To: <17300.29030.890744.137521@montanaro.dyndns.org>
References: <17300.21039.665687.816296@montanaro.dyndns.org>	<1f7befae0512050822n2e85234fvc31a6053a180bf3d@mail.gmail.com>
	<17300.29030.890744.137521@montanaro.dyndns.org>
Message-ID: <4394BDEA.4040409@v.loewis.de>

skip at pobox.com wrote:
> Okay, I got it.  I was asking for that revision on the release24-maint
> branch, but it was actually applied to the trunk.  It shows up now in the
> maintenance branch, but still has to be queried for on the trunk.

My subversion (1.2.3) gives me a slightly more legible output:

svn: File not found: revision 36760, path '/python/branches/release24-maint'

It complains that, in revision 36760, there was no release24-main
directory, so it can't know what the history is. I would consider it
a bug: it could know that release24-maint was a copy of trunk
which was made after 36760, so it could conclude I meant to
perform the same operation on trunk, then.

Anyway, to get this location-independent (and even without
a working copy), do

svn log http://svn.python.org/projects --verbose -r 36760

This works because revisions are repository revisions, so
the version number uniquely identifies the change in the
repository. Of course, this also builds on the knowledge
that http://svn.python.org/projects is the same as
svn+ssh://pythondev at svn.python.org (but more easy to type).

Regards,
Martin

From jjl at pobox.com  Tue Dec  6 00:24:04 2005
From: jjl at pobox.com (John J Lee)
Date: Mon, 5 Dec 2005 23:24:04 +0000 (UTC)
Subject: [Python-Dev] Patch reviews & request for patch review
Message-ID: <Pine.LNX.4.58.0512042309580.6221@alice>

Hi

I attended the bug day on Sunday and reviewed six bugs/patches (1212287,
1215184, 1115886, 1372650, 1216942, 878275).  So, I'm hoping one of those
nice people who offered 'review 5 get 1 free' might look at a patch of
mine.  Test, documentation, and explanatory comments in the tracker are
all there:

http://python.org/sf/1157027

"cookielib mis-handles RFC 2109 cookies in Netscape mode"

(It's an old SF patch tracker ID, but I have uploaded a new patch for
Python 2.5 since the old patch was not applied in 2.4.1 / 2.4.2.)



There's another patch I uploaded whose resolution was agreed upon back in
March, but the simple patch (including added test) never got applied:

http://python.org/sf/1117398

"cookielib LWPCookieJar and MozillaCookieJar exceptions"


Thanks in advance to anybody who has time to look at these,


John

From martin at v.loewis.de  Tue Dec  6 00:51:44 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 06 Dec 2005 00:51:44 +0100
Subject: [Python-Dev] svn problem - can't get log info for a specific
 revision
In-Reply-To: <4394BDEA.4040409@v.loewis.de>
References: <17300.21039.665687.816296@montanaro.dyndns.org>	<1f7befae0512050822n2e85234fvc31a6053a180bf3d@mail.gmail.com>	<17300.29030.890744.137521@montanaro.dyndns.org>
	<4394BDEA.4040409@v.loewis.de>
Message-ID: <4394D290.3060003@v.loewis.de>

Martin v. L?wis wrote:
> It complains that, in revision 36760, there was no release24-main
> directory, so it can't know what the history is. I would consider it
> a bug: it could know that release24-maint was a copy of trunk
> which was made after 36760, so it could conclude I meant to
> perform the same operation on trunk, then.

I just asked on the #svn irc channel, and was told that it indeed
should *not* follow copies when I do "svn log -r revno", instead,
it should give me information on the path as it looked in revno
(so in this case, it should tell the path didn't exist).

Sometimes, you want to know "how did foo look in revision revno,
even if it was called bar at that time". To do that, you refer
to foo at revno, and then can add a revision of that object.
They call this syntax "peg revision".

So to do what you want, you should be able to say

svn log -r 36760 . at HEAD

(i.e. find out what . as it is in HEAD was in revision
  36760).

Unfortunately, svn log does not support peg revisions,
so I filed a bug report at

http://subversion.tigris.org/issues/show_bug.cgi?id=2460

To see an example where peg revisions do work, do

martin at mira:~/work/py2.4$ LANG=C svn info -r 36760 . at HEAD
Path: trunk
URL: svn+ssh://pythondev at svn.python.org/python/trunk
Repository Root: svn+ssh://pythondev at svn.python.org
Repository UUID: 6015fed2-1504-0410-9fe1-9d1591cc4771
Revision: 36760
Node Kind: directory
Last Changed Author: mwh
Last Changed Rev: 36760
Last Changed Date: 2004-08-03 16:37:14 +0200 (Tue, 03 Aug 2004)

(But then, this is also what you get if you do
  "svn info -r 36760")

Regards,
Martin

P.S. Instead of HEAD, you would normally use BASE: HEAD
is the latest revision in the repository, whereas BASE
is the one that your working copy is based on.

From musicdev at gmail.com  Tue Dec  6 03:02:54 2005
From: musicdev at gmail.com (musicdev)
Date: Mon, 05 Dec 2005 21:02:54 -0500
Subject: [Python-Dev] Dynamic Link Library
Message-ID: <4394F14E.3030904@gmail.com>

Hi everyone,

Apologies if my question is a bit novice-ish. I was wondering if there 
was a way of creating a Dynamic Link Library with Python.

Please provide any answers you may have.

Warmest Regards,

musicdev

From martin at v.loewis.de  Tue Dec  6 03:09:47 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 06 Dec 2005 03:09:47 +0100
Subject: [Python-Dev] Dynamic Link Library
In-Reply-To: <4394F14E.3030904@gmail.com>
References: <4394F14E.3030904@gmail.com>
Message-ID: <4394F2EB.7020006@v.loewis.de>

musicdev wrote:
> Apologies if my question is a bit novice-ish. I was wondering if there 
> was a way of creating a Dynamic Link Library with Python.
> 
> Please provide any answers you may have.

Please don't use python-dev for such questions; this is a list for the
development of Python, not the development with Python. Use the
general python-list at python.org (news:comp.lang.python) for general
question.

Regards,
Martin

From skip at pobox.com  Tue Dec  6 05:26:05 2005
From: skip at pobox.com (skip@pobox.com)
Date: Mon, 5 Dec 2005 22:26:05 -0600
Subject: [Python-Dev] Broader iterable support for xmlrpclib
Message-ID: <17301.4829.921367.703349@montanaro.dyndns.org>


During the bug day Andrew Kuchling (I think) mentioned that he was working
on some xmlrpclib issues.  I then broached an idea I had a week or so ago to
allow sets to be marshalled as XML-RPC arrays.  This was met with some head
scratching by those present.  Forging ahead nonetheless, I then proposed the
even wackier idea to simply allow all currently unsupported iterables (sets
and arrays seem the most obvious candidates to me) to be marshalled as
lists.  After a couple messages back and forth and more head scratching by
those in attendance I offered to whip up a patch and rationale.  For that,
see here:

    http://python.org/sf/1374063

Skip

From ncoghlan at gmail.com  Tue Dec  6 12:44:12 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 06 Dec 2005 21:44:12 +1000
Subject: [Python-Dev] ast-objects branch created
In-Reply-To: <ee2a432c0512051124n50e1282drf592a2ad54e206db@mail.gmail.com>
References: <438E31F0.1020900@v.loewis.de>	<e8bf7a530512010511s12b68cf0xf9b0f04133e608d5@mail.gmail.com>	<438F7B59.6060206@v.loewis.de>	<e8bf7a530512042137v1b39f275g9887116e439a6ace@mail.gmail.com>	<4393E35D.1090504@v.loewis.de>	<e8bf7a530512050546u1bccec07jd53690aae6638cd6@mail.gmail.com>	<07E27726-7D67-48FB-80BB-1D59241EEEDB@fuhm.net>	<e8bf7a530512050836k3235ee72j94ecc3204a8811f3@mail.gmail.com>
	<ee2a432c0512051124n50e1282drf592a2ad54e206db@mail.gmail.com>
Message-ID: <4395798C.9010103@gmail.com>

Neal Norwitz wrote:
> This is my understanding of the two approaches from what I've seen so
> far (Jeremy or Martin should correct me if I'm wrong).
> 
> With current arena impl:
>  * need to call PyArena_AddPyObject() for any allocated PyObject
>  * need to call PyArena_AddMallocPointer() for any malloc()ed memory
> (there are current no manual calls like this, all the calls are in
> generated code?)
> 
> With the PyObject imp:
>  * need to init all PyObjects to NULL
>  * need to Py_XDECREF() on exit
>  * need to goto error if there is any failure
> 
> Both impls have a bit more details, but those are the highlights. 
>>From what I've seen of both, the arena is easier to deal with even
> though it is different from the rest of python.  There is only one
> thing to remember.

As Fredrik pointed out a while back, the PyObject approach doesn't *have* to 
involve manual decref operations - PyObject's come with a ready made arena 
structure, in the form of PyList.

However, whether the automatic management is done with a list or with Jeremy's 
arena structure, the style is still different from most of CPython, and either 
way there's going to be a small learning curve associated with getting used to it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From guido at python.org  Tue Dec  6 16:47:06 2005
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Dec 2005 07:47:06 -0800
Subject: [Python-Dev] [Python-checkins] commit of r41586 - in
	python/trunk: Lib/SimpleXMLRPCServer.py Misc/NEWS
In-Reply-To: <20051204153705.0D91F1E400B@bag.python.org>
References: <20051204153705.0D91F1E400B@bag.python.org>
Message-ID: <ca471dc20512060747t4fe10771ic26ab4e128d48b44@mail.gmail.com>

Happened to see this commit. What's the magic about 10MB? Is there any
understanding of what causes it to fail? What is the failure mode?
Could it just be fragmentation causing the malloc or realloc to fail?
Should we perhaps use a more conservative buffer size, e.g. 1MB or
even 8K (the international standard for I/O buffering :-)?

--Guido

On 12/4/05, andrew.kuchling <python-checkins at python.org> wrote:
> Author: andrew.kuchling
> Date: Sun Dec  4 16:36:57 2005
> New Revision: 41586
>
> Modified:
>    python/trunk/Lib/SimpleXMLRPCServer.py
>    python/trunk/Misc/NEWS
> Log:
> [Bug #792570] Under Windows, socket.read() seems to run into trouble when
> asked to read tens of megabytes of data.  On my Mac, it hits MemoryErrors
> when reading around 15Mb in one chunk.  The fix is to read the body in several
> parts, not as one big piece.
>
> It would be nice to fix the underlying socket.read() problem, too.
>
> 2.4 bugfix candidate.
>
>
> Modified: python/trunk/Lib/SimpleXMLRPCServer.py
> ==============================================================================
> --- python/trunk/Lib/SimpleXMLRPCServer.py      (original)
> +++ python/trunk/Lib/SimpleXMLRPCServer.py      Sun Dec  4 16:36:57 2005
> @@ -422,8 +422,19 @@
>          """
>
>          try:
> -            # get arguments
> -            data = self.rfile.read(int(self.headers["content-length"]))
> +            # Get arguments by reading body of request.
> +            # We read this in chunks to avoid straining
> +            # socket.read(); around the 10 or 15Mb mark, some platforms
> +            # begin to have problems (bug #792570).
> +            max_chunk_size = 10*1024*1024
> +            size_remaining = int(self.headers["content-length"])
> +            L = []
> +            while size_remaining:
> +                chunk_size = min(size_remaining, max_chunk_size)
> +                L.append(self.rfile.read(chunk_size))
> +                size_remaining -= len(L[-1])
> +            data = ''.join(L)
> +
>              # In previous versions of SimpleXMLRPCServer, _dispatch
>              # could be overridden in this class, instead of in
>              # SimpleXMLRPCDispatcher. To maintain backwards compatibility,
>
> Modified: python/trunk/Misc/NEWS
> ==============================================================================
> --- python/trunk/Misc/NEWS      (original)
> +++ python/trunk/Misc/NEWS      Sun Dec  4 16:36:57 2005
> @@ -451,6 +451,9 @@
>  - Bug #1222790: in SimpleXMLRPCServer, set the reuse-address and close-on-exec
>    flags on the HTTP listening socket.
>
> +- Bug #792570: SimpleXMLRPCServer had problems if the request grew too large.
> +  Fixed by reading the HTTP body in chunks instead of one big socket.read().
> +
>  - Bug #1110478: Revert os.environ.update to do putenv again.
>
>  - Bug #1103844: fix distutils.install.dump_dirs() with negated options.
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>


--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mal at egenix.com  Tue Dec  6 17:01:12 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 06 Dec 2005 17:01:12 +0100
Subject: [Python-Dev] [Python-checkins] commit of r41586 - in
 python/trunk:	Lib/SimpleXMLRPCServer.py Misc/NEWS
In-Reply-To: <ca471dc20512060747t4fe10771ic26ab4e128d48b44@mail.gmail.com>
References: <20051204153705.0D91F1E400B@bag.python.org>
	<ca471dc20512060747t4fe10771ic26ab4e128d48b44@mail.gmail.com>
Message-ID: <4395B5C8.4070406@egenix.com>

Guido van Rossum wrote:
> Happened to see this commit. What's the magic about 10MB? Is there any
> understanding of what causes it to fail? What is the failure mode?
> Could it just be fragmentation causing the malloc or realloc to fail?
> Should we perhaps use a more conservative buffer size, e.g. 1MB or
> even 8K (the international standard for I/O buffering :-)?

Just as data point: I've been hitting problems much earlier than
with 10MB (unrelated to SimpleXMLRPCServer.py, this is experience
from doing plain socket communication). Even 65535 is too much for
some platforms (AIX at the time). Things got stable at around 64000
bytes.

> --Guido
> 
> On 12/4/05, andrew.kuchling <python-checkins at python.org> wrote:
> 
>>Author: andrew.kuchling
>>Date: Sun Dec  4 16:36:57 2005
>>New Revision: 41586
>>
>>Modified:
>>   python/trunk/Lib/SimpleXMLRPCServer.py
>>   python/trunk/Misc/NEWS
>>Log:
>>[Bug #792570] Under Windows, socket.read() seems to run into trouble when
>>asked to read tens of megabytes of data.  On my Mac, it hits MemoryErrors
>>when reading around 15Mb in one chunk.  The fix is to read the body in several
>>parts, not as one big piece.
>>
>>It would be nice to fix the underlying socket.read() problem, too.
>>
>>2.4 bugfix candidate.
>>
>>
>>Modified: python/trunk/Lib/SimpleXMLRPCServer.py
>>==============================================================================
>>--- python/trunk/Lib/SimpleXMLRPCServer.py      (original)
>>+++ python/trunk/Lib/SimpleXMLRPCServer.py      Sun Dec  4 16:36:57 2005
>>@@ -422,8 +422,19 @@
>>         """
>>
>>         try:
>>-            # get arguments
>>-            data = self.rfile.read(int(self.headers["content-length"]))
>>+            # Get arguments by reading body of request.
>>+            # We read this in chunks to avoid straining
>>+            # socket.read(); around the 10 or 15Mb mark, some platforms
>>+            # begin to have problems (bug #792570).
>>+            max_chunk_size = 10*1024*1024
>>+            size_remaining = int(self.headers["content-length"])
>>+            L = []
>>+            while size_remaining:
>>+                chunk_size = min(size_remaining, max_chunk_size)
>>+                L.append(self.rfile.read(chunk_size))
>>+                size_remaining -= len(L[-1])
>>+            data = ''.join(L)
>>+
>>             # In previous versions of SimpleXMLRPCServer, _dispatch
>>             # could be overridden in this class, instead of in
>>             # SimpleXMLRPCDispatcher. To maintain backwards compatibility,
>>
>>Modified: python/trunk/Misc/NEWS
>>==============================================================================
>>--- python/trunk/Misc/NEWS      (original)
>>+++ python/trunk/Misc/NEWS      Sun Dec  4 16:36:57 2005
>>@@ -451,6 +451,9 @@
>> - Bug #1222790: in SimpleXMLRPCServer, set the reuse-address and close-on-exec
>>   flags on the HTTP listening socket.
>>
>>+- Bug #792570: SimpleXMLRPCServer had problems if the request grew too large.
>>+  Fixed by reading the HTTP body in chunks instead of one big socket.read().
>>+
>> - Bug #1110478: Revert os.environ.update to do putenv again.
>>
>> - Bug #1103844: fix distutils.install.dump_dirs() with negated options.
>>_______________________________________________
>>Python-checkins mailing list
>>Python-checkins at python.org
>>http://mail.python.org/mailman/listinfo/python-checkins
>>
> 
> 
> 
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 06 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From amk at amk.ca  Tue Dec  6 17:29:55 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Tue, 6 Dec 2005 11:29:55 -0500
Subject: [Python-Dev] [Python-checkins] commit of r41586 - in
	python/trunk: Lib/SimpleXMLRPCServer.py Misc/NEWS
In-Reply-To: <ca471dc20512060747t4fe10771ic26ab4e128d48b44@mail.gmail.com>
References: <20051204153705.0D91F1E400B@bag.python.org>
	<ca471dc20512060747t4fe10771ic26ab4e128d48b44@mail.gmail.com>
Message-ID: <20051206162955.GA3845@rogue.amk.ca>

On Tue, Dec 06, 2005 at 07:47:06AM -0800, Guido van Rossum wrote:
> Happened to see this commit. What's the magic about 10MB? Is there any
> understanding of what causes it to fail? What is the failure mode?
> Could it just be fragmentation causing the malloc or realloc to fail?
> Should we perhaps use a more conservative buffer size, e.g. 1MB or
> even 8K (the international standard for I/O buffering :-)?

At least on my Mac, it was a malloc error (a message is printed to
stderr by the malloc implementation).  SimpleXMLRPCServer ends up
catching the MemoryError and keeps running.  I don't know why the
malloc fails.

--amk

From amk at amk.ca  Tue Dec  6 17:39:31 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Tue, 6 Dec 2005 11:39:31 -0500
Subject: [Python-Dev] c.l.p post on docs
Message-ID: <20051206163931.GC3845@rogue.amk.ca>

I just posted a lengthy message to comp.lang.python/python-list about
Python's docs; the title is "Documentation suggestions".  A short
summary of the post is: "The RefGuide is hard to read and hard to keep
updated.  Do we need a friendly language description?  If we do that,
should the existing RefGuide be abandoned or maintained?"  See the
post for the full argument.

(Hey, the post just showed up in Google Groups:
<http://groups.google.com/group/comp.lang.python/browse_frm/thread/767470cb3cbc24d5/8875031c6ae10c2d#8875031c6ae10c2d>)

I suggest further discussion on this issue take place in c.l.py. 

--amk


From nas at arctrix.com  Tue Dec  6 21:38:18 2005
From: nas at arctrix.com (Neil Schemenauer)
Date: Tue, 6 Dec 2005 13:38:18 -0700
Subject: [Python-Dev] Documentation about Python's GC,
	python-dev list messages referenced in Modules/gcmodule.c not
	reachable anymore
In-Reply-To: <52D00EBDF4ECC74389E3B4F3D39F8A290FC561@ZHEXCHANGE.schmid-telecom.com>
References: <52D00EBDF4ECC74389E3B4F3D39F8A290FC561@ZHEXCHANGE.schmid-telecom.com>
Message-ID: <20051206203818.GA10711@mems-exchange.org>

On Tue, Dec 06, 2005 at 11:20:46AM +0100, Weber, Gregoire wrote:
> We're seriously evaluating Python for use in embedded realtime systems
> and need some informations about Pythons garbage collector.
> 
> What we're interested mostly in the runtime behaviour of the GC. The
> main question is:
> 
>   Does it interrupt the python interpreter to collect stuff or 
>   is collecting done more in the background (e.g. just incrementally 
>   collecting)? This is an important question for realtime systems with
>   well defined reaction times.

It does not run in the background.  One option would be to disable
the cyclic garbage collector and rely on the reference counting
alone.  In that case, you will need to be sure that your code does
not create reference cycles.  Unfortunately I suspect there is now
Python library code that requires the cyclic collector to be
running.

> Just point me to documentation if available.

At this point the best documentation is the gcmodule.c code itself
(assuming the Python library documentation and my web page are not
sufficient).  The main entry point is collect().

If you really want to look at those old mailing list messages, you
can go the the index and search through the subject titles:

    http://www.python.org/pipermail/python-dev/2000-March/

Here are some starting points:

    http://mail.python.org/pipermail/python-dev/2000-March/002385.html
    http://mail.python.org/pipermail/python-dev/2000-March/002497.html

Note that most of discussion was related to handling finalizers
(e.g. __del__ methods) and is not relevant to your concerns.
 
  Neil

From martin at v.loewis.de  Tue Dec  6 23:06:00 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 06 Dec 2005 23:06:00 +0100
Subject: [Python-Dev] ast-objects branch created
In-Reply-To: <4395798C.9010103@gmail.com>
References: <438E31F0.1020900@v.loewis.de>	<e8bf7a530512010511s12b68cf0xf9b0f04133e608d5@mail.gmail.com>	<438F7B59.6060206@v.loewis.de>	<e8bf7a530512042137v1b39f275g9887116e439a6ace@mail.gmail.com>	<4393E35D.1090504@v.loewis.de>	<e8bf7a530512050546u1bccec07jd53690aae6638cd6@mail.gmail.com>	<07E27726-7D67-48FB-80BB-1D59241EEEDB@fuhm.net>	<e8bf7a530512050836k3235ee72j94ecc3204a8811f3@mail.gmail.com>	<ee2a432c0512051124n50e1282drf592a2ad54e206db@mail.gmail.com>
	<4395798C.9010103@gmail.com>
Message-ID: <43960B48.1000000@v.loewis.de>

Nick Coghlan wrote:
> As Fredrik pointed out a while back, the PyObject approach doesn't *have* to 
> involve manual decref operations - PyObject's come with a ready made arena 
> structure, in the form of PyList.

That doesn't really work: PyList_Append (which you would have to use) 
duplicates the reference, so you would still have to decref it
explicitly.

Of course, you could do so right away, instead of doing it on exit.

Regards,
Martin

From tim.peters at gmail.com  Wed Dec  7 00:52:44 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 6 Dec 2005 18:52:44 -0500
Subject: [Python-Dev] Documentation about Python's GC,
	python-dev list messages referenced in Modules/gcmodule.c not
	reachable anymore
In-Reply-To: <20051206203818.GA10711@mems-exchange.org>
References: <52D00EBDF4ECC74389E3B4F3D39F8A290FC561@ZHEXCHANGE.schmid-telecom.com>
	<20051206203818.GA10711@mems-exchange.org>
Message-ID: <1f7befae0512061552r743e1fe8n60a6516f7ef4439c@mail.gmail.com>

[Weber, Gregoire]
>> We're seriously evaluating Python for use in embedded realtime systems
>> and need some informations about Pythons garbage collector.
...

[Neil Schemenauer]
> It does not run in the background.  One option would be to disable
> the cyclic garbage collector and rely on the reference counting
> alone.

Python-style refcounting isn't generally a good approach either when
real-time constraints must be met:  when a refcount on an object P
falls to 0, not only does the interpreter "pause" to reclaim P, but
also to reclaim all the objects that were reachable only from P.  For
example, after

    def f():
        dummy = xrange(10000000)

    f()

it's not just the `dummy` list object that's reclaimed when f exits,
it's also about 10 million integer objects.  Deeply nested lists and
tuples (etc) can provoke similar "burps".

> In that case, you will need to be sure that your code does
> not create reference cycles.  Unfortunately I suspect there is now
> Python library code that requires the cyclic collector to be
> running.

And in the core.  For example, new-style class objects are full of
cycles -- although it's unlikely most programs will create a large
number of new-style classes dynamically.

Are there any languages with gc that are suitable for real-time work? 
Probably not without a lot of effort specifically aimed at meeting
real-time constraints.  It's also generally true that the Python core
and libraries use algorithms with good expected-case behavior but
horrid worst-case behavior.  For most apps, that's a big win most of
the time; for real-time apps, that _can_ be disastrous.

From tim.peters at gmail.com  Wed Dec  7 01:08:12 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 6 Dec 2005 19:08:12 -0500
Subject: [Python-Dev] Documentation about Python's GC,
	python-dev list messages referenced in Modules/gcmodule.c not
	reachable anymore
In-Reply-To: <1f7befae0512061552r743e1fe8n60a6516f7ef4439c@mail.gmail.com>
References: <52D00EBDF4ECC74389E3B4F3D39F8A290FC561@ZHEXCHANGE.schmid-telecom.com>
	<20051206203818.GA10711@mems-exchange.org>
	<1f7befae0512061552r743e1fe8n60a6516f7ef4439c@mail.gmail.com>
Message-ID: <1f7befae0512061608o7a0329cew9f69831d45b76c57@mail.gmail.com>

[Tim Peters]
> ...
> For example, after
>
>    def f():
>        dummy = xrange(10000000)
>
>    f()
>
> it's not just the `dummy` list object that's reclaimed when f exits,
> it's also about 10 million integer objects.

Sorry, that example should have used "range" instead of "xrange". 
Using xrange, no integer objects are created ;-)

From raymond.hettinger at verizon.net  Wed Dec  7 01:58:10 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Tue, 06 Dec 2005 19:58:10 -0500
Subject: [Python-Dev] Short-circuiting iterators
In-Reply-To: <1133380323.19766.45.camel@localhost.localdomain>
Message-ID: <000401c5fac9$4b67ee00$70bb9d8d@oemcomputer>

[Matthew F. Barnes]  Perhaps it would be a useful addition to the
itertools
> module then?
> 
>         itertools.interruptable(iterable)

Any real-world use cases or compelling contrived examples?

ISTM, that the code calling it.stop() would already be in position to
break-out of the iteration directly or set a termination flag.  Instead
of:

    it = itertools.interruptable(iterable):
    for x in it:
        . . .
        if cond(x):
            it.stop()

Why not write:

    for x in iterable:
        . . .
        if cond(x):
            break

If needed, the for-loop can have an else-clause for any processing
needed in the event of interruption.



Raymond 


From simonwittber at gmail.com  Wed Dec  7 03:55:06 2005
From: simonwittber at gmail.com (Simon Wittber)
Date: Wed, 7 Dec 2005 10:55:06 +0800
Subject: [Python-Dev] Short-circuiting iterators
In-Reply-To: <000401c5fac9$4b67ee00$70bb9d8d@oemcomputer>
References: <1133380323.19766.45.camel@localhost.localdomain>
	<000401c5fac9$4b67ee00$70bb9d8d@oemcomputer>
Message-ID: <4e4a11f80512061855v1d149416lb125b51479833cc0@mail.gmail.com>

On 12/7/05, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> [Matthew F. Barnes]  Perhaps it would be a useful addition to the
> itertools
> > module then?
> >
> >         itertools.interruptable(iterable)
>
> Any real-world use cases or compelling contrived examples?

I use something like this in the nanothreads module.

http://metaplay.dyndns.org:8081/svn/fibranet/fibranet/nanothreads.py

This feature is implemented in the Fibra class, in the end and kill
methods. It is used to stop an iterator from parts of the code which
are not part of the loop that is actually iterating the iterator.

This usage is specific to situations where generators are being used
as cooperative threads.

-Sw,

From raymond.hettinger at verizon.net  Wed Dec  7 03:58:50 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Tue, 06 Dec 2005 21:58:50 -0500
Subject: [Python-Dev] Short-circuiting iterators
In-Reply-To: <4e4a11f80512061855v1d149416lb125b51479833cc0@mail.gmail.com>
Message-ID: <000b01c5fada$26e232a0$70bb9d8d@oemcomputer>

[Matthew F. Barnes]  
> > > Perhaps it would be a useful addition to the itertools
> > > module then?
> > >
> > >         itertools.interruptable(iterable)

[Raymond Hettinger]
> > Any real-world use cases or compelling contrived examples?

[Simon Wittber]
> I use something like this in the nanothreads module.
> 
> http://metaplay.dyndns.org:8081/svn/fibranet/fibranet/nanothreads.py
> 
> This feature is implemented in the Fibra class, in the end and kill
> methods. It is used to stop an iterator from parts of the code which
> are not part of the loop that is actually iterating the iterator.
> 
> This usage is specific to situations where generators are being used
> as cooperative threads.

Are there any generator specific needs that are not met by the PEP 342
implementation?  Given a choice between throw(), close(), and send(), I
would have thought that all the bases have been covered.


Raymond


From simonwittber at gmail.com  Wed Dec  7 04:05:24 2005
From: simonwittber at gmail.com (Simon Wittber)
Date: Wed, 7 Dec 2005 11:05:24 +0800
Subject: [Python-Dev] Short-circuiting iterators
In-Reply-To: <000b01c5fada$26e232a0$70bb9d8d@oemcomputer>
References: <4e4a11f80512061855v1d149416lb125b51479833cc0@mail.gmail.com>
	<000b01c5fada$26e232a0$70bb9d8d@oemcomputer>
Message-ID: <4e4a11f80512061905ie8b04e5h3592b1c775888960@mail.gmail.com>

On 12/7/05, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> Are there any generator specific needs that are not met by the PEP 342
> implementation?  Given a choice between throw(), close(), and send(), I
> would have thought that all the bases have been covered.

Agreed. When the new functionality in PEP 342 arrives, most of the
generator tricks in nanothreads.py will become redundant.

-Sw.

From raymond.hettinger at verizon.net  Wed Dec  7 09:03:42 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Wed, 07 Dec 2005 03:03:42 -0500
Subject: [Python-Dev] Broader iterable support for xmlrpclib
In-Reply-To: <17301.4829.921367.703349@montanaro.dyndns.org>
Message-ID: <000d01c5fb04$db976420$1421a044@oemcomputer>

[Skip]
> I then proposed the
> even wackier idea to simply allow all currently unsupported iterables
> (sets
> and arrays seem the most obvious candidates to me) to be marshalled as
> lists

Doesn't the appropriate conversion depend on the contract between the
sender and receiver (i.e. an array of type 'c' may either be converted
as list(arr) or arr.tostring() depending on the app)?  Is the goal to
save writing explicit conversions by presuming that most iterables
aspire to be lists for transport purposes?


Raymond


From skip at pobox.com  Wed Dec  7 12:33:37 2005
From: skip at pobox.com (skip@pobox.com)
Date: Wed, 7 Dec 2005 05:33:37 -0600
Subject: [Python-Dev] Tracker anonymity
In-Reply-To: <7xr78pfv90.fsf@ruckus.brouhaha.com>
Message-ID: <17302.51345.865707.117047@montanaro.dyndns.org>


In c.l.py Paul Rubin wrote:

    Paul> In the old days, it was possible to post stuff to Python's
    Paul> sourceforge pages without logging in.  That was turned off for
    Paul> various reasons that weren't bogus, but that didn't strike me as
    Paul> overwhelmingly compelling.  Maybe that could be revisited, at
    Paul> least for the category of documentation bugs and patches.

Any thoughts about maybe relaxing the login restriction?  I know we had
problems with anonymous submissions in the past (mostly inability to contact
the requester for more info I think), but perhaps that downside is less
important than the perception some people have that logging in is a barrier
to submission.

Try it for a few months and see?

Skip


From ncoghlan at gmail.com  Wed Dec  7 13:21:30 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 07 Dec 2005 22:21:30 +1000
Subject: [Python-Dev] Tracker anonymity
In-Reply-To: <17302.51345.865707.117047@montanaro.dyndns.org>
References: <17302.51345.865707.117047@montanaro.dyndns.org>
Message-ID: <4396D3CA.6040206@gmail.com>

skip at pobox.com wrote:
> In c.l.py Paul Rubin wrote:
> 
>     Paul> In the old days, it was possible to post stuff to Python's
>     Paul> sourceforge pages without logging in.  That was turned off for
>     Paul> various reasons that weren't bogus, but that didn't strike me as
>     Paul> overwhelmingly compelling.  Maybe that could be revisited, at
>     Paul> least for the category of documentation bugs and patches.
> 
> Any thoughts about maybe relaxing the login restriction?  I know we had
> problems with anonymous submissions in the past (mostly inability to contact
> the requester for more info I think), but perhaps that downside is less
> important than the perception some people have that logging in is a barrier
> to submission.
> 
> Try it for a few months and see?

If the submitter is anonymous, not only can we not follow up with any 
questions, neither does the submitter get notified of status changes on their 
tracker.

IMO, allowing anonymous access will lead to people getting ticked that their 
bug reports were closed "not enough information" or "works for me" and they 
weren't notified about it.

Then again, there are some genuinely simple reports (especially with docs), 
where the overhead of having to log in (or worse, create a SF account) means 
the problem doesn't get reported, and hence, doesn't get fixed.

Can we put a warning on the anonymous submission page pointing out the 
problems with using it for non-trivial bug reports?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From gregoire.weber at schmid-telecom.ch  Wed Dec  7 14:02:05 2005
From: gregoire.weber at schmid-telecom.ch (Weber, Gregoire)
Date: Wed, 7 Dec 2005 14:02:05 +0100
Subject: [Python-Dev] Documentation about Python's GC,
	python-dev list messages referenced in Modules/gcmodule.c not
	reachable anymore
Message-ID: <52D00EBDF4ECC74389E3B4F3D39F8A290FC565@ZHEXCHANGE.schmid-telecom.com>

Hi Neil,
Hi Tim,
Hi Simon,

your responsive and valuable answers cleared up most of the open
questions and gave a very positive impression to my project leader about
the python community and python itself. 

Thank you very much!

I'm trying to get more into the GC implementation before asking more
about some still open points.

Gregoire

From skip at pobox.com  Wed Dec  7 15:27:38 2005
From: skip at pobox.com (skip@pobox.com)
Date: Wed, 7 Dec 2005 08:27:38 -0600
Subject: [Python-Dev] Broader iterable support for xmlrpclib
In-Reply-To: <000d01c5fb04$db976420$1421a044@oemcomputer>
References: <17301.4829.921367.703349@montanaro.dyndns.org>
	<000d01c5fb04$db976420$1421a044@oemcomputer>
Message-ID: <17302.61786.385957.574216@montanaro.dyndns.org>


    >> I then proposed the even wackier idea to simply allow all currently
    >> unsupported iterables (sets and arrays seem the most obvious
    >> candidates to me) to be marshalled as lists

    Raymond> Doesn't the appropriate conversion depend on the contract
    Raymond> between the sender and receiver (i.e. an array of type 'c' may
    Raymond> either be converted as list(arr) or arr.tostring() depending on
    Raymond> the app)?  Is the goal to save writing explicit conversions by
    Raymond> presuming that most iterables aspire to be lists for transport
    Raymond> purposes?

Sure, I suspect it depends on the contract.  The contract my patch enforces
is whether or not list(obj) succeeds.  If that fails, a TypeError is raised
as before.  If it succeeds incorrectly, I suspect the programmer will figure
that out soon enough and make the appropriate adjustment.  In the common
case though, I suspect it will work though.  As indicated in the patch
submission, the goals are to:

  * extend the set of sequences that can be marshalled transparently

  * keep the caller from caring as much about the limitations of the XML-RPC
    datatypes

Skip

From skip at pobox.com  Wed Dec  7 15:50:02 2005
From: skip at pobox.com (skip@pobox.com)
Date: Wed, 7 Dec 2005 08:50:02 -0600
Subject: [Python-Dev] Tracker anonymity
In-Reply-To: <4396D3CA.6040206@gmail.com>
References: <17302.51345.865707.117047@montanaro.dyndns.org>
	<4396D3CA.6040206@gmail.com>
Message-ID: <17302.63130.72110.808997@montanaro.dyndns.org>


    >> Any thoughts about maybe relaxing the login restriction?

    Nick> If the submitter is anonymous, not only can we not follow up with
    Nick> any questions, neither does the submitter get notified of status
    Nick> changes on their tracker.

You're preaching to the choir.  I'm made the same argument over and over on
c.l.py, but there is a vocal minority there that believes the current state
of affairs is a barrier that prevents submissions.  Here's my latest post on
that topic explaining all the reasons why the way we do it today is the best
we can do -- for now at least.

  http://mail.python.org/pipermail/python-list/2005-December/314298.html

    Nick> Can we put a warning on the anonymous submission page pointing out
    Nick> the problems with using it for non-trivial bug reports?

I don't think we have any control over the boilerplate SF displays.

My biggest frustration is that SF doesn't support tracker interaction via
email.  That would present its own set of problems (think spam & virii), but
seems like it ought to lower the barrier to submission a bit.

Skip

From guido at python.org  Wed Dec  7 16:39:39 2005
From: guido at python.org (Guido van Rossum)
Date: Wed, 7 Dec 2005 07:39:39 -0800
Subject: [Python-Dev] Broader iterable support for xmlrpclib
In-Reply-To: <17302.61786.385957.574216@montanaro.dyndns.org>
References: <17301.4829.921367.703349@montanaro.dyndns.org>
	<000d01c5fb04$db976420$1421a044@oemcomputer>
	<17302.61786.385957.574216@montanaro.dyndns.org>
Message-ID: <ca471dc20512070739q6c93abaag1db55e1bd34a3553@mail.gmail.com>

On 12/7/05, skip at pobox.com <skip at pobox.com> wrote:
>
>     >> I then proposed the even wackier idea to simply allow all currently
>     >> unsupported iterables (sets and arrays seem the most obvious
>     >> candidates to me) to be marshalled as lists
>
>     Raymond> Doesn't the appropriate conversion depend on the contract
>     Raymond> between the sender and receiver (i.e. an array of type 'c' may
>     Raymond> either be converted as list(arr) or arr.tostring() depending on
>     Raymond> the app)?  Is the goal to save writing explicit conversions by
>     Raymond> presuming that most iterables aspire to be lists for transport
>     Raymond> purposes?
>
> Sure, I suspect it depends on the contract.  The contract my patch enforces
> is whether or not list(obj) succeeds.  If that fails, a TypeError is raised
> as before.  If it succeeds incorrectly, I suspect the programmer will figure
> that out soon enough and make the appropriate adjustment.  In the common
> case though, I suspect it will work though.  As indicated in the patch
> submission, the goals are to:
>
>   * extend the set of sequences that can be marshalled transparently
>
>   * keep the caller from caring as much about the limitations of the XML-RPC
>     datatypes

I think this is a bad idea. XML-RPC is severely limited in what it can
handle; it is NOT a general marshalling protocol for Python data
types. It's better to be aware of this than to try and ignore it.
Having to write list(x) means that you are aware that you're consuming
any iterators. It also means that mistakes (e.g. passing in a file)
are caught earlier.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Dec  7 17:30:32 2005
From: guido at python.org (Guido van Rossum)
Date: Wed, 7 Dec 2005 08:30:32 -0800
Subject: [Python-Dev] Tracker anonymity
In-Reply-To: <17302.63130.72110.808997@montanaro.dyndns.org>
References: <17302.51345.865707.117047@montanaro.dyndns.org>
	<4396D3CA.6040206@gmail.com>
	<17302.63130.72110.808997@montanaro.dyndns.org>
Message-ID: <ca471dc20512070830q4ccd8a35oe0184342380d6dd2@mail.gmail.com>

On 12/7/05, skip at pobox.com <skip at pobox.com> wrote:
> You're preaching to the choir.  I'm made the same argument over and over on
> c.l.py, but there is a vocal minority there that believes the current state
> of affairs is a barrier that prevents submissions.

I definitely don't want anonymous bug reports (even for trivial ones).
I expect the problem is more that in order to submit non-anonymously
not only do you have to provide an email address, but you have to go
through the whole rigmarole of signing up as a SF user.

I don't think that the issue is so much the need for anonymity but the
signup hassle. (After all, if you really want to be anonymous you can
sign up under an alias or use an anonymizer; if you really don't want
to read the SF email you can direct the mail to /dev/null.)

Anyway, now that we've moved to our own Subversion, the trackers are
the lsat part of the Python infrastructure that remains on SF. There's
a perfectly capable replacement ready to wait on python.org. Maybe we
should finally switch to roundup so we can abandon SF?

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Wed Dec  7 17:42:45 2005
From: skip at pobox.com (skip@pobox.com)
Date: Wed, 7 Dec 2005 10:42:45 -0600
Subject: [Python-Dev] Tracker anonymity
In-Reply-To: <ca471dc20512070830q4ccd8a35oe0184342380d6dd2@mail.gmail.com>
References: <17302.51345.865707.117047@montanaro.dyndns.org>
	<4396D3CA.6040206@gmail.com>
	<17302.63130.72110.808997@montanaro.dyndns.org>
	<ca471dc20512070830q4ccd8a35oe0184342380d6dd2@mail.gmail.com>
Message-ID: <17303.4357.114909.921727@montanaro.dyndns.org>


    Guido> Anyway, now that we've moved to our own Subversion, the trackers
    Guido> are the lsat part of the Python infrastructure that remains on
    Guido> SF. There's a perfectly capable replacement ready to wait on
    Guido> python.org. Maybe we should finally switch to roundup so we can
    Guido> abandon SF?

Fine with me.  Is it ready to go though?  If not, what more needs to be
done?  I was under the assumption that it wasn't ready for prime time and
solicited inputs on c.l.py in a couple messages yesterday and today.

Skip


From mwh at python.net  Wed Dec  7 18:51:10 2005
From: mwh at python.net (Michael Hudson)
Date: Wed, 07 Dec 2005 17:51:10 +0000
Subject: [Python-Dev] Tracker anonymity
In-Reply-To: <17302.51345.865707.117047@montanaro.dyndns.org> (skip@pobox.com's
	message of "Wed, 7 Dec 2005 05:33:37 -0600")
References: <17302.51345.865707.117047@montanaro.dyndns.org>
Message-ID: <2mwtig94oh.fsf@starship.python.net>

skip at pobox.com writes:

> In c.l.py Paul Rubin wrote:
>
>     Paul> In the old days, it was possible to post stuff to Python's
>     Paul> sourceforge pages without logging in.  That was turned off for
>     Paul> various reasons that weren't bogus, but that didn't strike me as
>     Paul> overwhelmingly compelling.  Maybe that could be revisited, at
>     Paul> least for the category of documentation bugs and patches.
>
> Any thoughts about maybe relaxing the login restriction?  I know we had
> problems with anonymous submissions in the past (mostly inability to contact
> the requester for more info I think), but perhaps that downside is less
> important than the perception some people have that logging in is a barrier
> to submission.
>
> Try it for a few months and see?

No way.  I see no reason to force the people who complain about this
to find another excuse to not do anything useful.

Cheers,
mwh
(harsh, but...)

-- 
  MacOSX: Sort of like a pedigree persian cat. Very sleek, very
  sexy, but a little too prone to going cross-eyed, biting you on
  your thumb and then throwing up on your trousers.
                           -- Jim's pedigree of operating systems, asr

From tl at gocept.com  Wed Dec  7 19:40:36 2005
From: tl at gocept.com (Thomas Lotze)
Date: Wed, 07 Dec 2005 19:40:36 +0100
Subject: [Python-Dev] hasattr and properties
Message-ID: <pan.2005.12.07.18.40.36.293136@ID-174572.user.uni-berlin.de>

Hi,

I've noticed some behaviour of hasattr when used on properties which I'm
inclined to call a bug, or at least unexpected behaviour:

Python 2.4.2 (#1, Oct 29 2005, 13:11:33) 
[GCC 3.3.6 (Gentoo 3.3.6, ssp-3.3.6-1.0, pie-8.7.8)] on linux2
[...]
>>> class Foo(object):
...     def get(self):
...         print "hi there"
...         raise Exception
...     bar = property(get)
... 
>>> hasattr(Foo, "bar")
True
>>> hasattr(Foo(), "bar")
hi there
False

One would expect hasattr to yield the same result in both cases, and the
result to be True.

Apparently, when applied to a class instance, hasattr calls getattr and
decides that the attribute doesn't exist if the call raises any exception.

- Wouldn't it make sense to only report a missing attribute if an
AttributeError is raised?

- As far as properties are concerned, it would make even more sense to not
call getattr but try to look up the attribute the same way getattr would.
This would, however, not work consistently anymore if one customizes
attribute access. Has anyone thought about that matter?

-- 
Thomas


From t-meyer at ihug.co.nz  Wed Dec  7 21:53:28 2005
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Thu, 8 Dec 2005 09:53:28 +1300
Subject: [Python-Dev] Tracker anonymity
In-Reply-To: <17302.63130.72110.808997@montanaro.dyndns.org>
References: <17302.51345.865707.117047@montanaro.dyndns.org>
	<4396D3CA.6040206@gmail.com>
	<17302.63130.72110.808997@montanaro.dyndns.org>
Message-ID: <2F638531-5686-4C1C-916B-4F7ED5BE1171@ihug.co.nz>

[Nick]
>> Can we put a warning on the anonymous submission page pointing out
>> the problems with using it for non-trivial bug reports?

[Skip]
> I don't think we have any control over the boilerplate SF displays.

There must be some control.  There's a "Outlook users please see the  
list of frequently reported bugs" message on the bugs page for  
spambayes.  My guess would be that Mark Hammond put it there, so he  
probably knows (or knew :) how to do it.

=Tony.Meyer

From tdelaney at avaya.com  Wed Dec  7 22:04:30 2005
From: tdelaney at avaya.com (Delaney, Timothy (Tim))
Date: Thu, 8 Dec 2005 08:04:30 +1100
Subject: [Python-Dev] ast-objects branch created
Message-ID: <2773CAC687FD5F4689F526998C7E4E5F4DB85C@au3010avexu1.global.avaya.com>

"Martin v. L?wis" wrote:

> Nick Coghlan wrote:
>> As Fredrik pointed out a while back, the PyObject approach doesn't
>> *have* to involve manual decref operations - PyObject's come with a
>> ready made arena structure, in the form of PyList.
> 
> That doesn't really work: PyList_Append (which you would have to use)
> duplicates the reference, so you would still have to decref it
> explicitly.

Hmm - perhaps we should have a set of Arena functions/macros e.g. PyArena_Add - works like PyList_Append, but doesn't duplicate the reference (or immediately decrefs it).

I'm sure there are other parts of the code base that would benefit from this.

Tim Delaney

From fredrik at pythonware.com  Wed Dec  7 23:20:34 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 7 Dec 2005 23:20:34 +0100
Subject: [Python-Dev] ast-objects branch created
References: <2773CAC687FD5F4689F526998C7E4E5F4DB85C@au3010avexu1.global.avaya.com>
Message-ID: <dn7n7k$4mj$1@sea.gmane.org>

"Delaney, Timothy (Tim)" wrote:

> > Nick Coghlan wrote:
> >> As Fredrik pointed out a while back, the PyObject approach doesn't
> >> *have* to involve manual decref operations - PyObject's come with a
> >> ready made arena structure, in the form of PyList.
> >
> > That doesn't really work: PyList_Append (which you would have to use)
> > duplicates the reference, so you would still have to decref it
> > explicitly.
>
> Hmm - perhaps we should have a set of Arena functions/macros e.g. PyArena_Add - works like
> PyList_Append, but doesn't duplicate the reference (or immediately decrefs it).
>
> I'm sure there are other parts of the code base that would benefit from this.

if you check my original post, you'll find code for a new list helper function,
which would solve this in a convenient way.

</F>




From jeremy at alum.mit.edu  Wed Dec  7 23:21:14 2005
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 7 Dec 2005 17:21:14 -0500
Subject: [Python-Dev] ast-objects branch created
In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F4DB85C@au3010avexu1.global.avaya.com>
References: <2773CAC687FD5F4689F526998C7E4E5F4DB85C@au3010avexu1.global.avaya.com>
Message-ID: <e8bf7a530512071421v77426a08h1f0eb44de530718f@mail.gmail.com>

On 12/7/05, Delaney, Timothy (Tim) <tdelaney at avaya.com> wrote:
> "Martin v. L?wis" wrote:
>
> > Nick Coghlan wrote:
> >> As Fredrik pointed out a while back, the PyObject approach doesn't
> >> *have* to involve manual decref operations - PyObject's come with a
> >> ready made arena structure, in the form of PyList.
> >
> > That doesn't really work: PyList_Append (which you would have to use)
> > duplicates the reference, so you would still have to decref it
> > explicitly.
>
> Hmm - perhaps we should have a set of Arena functions/macros e.g. PyArena_Add - works like PyList_Append, but doesn't duplicate the reference (or immediately decrefs it).
>
> I'm sure there are other parts of the code base that would benefit from this.

There is such a function on the ast-arena branch.  The current
implementation uses a linked list, so it consumes more memory than
using a PyList.

Jeremy

From greg.ewing at canterbury.ac.nz  Thu Dec  8 01:05:14 2005
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 08 Dec 2005 13:05:14 +1300
Subject: [Python-Dev] hasattr and properties
In-Reply-To: <pan.2005.12.07.18.40.36.293136@ID-174572.user.uni-berlin.de>
References: <pan.2005.12.07.18.40.36.293136@ID-174572.user.uni-berlin.de>
Message-ID: <439778BA.3020908@canterbury.ac.nz>

Thomas Lotze wrote:

> Apparently, when applied to a class instance, hasattr calls getattr and
> decides that the attribute doesn't exist if the call raises any exception.
 > - Wouldn't it make sense to only report a missing attribute if an
 > AttributeError is raised?

That would be an improvement, but calling the property access
code as a side effect of hasattr seems like a misfeature to me
in the first place.

> - As far as properties are concerned, it would make even more sense to not
> call getattr but try to look up the attribute the same way getattr would.
> This would, however, not work consistently anymore if one customizes
> attribute access.

Maybe descriptors need a fourth slot for hasattr
customisation?

The logic would then be

   if there is a descriptor for the attribute:
     if the descriptor's hasattr slot is populated:
       return the result of calling it
     else:
       return True
   else:
     look in the instance dict for the attribute

There wouldn't be a need to rely on catching exceptions
at all, then.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From guido at python.org  Thu Dec  8 02:38:54 2005
From: guido at python.org (Guido van Rossum)
Date: Wed, 7 Dec 2005 17:38:54 -0800
Subject: [Python-Dev] hasattr and properties
In-Reply-To: <439778BA.3020908@canterbury.ac.nz>
References: <pan.2005.12.07.18.40.36.293136@ID-174572.user.uni-berlin.de>
	<439778BA.3020908@canterbury.ac.nz>
Message-ID: <ca471dc20512071738y588020c2n429880e26f1dc6ca@mail.gmail.com>

On 12/7/05, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Maybe descriptors need a fourth slot for hasattr
> customisation?
>
> The logic would then be
>
>    if there is a descriptor for the attribute:
>      if the descriptor's hasattr slot is populated:
>        return the result of calling it
>      else:
>        return True
>    else:
>      look in the instance dict for the attribute

Um, that does't work for types which customize __getattribute__ or
__getattr__ in various ways.

IMO a property that has a side effect (other than updating a cache or
statistics or perhaps logging) is a misfeature anyway, so I don't see
what's wrong with hasattr() trying getattr() and reporting False IFF
that raises an exception.

If you want only AttributeError to be handled, use getattr(x, 'name', None).

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mfb at lotusland.dyndns.org  Wed Dec  7 19:57:36 2005
From: mfb at lotusland.dyndns.org (Matthew F. Barnes)
Date: Wed, 07 Dec 2005 12:57:36 -0600
Subject: [Python-Dev] Short-circuiting iterators
In-Reply-To: <000401c5fac9$4b67ee00$70bb9d8d@oemcomputer>
References: <000401c5fac9$4b67ee00$70bb9d8d@oemcomputer>
Message-ID: <1133981856.14376.30.camel@localhost.localdomain>

On Tue, 2005-12-06 at 19:58 -0500, Raymond Hettinger wrote:
> Any real-world use cases or compelling contrived examples?
> 
> ISTM, that the code calling it.stop() would already be in position to
> break-out of the iteration directly or set a termination flag.  Instead
> of:
> 
>     it = itertools.interruptable(iterable):
>     for x in it:
>         . . .
>         if cond(x):
>             it.stop()
> 
> Why not write:
> 
>     for x in iterable:
>         . . .
>         if cond(x):
>             break
> 
> If needed, the for-loop can have an else-clause for any processing
> needed in the event of interruption.

The idea was motivated by a case of nested loops, similar to:

    for x in iterable1:
        for y in iterable2:
            for z in iterable3:
                . . .
                if cond1(x):
                    iterable1.stop()
                if cond2(y):
                    iterable2.stop()
                if cond3(z):
                    iterable3.stop()
                . . .

It seemed more convenient at the time than having to deal with multiple
termination flags, or breaks, or a combination thereof.

The ability to remotely terminate a for-loop also struck me as somewhat
interesting:

    def estimate(item, iterable):
        . . .
        if good_enough:
            iterable.stop()
            return result

    for x in iterable:
        . . .
        approx *= estimate(x, iterable)

But these are highly contrived and hardly compelling.

I was primarily interested in whether anyone recalls discussing the
ability to prematurely terminate an iterator and whether there are any
technical drawbacks other than it being redundant.

Matthew Barnes

From greg.ewing at canterbury.ac.nz  Thu Dec  8 00:47:22 2005
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 08 Dec 2005 12:47:22 +1300
Subject: [Python-Dev] Tracker anonymity
In-Reply-To: <4396D3CA.6040206@gmail.com>
References: <17302.51345.865707.117047@montanaro.dyndns.org>
	<4396D3CA.6040206@gmail.com>
Message-ID: <4397748A.1080107@canterbury.ac.nz>

Nick Coghlan wrote:

> If the submitter is anonymous, not only can we not follow up with any 
> questions, neither does the submitter get notified of status changes on their 
> tracker.

Would not simply requesting an email address along with
the report provide enough information for questioning and
notifying the submitter? I don't see why a full-blown
registration and login should be required.

If the submitter chooses not to supply an email address,
then they have no grounds for being upset at not receiving
any notifications!

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From raymond.hettinger at verizon.net  Thu Dec  8 05:15:55 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Wed, 07 Dec 2005 23:15:55 -0500
Subject: [Python-Dev] Short-circuiting iterators
In-Reply-To: <1133981856.14376.30.camel@localhost.localdomain>
Message-ID: <006d01c5fbae$166f4780$1421a044@oemcomputer>

[Matthew F. Barnes]
> The ability to remotely terminate a for-loop also struck me as
somewhat
> interesting:
> 
>     def estimate(item, iterable):
>         . . .
>         if good_enough:
>             iterable.stop()
>             return result
> 
>     for x in iterable:
>         . . .
>         approx *= estimate(x, iterable)

Good inspiration; wrong technique.  For the RightWay(tm), try a
functional approach composing a target calculation with a function
generating successively more accurate approximations and a consumer
function that stops when the desired accuracy is achieved.  The idea is
to decouple the steps into side-effect-free, reusable components.  For a
worked-out, comprehensive example, see
http://www.md.chalmers.se/~rjmh/Papers/whyfp.pdf .


> But these are highly contrived and hardly compelling.

That suggests an answer to your earlier question as to whether
itertools.interruptable(iterable) would be a useful addition ;-)



Raymond


From ironfroggy at gmail.com  Thu Dec  8 07:45:20 2005
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Thu, 8 Dec 2005 01:45:20 -0500
Subject: [Python-Dev] hasattr and properties
In-Reply-To: <439778BA.3020908@canterbury.ac.nz>
References: <pan.2005.12.07.18.40.36.293136@ID-174572.user.uni-berlin.de>
	<439778BA.3020908@canterbury.ac.nz>
Message-ID: <76fd5acf0512072245v50b0f4bdr3e24d35df1bd9a69@mail.gmail.com>

On 12/7/05, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Thomas Lotze wrote:
>
> > Apparently, when applied to a class instance, hasattr calls getattr and
> > decides that the attribute doesn't exist if the call raises any exception.
>  > - Wouldn't it make sense to only report a missing attribute if an
>  > AttributeError is raised?
>
> That would be an improvement, but calling the property access
> code as a side effect of hasattr seems like a misfeature to me
> in the first place.

I will have to disagree with you there. If hasattr(a,b) returns True,
one should be able to expect a.b will work properly. Otherwise, the
majority of use cases for hasattr will be completely thrown out the
window. How can hasattr work properly with properties if it doesn't
call the property access code?

> > - As far as properties are concerned, it would make even more sense to not
> > call getattr but try to look up the attribute the same way getattr would.
> > This would, however, not work consistently anymore if one customizes
> > attribute access.
>
> Maybe descriptors need a fourth slot for hasattr
> customisation?
>
> The logic would then be
>
>    if there is a descriptor for the attribute:
>      if the descriptor's hasattr slot is populated:
>        return the result of calling it
>      else:
>        return True
>    else:
>      look in the instance dict for the attribute
>
> There wouldn't be a need to rely on catching exceptions
> at all, then.

If there is a descriptor with no hasattr slot, it should call __get__
to make sure it is successful (ie, doesn't raise any exceptions).

From martin at v.loewis.de  Thu Dec  8 09:52:18 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 08 Dec 2005 09:52:18 +0100
Subject: [Python-Dev] Documentation about Python's GC,
 python-dev list messages referenced in Modules/gcmodule.c
 not	reachable anymore
In-Reply-To: <1f7befae0512061552r743e1fe8n60a6516f7ef4439c@mail.gmail.com>
References: <52D00EBDF4ECC74389E3B4F3D39F8A290FC561@ZHEXCHANGE.schmid-telecom.com>	<20051206203818.GA10711@mems-exchange.org>
	<1f7befae0512061552r743e1fe8n60a6516f7ef4439c@mail.gmail.com>
Message-ID: <4397F442.8000604@v.loewis.de>

Tim Peters wrote:
> Python-style refcounting isn't generally a good approach either when
> real-time constraints must be met:  when a refcount on an object P
> falls to 0, not only does the interpreter "pause" to reclaim P, but
> also to reclaim all the objects that were reachable only from P.

Sure - but that still allows for meeting real-time constraints.
You need to understand, or estimate, the worst-case execution time
that any statement may have. In general, any assignment in Python
could trigger releasing many objects, as the old value is released.

In a specific program, analysis is much brighter. You *know* what
variables carry references to huge data structures, and you *know*
where these variables are assigned to. Also, you can often give
an upper bound to the number of objects that may get released in
the worst case if you assign to a variable.

This is different from garbage collection: it is generally very
difficult to predict when precisely the garbage collector will
be invoked; in the current Python implementation, you would have
to predict at what point in the code you hit the 1000 objects
quota. This is nothing that can be learned from local inspection,
and thus hard to tell.

For a real time system, it is not just important that all
actions complete fast. Instead, it is important to tell what
statements might be long-running, and how long. In Python's
reference counting, this is possible, hence it is (IMO)
suitable for real-time purposes.

Regards,
Martin

From martin at v.loewis.de  Thu Dec  8 09:57:10 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 08 Dec 2005 09:57:10 +0100
Subject: [Python-Dev] Tracker anonymity
In-Reply-To: <17302.51345.865707.117047@montanaro.dyndns.org>
References: <17302.51345.865707.117047@montanaro.dyndns.org>
Message-ID: <4397F566.8080509@v.loewis.de>

skip at pobox.com wrote:
>     Paul> In the old days, it was possible to post stuff to Python's
>     Paul> sourceforge pages without logging in.  That was turned off for
>     Paul> various reasons that weren't bogus, but that didn't strike me as
>     Paul> overwhelmingly compelling.  Maybe that could be revisited, at
>     Paul> least for the category of documentation bugs and patches.
> 
> Any thoughts about maybe relaxing the login restriction?

I can't understand why he could expect that patch submissions could be
anonymous. We are moving towards more bureaucracy here, not less: we
should really require that the contributor form is filled out for
submissions (both code and documentation): on documentation, there is
also copyright, and formally, the contributor needs to declare under
what license we can use the text that he wrote.

Now, there is always the issue with "trivial" modifications (like
typos); we haven't been able to clarify yet what the bar for "this
has copyright" really is, that's why I haven't been pushing for
executing a "require forms signed" policy, yet.

For bug reports, this issue does not exist - but the other issues
(of being able to get feedback) do, of course.

Regards,
Martin

From martin at v.loewis.de  Thu Dec  8 10:04:06 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 08 Dec 2005 10:04:06 +0100
Subject: [Python-Dev] Tracker anonymity
In-Reply-To: <17303.4357.114909.921727@montanaro.dyndns.org>
References: <17302.51345.865707.117047@montanaro.dyndns.org>	<4396D3CA.6040206@gmail.com>	<17302.63130.72110.808997@montanaro.dyndns.org>	<ca471dc20512070830q4ccd8a35oe0184342380d6dd2@mail.gmail.com>
	<17303.4357.114909.921727@montanaro.dyndns.org>
Message-ID: <4397F706.9050506@v.loewis.de>

skip at pobox.com wrote:
> Fine with me.  Is it ready to go though?  If not, what more needs to be
> done?  I was under the assumption that it wasn't ready for prime time and
> solicited inputs on c.l.py in a couple messages yesterday and today.

Primarily, it needs a dedicated operator. Somebody who is willing to get
it done, and somebody who doesn't run away when it is done, but reacts
in a timely manner (similar to what I have been doing with subversion).

So far, nobody has stepped forward. Without such a person, I would be
very hesitant to accept switching, even if the technology was
demonstrable ready and easy to operate.

In addition, the pending technical issues are perhaps deeper than in
the case for subversion. The conversion tool, for subversion, was ready
to go, and nearly perfect (except that I managed to get the time not
increasing monotonically); I'm just not sure what the status of the
roundup conversion tool is (but I do remember that it exists, and
is able to process the data obtained from SF XML export).

One other issue I remember is that the SF redirector replacement is
not implemented: I would expect that python.org/sf/<sf bug id>
will map the SF bug ids into roundup bug ids, and redirect
appropriately; this hasn't been done (to my knowledge).

Regards,
Martin

From skip at pobox.com  Thu Dec  8 20:04:59 2005
From: skip at pobox.com (skip@pobox.com)
Date: Thu, 8 Dec 2005 13:04:59 -0600
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
Message-ID: <17304.33755.693941.811233@montanaro.dyndns.org>


Passing along from c.l.py.  I think ElementTree is the poster child for
best-of-breed code belonging in the standard distribution.  Its API is so
much better than what we have there now that assuming any non-technical
issues can be solved (licensing, duplicate copies of the source code) that
ElementTree and any necessary C helper/accelerator modules it uses should
wind up in the distribution.

Skip

-------------- next part --------------
An embedded message was scrubbed...
From: Steven Bethard <steven.bethard at gmail.com>
Subject: Re: ElementTree - Why not part of the core?
Date: Thu, 08 Dec 2005 11:46:38 -0700
Size: 6370
Url: http://mail.python.org/pipermail/python-dev/attachments/20051208/b0771a4b/attachment.mht

From tim.peters at gmail.com  Thu Dec  8 20:55:12 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 8 Dec 2005 14:55:12 -0500
Subject: [Python-Dev] Documentation about Python's GC,
	python-dev list messages referenced in Modules/gcmodule.c not
	reachable anymore
In-Reply-To: <4397F442.8000604@v.loewis.de>
References: <52D00EBDF4ECC74389E3B4F3D39F8A290FC561@ZHEXCHANGE.schmid-telecom.com>
	<20051206203818.GA10711@mems-exchange.org>
	<1f7befae0512061552r743e1fe8n60a6516f7ef4439c@mail.gmail.com>
	<4397F442.8000604@v.loewis.de>
Message-ID: <1f7befae0512081155v5d54bb8foe3e7ecd851c1736b@mail.gmail.com>

[Tim Peters]
>> Python-style refcounting isn't generally a good approach either when
>> real-time constraints must be met:  when a refcount on an object P
>> falls to 0, not only does the interpreter "pause" to reclaim P, but
>> also to reclaim all the objects that were reachable only from P.

[Martin v. L?wis]
> Sure - but that still allows for meeting real-time constraints.

I doubt people who work in hard real-time would agree -- I'm just
regurgitating the standard line here.  For recent work, see:

    The Space Cost of Lazy Reference Counting
    Boehm, Hans-J.
    POPL 2004, or online at
    http://www.hpl.hp.com/techreports/2003/HPL-2003-215.html

    Reference counting memory management is often advocated as a
    technique for reducing or avoiding the pauses associated with tracing
    garbage collection. We present some measurements to remind the
    reader that classic reference count implementations

    [i.e., "Python-style refcounting" - tim]

    may in fact exhibit longer pauses than tracing collectors. We then
    analyze reference counting with lazy deletion, the standard technique for
    avoiding long pauses by deferring deletions and associated reference
    count decrements,

    [which is indeed the standard technique for making refcounting
     friendlier to real-time work - tim]

    usually to allocation time. ...

> ...

> In a specific program, analysis is much brighter. You *know* what
> variables carry references to huge data structures, and you *know*
> where these variables are assigned to.

> ...

Most people work with countless lines of library code they didn't
write, and whose internals aren't documented at all.  When I do

    myinstance.a = zope.app.frobulater.gringo()

did I just create a reference to a huge new data structure?  I have no
idea.  How long will refcounting take to clean up mysinstance.a when
myinstance becomes trash?  Ditto.  Multiply by many thousands.  "Lazy"
refcounting guarantees "not long" regardless -- although it introduces
other problems.

> ...

From martin at v.loewis.de  Thu Dec  8 21:42:01 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 08 Dec 2005 21:42:01 +0100
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <17304.33755.693941.811233@montanaro.dyndns.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
Message-ID: <43989A99.9000402@v.loewis.de>

skip at pobox.com wrote:
> Passing along from c.l.py.  I think ElementTree is the poster child for
> best-of-breed code belonging in the standard distribution.

That's primarily for the author of the software to decide, at this
point. Fredrik Lundh would have to offer it for contribution first.

I don't know what his current position is, but I think it is unlikely
that he will contribute it: in the past, he often indicated that he
a) dislikes the growth of the standard Python library, and
b) dislikes forking his own branch for inclusion in another package
    (which would happen if he contributed one version for the
     standard library, and would continue to maintain the code
     outside of Python also).

That said, I agree that ElementTree would be a valuable addition
to the Python library, and has certainly passed the "collect feedback
in the real world" test.

Regards,
Martin

From skip at pobox.com  Thu Dec  8 21:59:05 2005
From: skip at pobox.com (skip@pobox.com)
Date: Thu, 8 Dec 2005 14:59:05 -0600
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <43989A99.9000402@v.loewis.de>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de>
Message-ID: <17304.40601.540045.702729@montanaro.dyndns.org>


    Martin> skip at pobox.com wrote:
    >> Passing along from c.l.py.  I think ElementTree is the poster child
    >> for best-of-breed code belonging in the standard distribution.

    Martin> That's primarily for the author of the software to decide, at
    Martin> this point. Fredrik Lundh would have to offer it for
    Martin> contribution first.

Understood.  Hence the reference in my note to "non-technical issues".
Fredrik has been participating in the c.l.py thread.  I doubt he will be shy
about voicing his opinion here.

Skip

From jcarlson at uci.edu  Thu Dec  8 21:58:57 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu, 08 Dec 2005 12:58:57 -0800
Subject: [Python-Dev] Documentation about Python's GC,
	python-dev list messages referenced in Modules/gcmodule.c not
	reachable anymore
In-Reply-To: <1f7befae0512081155v5d54bb8foe3e7ecd851c1736b@mail.gmail.com>
References: <4397F442.8000604@v.loewis.de>
	<1f7befae0512081155v5d54bb8foe3e7ecd851c1736b@mail.gmail.com>
Message-ID: <20051208124633.8D24.JCARLSON@uci.edu>


Tim Peters <tim.peters at gmail.com> wrote:
> > ...
> 
> > In a specific program, analysis is much brighter. You *know* what
> > variables carry references to huge data structures, and you *know*
> > where these variables are assigned to.
> 
> > ...
> 
> Most people work with countless lines of library code they didn't
> write, and whose internals aren't documented at all.  When I do
> 
>     myinstance.a = zope.app.frobulater.gringo()
> 
> did I just create a reference to a huge new data structure?  I have no
> idea.  How long will refcounting take to clean up mysinstance.a when
> myinstance becomes trash?  Ditto.  Multiply by many thousands.  "Lazy"
> refcounting guarantees "not long" regardless -- although it introduces
> other problems.

I believe this particular argument is specious.  Using Python won't
change requirements for knowing what is or is not referenced during
program execution for "real time" development.  Further, "real time"
developers won't be using arbitrary 3rd party code which may do unknown
crazy things, it'll all be known crazy things.

My flight-controller-software-validator friend tells me that they use a
slimmed-down variant of Python for some of their software, and that it
works great.

 - Josiah


From tim.peters at gmail.com  Thu Dec  8 22:13:45 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 8 Dec 2005 16:13:45 -0500
Subject: [Python-Dev] Documentation about Python's GC,
	python-dev list messages referenced in Modules/gcmodule.c not
	reachable anymore
In-Reply-To: <20051208124633.8D24.JCARLSON@uci.edu>
References: <4397F442.8000604@v.loewis.de>
	<1f7befae0512081155v5d54bb8foe3e7ecd851c1736b@mail.gmail.com>
	<20051208124633.8D24.JCARLSON@uci.edu>
Message-ID: <1f7befae0512081313r40788251l953681a4d7cf16f4@mail.gmail.com>

[Josiah Carlson]
> I believe this particular argument is specious.

Not at all, but it's not compelling on its own.

> Using Python won't change requirements for knowing what is or is
> not referenced during program execution for "real time" development.
>  Further, "real time" developers won't be using arbitrary 3rd party code
> which may do unknown crazy things, it'll all be known crazy things.

So you believe they're not going to use any code they didn't write
from scratch themselves?  For example, they're not going to use the
Python libraries?  Not going to use core Python data types?  "3rd
party" seems a wrong distinction if "wholly understood" is what's
relevant.  Even something as simple as list.append(3) _can_ cause a
long delay, and that's not a "_known_ crazy thing" unless you're an
expert in Python internals.

> My flight-controller-software-validator friend tells me that they use a
> slimmed-down variant of Python for some of their software, and that it
> works great.

Then I hope that and a marketing department makes you rich ;-)

From jim at zope.com  Thu Dec  8 21:53:17 2005
From: jim at zope.com (Jim Fulton)
Date: Thu, 08 Dec 2005 15:53:17 -0500
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <43989A99.9000402@v.loewis.de>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de>
Message-ID: <43989D3D.1030001@zope.com>

Martin v. L?wis wrote:
> skip at pobox.com wrote:
> 
>>Passing along from c.l.py.  I think ElementTree is the poster child for
>>best-of-breed code belonging in the standard distribution.
> 
> 
> That's primarily for the author of the software to decide, at this
> point. Fredrik Lundh would have to offer it for contribution first.
> 
> I don't know what his current position is, but I think it is unlikely
> that he will contribute it: in the past, he often indicated that he
> a) dislikes the growth of the standard Python library, and
> b) dislikes forking his own branch for inclusion in another package
>     (which would happen if he contributed one version for the
>      standard library, and would continue to maintain the code
>      outside of Python also).

Ooh. Well said. I agree with both of these points. :)

+1

> That said, I agree that ElementTree would be a valuable addition
> to the Python library, and has certainly passed the "collect feedback
> in the real world" test.

I hope that packaging progress will someday make it matter much less
whether something is in the standard library.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From steve at holdenweb.com  Thu Dec  8 22:32:59 2005
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 08 Dec 2005 21:32:59 +0000
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <43989D3D.1030001@zope.com>
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de>
	<43989D3D.1030001@zope.com>
Message-ID: <4398A68B.70209@holdenweb.com>

Jim Fulton wrote:
[...]
> 
> I hope that packaging progress will someday make it matter much less
> whether something is in the standard library.
> 
For which we need a *mechanism* that all package providers can 
implement, rather than a repository to which all package providers must 
contribute.

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From jim at zope.com  Thu Dec  8 22:45:29 2005
From: jim at zope.com (Jim Fulton)
Date: Thu, 08 Dec 2005 16:45:29 -0500
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <4398A68B.70209@holdenweb.com>
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de>	<43989D3D.1030001@zope.com>
	<4398A68B.70209@holdenweb.com>
Message-ID: <4398A979.8080704@zope.com>

Steve Holden wrote:
> Jim Fulton wrote:
> [...]
> 
>>I hope that packaging progress will someday make it matter much less
>>whether something is in the standard library.
>>
> 
> For which we need a *mechanism* that all package providers can 
> implement, rather than a repository to which all package providers must 
> contribute.

I think we need both.  We need the mechanism and repositories, although
non necessarily one repository.  Phillip Eby and others seem to be making
wonderful progress on the mechanism.  I haven't had a chance to play with this
yet, but I expect to  over the next few months.  People who have seem
quite enthusiastic.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From skip at pobox.com  Thu Dec  8 22:52:11 2005
From: skip at pobox.com (skip@pobox.com)
Date: Thu, 8 Dec 2005 15:52:11 -0600
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <43989D3D.1030001@zope.com>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <43989D3D.1030001@zope.com>
Message-ID: <17304.43787.15359.734400@montanaro.dyndns.org>


    Jim> I hope that packaging progress will someday make it matter much
    Jim> less whether something is in the standard library.

It undoubtedly will.  The point I was trying to raise here is that
ElementTree is so much better than the stuff we currently distribute (*)
that it should be included in the standard distribution if for no other
reason than to discourage use of the current stuff in new applications.

Here are a couple perhaps useful BDFL references:

    http://mail.python.org/pipermail/python-dev/2003-December/040928.html
    http://mail.python.org/pipermail/python-dev/2003-April/034881.html

In the first, Guido tells a potential submitter to "do the math" to make
sure his package is "best of breed".  In the second Guido warns that having
code in the standard distribution tends to suppress usage of other packages,
even though they may be better:

  We can't put every approach in the core, but putting one package in
  the core may damage the viability of another, possibly better (for
  some users) solution.  To some extent this has happened with GUI
  toolkits: the presence of Tkinter in the core makes it harder for
  other GUI toolkits to compete (leaving aside whether Tkinter is
  better or not -- it's just not a level playing field).

I think that's sort of the reverse of the point I'm trying to make.  ET
belongs in the standard distribution to create a level playing field for a
module many people feel is superior to the current XML-related modules.
Think of it as Pythonic affirmative action. ;-)

Skip

(*) As in so much better that I was actually able to a) understand how to
use it and then b) actually use it in a real application.  With the DOM
stuff the barrier was always too high for me to ever want to solve real
problems with them.

From jim at zope.com  Thu Dec  8 23:01:54 2005
From: jim at zope.com (Jim Fulton)
Date: Thu, 08 Dec 2005 17:01:54 -0500
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <17304.43787.15359.734400@montanaro.dyndns.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <43989D3D.1030001@zope.com>
	<17304.43787.15359.734400@montanaro.dyndns.org>
Message-ID: <4398AD52.30908@zope.com>

skip at pobox.com wrote:
>     Jim> I hope that packaging progress will someday make it matter much
>     Jim> less whether something is in the standard library.
> 
> It undoubtedly will.  The point I was trying to raise here is that
> ElementTree is so much better than the stuff we currently distribute (*)
> that it should be included in the standard distribution if for no other
> reason than to discourage use of the current stuff in new applications.

Maybe we should deprecate the current stuff. It's been done before.
Of course, much of the current XML support is still useful, if only
because lots of existing 3rd-party code depend on it.  When we have a
packaging system we could move these out of the core without
disparaging them and without breaking third-party modules.

> 
> Here are a couple perhaps useful BDFL references:
> 
>     http://mail.python.org/pipermail/python-dev/2003-December/040928.html
>     http://mail.python.org/pipermail/python-dev/2003-April/034881.html
> 
> In the first, Guido tells a potential submitter to "do the math" to make
> sure his package is "best of breed".  In the second Guido warns that having
> code in the standard distribution tends to suppress usage of other packages,
> even though they may be better:
> 
>   We can't put every approach in the core, but putting one package in
>   the core may damage the viability of another, possibly better (for
>   some users) solution.  To some extent this has happened with GUI
>   toolkits: the presence of Tkinter in the core makes it harder for
>   other GUI toolkits to compete (leaving aside whether Tkinter is
>   better or not -- it's just not a level playing field).

Well said. +1 I agree with this too. :)

> I think that's sort of the reverse of the point I'm trying to make. 

But it's one of the reasons why I'd like to see fewer application-level
facilities added to the core.  I'd rather make it easier to try out different
tools and figure out what's best for a particular situation.

 > ET
> belongs in the standard distribution to create a level playing field for a
> module many people feel is superior to the current XML-related modules.
> Think of it as Pythonic affirmative action. ;-)

I would only think of it as Pythonoc affirmative action if you also included
the FourSuite stuff and lxml and ...  which, of course, would be bad.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From pje at telecommunity.com  Thu Dec  8 23:47:37 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 08 Dec 2005 17:47:37 -0500
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <4398A979.8080704@zope.com>
References: <4398A68B.70209@holdenweb.com>
	<17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <43989D3D.1030001@zope.com>
	<4398A68B.70209@holdenweb.com>
Message-ID: <5.1.1.6.0.20051208174347.02c1f348@mail.telecommunity.com>

At 04:45 PM 12/8/2005 -0500, Jim Fulton wrote:
>I think we need both.  We need the mechanism and repositories, although
>non necessarily one repository.  Phillip Eby and others seem to be making
>wonderful progress on the mechanism.

And FYI, it does not depend on a single repository.  TurboGears, for 
example, depends on a number of packages that are not distributed on PyPI, 
or which lack a setup script.  A packager can simply publish a page of 
usable download links (or even Subversion URLs) in order to participate.

Of course, it's easier if packages are linked from PyPI, and you can avoid 
naming collisions that way, but the basic requirements for a "repository" 
can actually be met with nothing more than an Apache directory index.


From victor.stinner-linux at haypocalc.com  Fri Dec  9 01:44:02 2005
From: victor.stinner-linux at haypocalc.com (Victor Stinner)
Date: Fri, 9 Dec 2005 01:44:02 +0100
Subject: [Python-Dev] Bug bz2.BZ2File(...).seek(0,2) + patch
In-Reply-To: <20051125145447.GA25513@panix.com>
References: <1132885900.18774.5.camel@haypopc>
	<20051125145447.GA25513@panix.com>
Message-ID: <200512090144.03110.victor.stinner-linux@haypocalc.com>

Le Vendredi 25 Novembre 2005 15:54, Aahz a ?crit?:
> On Fri, Nov 25, 2005, Victor STINNER wrote:
> > I found a bug in bz2 python module. Example:
> >
> > Details and *patch* at:
> > http://sourceforge.net/tracker/index.php?func=detail&aid=1366000&group_id
> >=5470&atid=105470
>
> Thanks!  Particularly with the Thanksgiving weekend, you may not get any
> other responses for a while.  Please be patient.

No other reaction!? A soooo small patch :-)

Haypo
-- 
Victor Stinner - student at the UTBM (Belfort, France)
http://www.haypocalc.com/wiki/Victor_Stinner

From jcarlson at uci.edu  Fri Dec  9 02:02:58 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu, 08 Dec 2005 17:02:58 -0800
Subject: [Python-Dev] Documentation about Python's GC,
	python-dev list messages referenced in Modules/gcmodule.c not
	reachable anymore
In-Reply-To: <1f7befae0512081313r40788251l953681a4d7cf16f4@mail.gmail.com>
References: <20051208124633.8D24.JCARLSON@uci.edu>
	<1f7befae0512081313r40788251l953681a4d7cf16f4@mail.gmail.com>
Message-ID: <20051208164858.8D2A.JCARLSON@uci.edu>


Tim Peters <tim.peters at gmail.com> wrote:
> 
> [Josiah Carlson]
> > I believe this particular argument is specious.
> 
> Not at all, but it's not compelling on its own.

I like that better.

> > Using Python won't change requirements for knowing what is or is
> > not referenced during program execution for "real time" development.
> >  Further, "real time" developers won't be using arbitrary 3rd party code
> > which may do unknown crazy things, it'll all be known crazy things.
> 
> So you believe they're not going to use any code they didn't write
> from scratch themselves?

Of course not.

> For example, they're not going to use the
> Python libraries?  Not going to use core Python data types?  "3rd
> party" seems a wrong distinction if "wholly understood" is what's
> relevant.  Even something as simple as list.append(3) _can_ cause a
> long delay, and that's not a "_known_ crazy thing" unless you're an
> expert in Python internals.

If someone is planning on using Python for real time systems, and they
don't know that Python uses arrays as lists, then they aren't paying
attention to the stuff they should be - exactly those Python internals
that _can_ affect their running time.  Considering that Mr. Weber was
asking about the Python garbage collector specifically, I would imagine
that they have at least started delving into the inner workings of
Python data structures.

I personally don't build real-time systems, but if I did, the first
thing I would do is remove everything from the standard library.  As the
code in the standard library is verified, validated, and its running
time documented (as such module functionality is requested), I'd add
them into the standard library.  The first things that I would document
would be all of the objects which sit in __builtins__ .  But who knows,
maybe real-time people aren't as anal about correctness as a CS Theory
grad student - though I would guess they are a bit more anal than I am
(what with all that formal correctness and proof training that some of
my eastern European friends poke me in the ribs about).


> > My flight-controller-software-validator friend tells me that they use a
> > slimmed-down variant of Python for some of their software, and that it
> > works great.
> 
> Then I hope that and a marketing department makes you rich ;-)

I'm completely detached from his work and what he does, and he's a
salaried employee.  Only the guys on the top are getting rich.

 - Josiah


From gregoire.weber at schmid-telecom.ch  Tue Dec  6 11:20:46 2005
From: gregoire.weber at schmid-telecom.ch (Weber, Gregoire)
Date: Tue, 6 Dec 2005 11:20:46 +0100
Subject: [Python-Dev] Documentation about Python's GC,
	python-dev list messages referenced in Modules/gcmodule.c not
	reachable anymore
Message-ID: <52D00EBDF4ECC74389E3B4F3D39F8A290FC561@ZHEXCHANGE.schmid-telecom.com>

Hi All,
Hi Neil,

While searching documentation about Pythons GC and it's behaviour I
found references to the python-dev list in ``Modules/gcmodule.c``. 

But unfortunately the links to the list do not work anymore. May someone
give me a hint how to find the messages (2. to 4. below).

Is the article (Link 1. below) up to date or were there major changes in
the GC strategies since year 2000?

http://svn.python.org/projects/python/trunk/Modules/gcmodule.c

  1. http://www.arctrix.com/nas/python/gc/
  2. http://www.python.org/pipermail/python-dev/2000-March/003869.html
  3. http://www.python.org/pipermail/python-dev/2000-March/004010.html
  4. http://www.python.org/pipermail/python-dev/2000-March/004022.html


Background Info:

We're seriously evaluating Python for use in embedded realtime systems
and need some informations about Pythons garbage collector.

What we're interested mostly in the runtime behaviour of the GC. The
main question is:

  Does it interrupt the python interpreter to collect stuff or 
  is collecting done more in the background (e.g. just incrementally 
  collecting)? This is an important question for realtime systems with
  well defined reaction times.

Just point me to documentation if available.

Gregoire

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20051206/666097f3/attachment.html

From greg.ewing at canterbury.ac.nz  Fri Dec  9 04:31:19 2005
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 09 Dec 2005 16:31:19 +1300
Subject: [Python-Dev] hasattr and properties
In-Reply-To: <ca471dc20512071738y588020c2n429880e26f1dc6ca@mail.gmail.com>
References: <pan.2005.12.07.18.40.36.293136@ID-174572.user.uni-berlin.de>
	<439778BA.3020908@canterbury.ac.nz>
	<ca471dc20512071738y588020c2n429880e26f1dc6ca@mail.gmail.com>
Message-ID: <4398FA87.2040503@canterbury.ac.nz>

Guido van Rossum wrote:

> Um, that does't work for types which customize __getattribute__ or
> __getattr__ in various ways.

There could be a __hasattr__ slot in the class itself for
that purpose.

> IMO a property that has a side effect (other than updating a cache or
> statistics or perhaps logging) is a misfeature anyway,

Even if it doesn't have side effects, getting the attribute's
value could be expensive. One shouldn't have to pay the full
cost of getting the value just to find out whether the attribute
itself exists.

To put it another way, hasattr is enquiring about the object's
interface, not asking it to perform a computation. Doing the
latter when only being asked to do the former is wrong, IMO.

Greg

From greg.ewing at canterbury.ac.nz  Fri Dec  9 04:36:48 2005
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 09 Dec 2005 16:36:48 +1300
Subject: [Python-Dev] hasattr and properties
In-Reply-To: <76fd5acf0512072245v50b0f4bdr3e24d35df1bd9a69@mail.gmail.com>
References: <pan.2005.12.07.18.40.36.293136@ID-174572.user.uni-berlin.de>
	<439778BA.3020908@canterbury.ac.nz>
	<76fd5acf0512072245v50b0f4bdr3e24d35df1bd9a69@mail.gmail.com>
Message-ID: <4398FBD0.7070003@canterbury.ac.nz>

Calvin Spealman wrote:

> I will have to disagree with you there. If hasattr(a,b) returns True,
> one should be able to expect a.b will work properly. Otherwise, the
> majority of use cases for hasattr will be completely thrown out the
> window. How can hasattr work properly with properties if it doesn't
> call the property access code?

In the vast majority of code that I write, if a class has
a descriptor for a given property, but the accessor code
doesn't work, then there is a bug.

In a case where that wasn't true, a hasattr slot in the
descriptor would enable me to code it so it did the right
thing.

Greg

From skip at pobox.com  Fri Dec  9 05:04:39 2005
From: skip at pobox.com (skip@pobox.com)
Date: Thu, 8 Dec 2005 22:04:39 -0600
Subject: [Python-Dev] Documentation about Python's GC,
 python-dev list messages referenced in Modules/gcmodule.c not
 reachable anymore
In-Reply-To: <52D00EBDF4ECC74389E3B4F3D39F8A290FC561@ZHEXCHANGE.schmid-telecom.com>
References: <52D00EBDF4ECC74389E3B4F3D39F8A290FC561@ZHEXCHANGE.schmid-telecom.com>
Message-ID: <17305.599.232402.530147@montanaro.dyndns.org>


    Gregoire> But unfortunately the links to the list do not work
    Gregoire> anymore. May someone give me a hint how to find the messages
    Gregoire> (2. to 4. below).

    Gregoire>   2. http://www.python.org/pipermail/python-dev/2000-March/003869.html
    Gregoire>   3. http://www.python.org/pipermail/python-dev/2000-March/004010.html
    Gregoire>   4. http://www.python.org/pipermail/python-dev/2000-March/004022.html

I don't know what happened to the numbering.  Those article numbers all
appear in the May 2000 archive and don't appear related to cyclic GC.  Try
browsing from this point:

    http://mail.python.org/pipermail/python-dev/2000-March/thread.html

I think this may be one of the articles:

    http://mail.python.org/pipermail/python-dev/2000-March/002385.html

That article is the starting point for an extensive thread.  In general,
March 2000 seems to have been a fertile month for cyclic GC.

-- 
Skip Montanaro
Katrina Benefit Concerts: http://www.musi-cal.com/katrina
skip at pobox.com

From tdelaney at avaya.com  Wed Dec  7 23:40:17 2005
From: tdelaney at avaya.com (Delaney, Timothy (Tim))
Date: Thu, 8 Dec 2005 09:40:17 +1100
Subject: [Python-Dev] ast-objects branch created
Message-ID: <2773CAC687FD5F4689F526998C7E4E5F074317@au3010avexu1.global.avaya.com>

Fredrik Lundh wrote:

> if you check my original post, you'll find code for a new list helper
> function, which would solve this in a convenient way.

Yep - I thought I'd seen something like this, but couldn't find it
(eventually found it by searching for Lundh ;). That's exactly what I
was thinking of. However, I'm also thinking that it's worthwhile to have
aliases that state that this is being done for memory management - hence
the idea of _PyArena_ADD (and probably _PyArena_REMOVE, which would have
to do an identity removal).

I'm taking some leave over Christmas/New Year, so I might have a look at
some other parts of the python codebase and see if there are other areas
that might benefit from using lists as arenas like this.

Tim Delaney

From guido at python.org  Fri Dec  9 19:49:37 2005
From: guido at python.org (Guido van Rossum)
Date: Fri, 9 Dec 2005 10:49:37 -0800
Subject: [Python-Dev] imaplib module with IDLE implememted via threads
In-Reply-To: <1134101943.97.669843608@cs.usyd.edu.au>
References: <1134101943.97.669843608@cs.usyd.edu.au>
Message-ID: <ca471dc20512091049jea3f0f0o75b6a53867ba1e31@mail.gmail.com>

I hope you'll check this into SVN too?

--Guido

On 12/8/05, Piers Lauder <piers at cs.usyd.edu.au> wrote:
> If anyone needs the IMAP4 extension "IDLE", there is a copy of an enhanced
> imaplib module available for download here:
>
>   http://www.cs.usyd.edu.au/~piers/python/imaplib.html
>
> This is an IMAP4rev1 mail protocol client class using threads for parallel
> operation. It is modified from the non-threaded version included in the
> standard Python distributions, but presents (a superset of) the same API.
>
> I have named this version "imaplib2" as it uses threads to implememt
> the necessary callbacks, and it doesn't seem necessary to burden the
> orginal module with requiring threading.
>
> The module has been in production use for several months, and seems stable.
>
> Piers Lauder
>
> <P><A HREF="http://www.cs.usyd.edu.au/~piers/python/imaplib.html">imaplib2</A>
> IMAP4rev1 mail protocol client class using threads to implement the IDLE extension.
> (09-Dec-2005)
>
>
>
> --
> http://mail.python.org/mailman/listinfo/python-announce-list
>
>         Support the Python Software Foundation:
>         http://www.python.org/psf/donations.html
>


--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ianb at colorstudy.com  Fri Dec  9 22:38:47 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 09 Dec 2005 15:38:47 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
Message-ID: <4399F967.3080300@colorstudy.com>

I was reading through PEP 8, and I think there's a few things that could 
be clarified or updated:

     Exception Names

       If a module defines a single exception raised for all sorts of
       conditions, it is generally called "error" or "Error".  It seems
       that built-in (extension) modules use "error" (e.g. os.error),
       while Python modules generally use "Error" (e.g. xdrlib.Error).
       The trend seems to be toward CapWords exception names.

To my knowledge, except for some outlying cases like os.error or 
socket.error (which are themselves old modules), CapWords are always 
used.  The less obvious question I'm wondering about is if exceptions 
should have names that are relatively unique, or simply unique within 
their namespace.  Built-in exceptions use fairly long names, but then 
they have no namespace.  Looking at some newer stdlib modules: email and 
optparse use longer-named exceptions; csv uses csv.Error.  Should 
"error" exceptions be discouraged?  Would http.ServerError or 
http.HTTPServerError be considered better?

Also, perhaps somewhere in the description of CapWords, how should they 
deal with acronyms?  It seems like the convention is, to give an 
example, HTTPRedirect over HttpRedirect.  I would appreciate an explicit 
preferred style.

     Global Variable Names

       (Let's hope that these variables are meant for use inside one
       module only.)  The conventions are about the same as those for
       functions.  Modules that are designed for use via "from M import *"
       should prefix their globals (and internal functions and classes)
       with an underscore to prevent exporting them.

It seems like __all__ is a better technique than leading underscores.

     Designing for inheritance

       Always decide whether a class's methods and instance variables
       should be public or non-public.  In general, never make data
       variables public unless you're implementing essentially a
       record.  It's almost always preferrable to give a functional
       interface to your class instead (and some Python 2.2
       developments will make this much nicer).

Yes, Python 2.2 developments have made this better.  Use of property() 
should be suggested.

       Also decide whether your attributes should be private or not.
       The difference between private and non-public is that the former
       will never be useful for a derived class, while the latter might
       be.  Yes, you should design your classes with inheritence in
       mind!

       Private attributes should have two leading underscores, no
       trailing underscores.

This conflicts with a previous suggestion "Generally, double leading 
underscores should be used only to avoid name conflicts with attributes 
in classes designed to be subclassed."  Or perhaps "private attributes" 
needs to be better explained.

       Non-public attributes should have a single leading underscore,
       no trailing underscores.

       Public attributes should have no leading or trailing
       underscores, unless they conflict with reserved words, in which
       case, a single trailing underscore is preferrable to a leading
       one, or a corrupted spelling, e.g. class_ rather than klass.
       (This last point is a bit controversial; if you prefer klass
       over class_ then just be consistent. :).

With class methods, this has become a more important.  Can PEP 8 include 
a preferred name for the class argument to classmethods?  I personally 
prefer cls, there are some who use klass, and I haven't see class_ used.

     - Class-based exceptions are always preferred over string-based
       exceptions.  Modules or packages should define their own
       domain-specific base exception class, which should be subclassed
       from the built-in Exception class.  Always include a class
       docstring.  E.g.:

         class MessageError(Exception):
             """Base class for errors in the email package."""

I think the language against string-based exceptions can be stronger. 
And this kind of implicitly indicates that longer names for exceptions 
are better; how long?  Should they generally end in "Error"?



-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From barry at python.org  Fri Dec  9 23:51:23 2005
From: barry at python.org (Barry Warsaw)
Date: Fri, 09 Dec 2005 17:51:23 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <4399F967.3080300@colorstudy.com>
References: <4399F967.3080300@colorstudy.com>
Message-ID: <1134168683.19370.29.camel@geddy.wooz.org>

On Fri, 2005-12-09 at 15:38 -0600, Ian Bicking wrote:
> I was reading through PEP 8, and I think there's a few things that could 
> be clarified or updated:

BTW, I'm willing to make updates to PEP 8, if we agree on what to
change.

>      Exception Names
> 
>        If a module defines a single exception raised for all sorts of
>        conditions, it is generally called "error" or "Error".  It seems
>        that built-in (extension) modules use "error" (e.g. os.error),
>        while Python modules generally use "Error" (e.g. xdrlib.Error).
>        The trend seems to be toward CapWords exception names.
> 
> To my knowledge, except for some outlying cases like os.error or 
> socket.error (which are themselves old modules), CapWords are always 
> used.  

My own preference is away from "error" and toward CapWordsEndingInError.
This is especially true now that we're recommending exceptions be
classes.  So really, the exception naming scheme is just the class
naming scheme.

> The less obvious question I'm wondering about is if exceptions 
> should have names that are relatively unique, or simply unique within 
> their namespace.  

It depends.  If the exception class is intended to be imported with
other symbols via import-* it needs to be unique of course.  Otherwise I
think it's fine that it simply be unique in its own namespace (though I
tend to make them unique anyway).

> Built-in exceptions use fairly long names, but then 
> they have no namespace.  Looking at some newer stdlib modules: email and 
> optparse use longer-named exceptions; csv uses csv.Error.  Should 
> "error" exceptions be discouraged?  Would http.ServerError or 
> http.HTTPServerError be considered better?

I think the latter are best, but "error" should definitely be out.
csv.Error is okay as a base exception, though I think I'd opt for
something longer.

> Also, perhaps somewhere in the description of CapWords, how should they 
> deal with acronyms?  It seems like the convention is, to give an 
> example, HTTPRedirect over HttpRedirect.  I would appreciate an explicit 
> preferred style.

My own preference here is for HTTPRedirect -- IOW capitalize all letters
of the acronym.

>      Global Variable Names
> 
>        (Let's hope that these variables are meant for use inside one
>        module only.)  The conventions are about the same as those for
>        functions.  Modules that are designed for use via "from M import *"
>        should prefix their globals (and internal functions and classes)
>        with an underscore to prevent exporting them.
> 
> It seems like __all__ is a better technique than leading underscores.

Yep, good point.

>      Designing for inheritance
> 
>        Always decide whether a class's methods and instance variables
>        should be public or non-public.  In general, never make data
>        variables public unless you're implementing essentially a
>        record.  It's almost always preferrable to give a functional
>        interface to your class instead (and some Python 2.2
>        developments will make this much nicer).
> 
> Yes, Python 2.2 developments have made this better.  Use of property() 
> should be suggested.

Again, good point.

>        Also decide whether your attributes should be private or not.
>        The difference between private and non-public is that the former
>        will never be useful for a derived class, while the latter might
>        be.  Yes, you should design your classes with inheritence in
>        mind!
> 
>        Private attributes should have two leading underscores, no
>        trailing underscores.
> 
> This conflicts with a previous suggestion "Generally, double leading 
> underscores should be used only to avoid name conflicts with attributes 
> in classes designed to be subclassed."  Or perhaps "private attributes" 
> needs to be better explained.

Maybe the right thing to say is that non-public attributes should always
start with at least one, and usually only one, underscore.  If it is a
private attribute of a class that is intended to be inherited from, and
there is a likelihood that subclass attributes may conflict with this
attribute's name, use two leading and no trailing underscores.

> 
>        Non-public attributes should have a single leading underscore,
>        no trailing underscores.
> 
>        Public attributes should have no leading or trailing
>        underscores, unless they conflict with reserved words, in which
>        case, a single trailing underscore is preferrable to a leading
>        one, or a corrupted spelling, e.g. class_ rather than klass.
>        (This last point is a bit controversial; if you prefer klass
>        over class_ then just be consistent. :).
> 
> With class methods, this has become a more important.  Can PEP 8 include 
> a preferred name for the class argument to classmethods?  I personally 
> prefer cls, there are some who use klass, and I haven't see class_ used.

It does seem like the more popular convention is to use "cls" than
"class_".  I'll admit the latter does look kind of ugly.  Maybe the
suggestion should be to use either a trailing single underscore or an
abbreviation instead of a spelling corruption.  We could then list some
common attribute names for common keywords, e.g. cls for class (what
else?).

>      - Class-based exceptions are always preferred over string-based
>        exceptions.  Modules or packages should define their own
>        domain-specific base exception class, which should be subclassed
>        from the built-in Exception class.  Always include a class
>        docstring.  E.g.:
> 
>          class MessageError(Exception):
>              """Base class for errors in the email package."""
> 
> I think the language against string-based exceptions can be stronger. 

Let's say something like "string-based exceptions are strongly
discouraged, and in fact may be deprecated or disappear in a future
Python version.  Use class-based exceptions."

> And this kind of implicitly indicates that longer names for exceptions 
> are better; how long?  Should they generally end in "Error"?

Again, I'd say something like: Since your exceptions will be classes,
use the CapWord naming convention for classes to name your exceptions.
It is recommended that your exception class end in the word "Error".

Also, I have some additional guidelines adapted from the Mailman coding
standards: http://barry.warsaw.us/software/STYLEGUIDE.txt

Other than the one about the inequality operator (which I know is sadly
doomed), what do you think about adding some of those suggestions to PEP
8?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051209/0555927a/attachment.pgp

From ianb at colorstudy.com  Sat Dec 10 00:19:42 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 09 Dec 2005 17:19:42 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134168683.19370.29.camel@geddy.wooz.org>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
Message-ID: <439A110E.8090605@colorstudy.com>

Barry Warsaw wrote:
> It does seem like the more popular convention is to use "cls" than
> "class_".  I'll admit the latter does look kind of ugly.  Maybe the
> suggestion should be to use either a trailing single underscore or an
> abbreviation instead of a spelling corruption.  We could then list some
> common attribute names for common keywords, e.g. cls for class (what
> else?).

I personally feel "cls" should be used for classmethods, and not 
elsewhere.  Just like I wouldn't like someone using "self" outside of 
the first argument of instance methods.  So class_ still would be a good 
spelling elsewhere.

Most other keywords don't come up in my experience.  Suggestions for 
some non-keyword abbreviations might be useful, particularly for list, 
dict, and type, which come up a lot.  I often use lst, d, and t, but I 
don't actually like any of them.  I personally am comfortable reusing 
the variables dir, input, and vars, as I never use them in code.  id, 
type, and file are somewhere in-between.  I'll use the name of builtin 
as function arguments if it is meant to be used as a keyword argument, 
and the name is appropriate.

I'm not really sure there's a useful conclusion we can come to on these.

> Also, I have some additional guidelines adapted from the Mailman coding
> standards: http://barry.warsaw.us/software/STYLEGUIDE.txt

I looked at that too, but most of these didn't jump out at me.  I'll 
copy in the parts that aren't already in PEP 8 that seem possible:

   From-imports should follow non-from imports.  Dotted imports should 
follow
   non-dotted imports.  Non-dotted imports should be grouped by increasing
   length, while dotted imports should be grouped roughly alphabetically.

This seems too complex to me for PEP 8.

   In general, there should be at most one class per module, if the module
   contains class definitions.  If it's a module of functions, that's fine,
   group them as common sense dictates.  A class-containing module can also
   contain some helper functions, but it's best to keep these non-public
   (i.e. use a single leading underscore).

This doesn't effect me that much as a library user, and I'd defer to 
whatever the package maintainer preferred in terms of file layout.

- Right hanging comments are discouraged, in favor of preceding comments.
   E.g.

     foo = blarzigop(bar)  # if you don't blarzigop it, it'll shlorp

   should be written as

     # if you don't blarzigop it, it'll shlorp
     foo = blarzigop(bar)

I agree with this, but only as a loose suggestion.

There's some sections on vertical whitespace and ^L.  In these cases I'd 
defer to the package maintainer, like with file layout.  The PEP has 
some suggestions, which I think are sufficient.

- Unless internal quote characters would mess things up, the general rule is
   that single quotes should be used for short strings, double quotes for
   triple-quoted multi-line strings and docstrings.  E.g.

     foo = 'a foo thing'
     warn = "Don't mess things up"
     notice = """Our three chief weapons are:
              - surprise
              - deception
              - an almost fanatical devotion to the pope
              """

This is more prescriptive than would be appropriate for PEP 8.  It might 
be lightly suggested that double quotes signify "data", while single 
quotes signify "symbols".  But I think that might be too subtle a 
distinction for the PEP.  I personally don't care about this, and quotes 
for docstrings are already covered.

   Do not indent subsequent lines in a triple-quoted string; you should
   consider the opening quote to be the left margin.

I don't understand exactly what you are saying here.

   Always use True and False instead of 1 and 0 for boolean
   values.

I would agree.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From ianb at colorstudy.com  Sat Dec 10 00:24:29 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 09 Dec 2005 17:24:29 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <4399F967.3080300@colorstudy.com>
References: <4399F967.3080300@colorstudy.com>
Message-ID: <439A122D.5080106@colorstudy.com>

Ian Bicking wrote:
>        (Let's hope that these variables are meant for use inside one
>        module only.)  The conventions are about the same as those for
>        functions.  Modules that are designed for use via "from M import *"
>        should prefix their globals (and internal functions and classes)
>        with an underscore to prevent exporting them.
> 
> It seems like __all__ is a better technique than leading underscores.

I think it should also go in (perhaps in the imports section) that 
__all__ comes after the imports, but before code.  I thought this was in 
there already, because I know I've seen it documented elsewhere.


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From barry at python.org  Sat Dec 10 00:49:23 2005
From: barry at python.org (Barry Warsaw)
Date: Fri, 09 Dec 2005 18:49:23 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439A110E.8090605@colorstudy.com>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
Message-ID: <1134172163.14849.33.camel@geddy.wooz.org>

On Fri, 2005-12-09 at 17:19 -0600, Ian Bicking wrote:

> I personally feel "cls" should be used for classmethods, and not 
> elsewhere.  Just like I wouldn't like someone using "self" outside of 
> the first argument of instance methods.  So class_ still would be a good 
> spelling elsewhere.

Cool.

>    Do not indent subsequent lines in a triple-quoted string; you should
>    consider the opening quote to be the left margin.
> 
> I don't understand exactly what you are saying here.

Just that I dislike:

def foo():
    """Here is a triple quoted, multiline string

       Some people write the continuation lines starting
       in this column, but I don't like that.
    """

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051209/f7dff715/attachment-0001.pgp

From fumanchu at amor.org  Sat Dec 10 01:23:50 2005
From: fumanchu at amor.org (Robert Brewer)
Date: Fri, 9 Dec 2005 16:23:50 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
Message-ID: <A77618B80CDD2543B705C82B7665D9F9032F2997@ex9.hostedexchange.local>

Barry Warsaw wrote:
> Again, I'd say something like: Since your exceptions
> will be classes, use the CapWord naming convention for
> classes to name your exceptions. It is recommended
> that your exception class end in the word "Error".

Unless, of course, your exception is not an error (like the
aforementioned HTTPRedirect). ;)


Robert Brewer
System Architect
Amor Ministries
fumanchu at amor.org

From steven.bethard at gmail.com  Sat Dec 10 02:11:42 2005
From: steven.bethard at gmail.com (Steven Bethard)
Date: Fri, 9 Dec 2005 18:11:42 -0700
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134168683.19370.29.camel@geddy.wooz.org>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
Message-ID: <d11dcfba0512091711j1da98850xe412907a40a90003@mail.gmail.com>

Barry Warsaw wrote:
> On Fri, 2005-12-09 at 15:38 -0600, Ian Bicking wrote:
> >        Also decide whether your attributes should be private or not.
> >        The difference between private and non-public is that the former
> >        will never be useful for a derived class, while the latter might
> >        be.  Yes, you should design your classes with inheritence in
> >        mind!
> >
> >        Private attributes should have two leading underscores, no
> >        trailing underscores.
> >
> > This conflicts with a previous suggestion "Generally, double leading
> > underscores should be used only to avoid name conflicts with attributes
> > in classes designed to be subclassed."  Or perhaps "private attributes"
> > needs to be better explained.
>
> Maybe the right thing to say is that non-public attributes should always
> start with at least one, and usually only one, underscore.  If it is a
> private attribute of a class that is intended to be inherited from, and
> there is a likelihood that subclass attributes may conflict with this
> attribute's name, use two leading and no trailing underscores.

I'd prefer language that discouraged double-underscores more since
they can't prevent all name conflicts, e.g.:

---------- mod1.py ----------
class C(object):
     __x = 'mod1.C'
     @classmethod
     def getx(cls):
         return cls.__x
-----------------------------
---------- mod2.py ----------
import mod1
class C(mod1.C):
     __x = 'mod2.C'
-----------------------------

py> import mod1, mod2
py> mod1.C.getx()
'mod1.C'
py> mod2.C.getx()
'mod2.C'

In this example, there should be two __x attributes, one for the
superclass and one for the subclass.  But since the name mangling
doesn't include the module name, the two classes share the same __x
attribute.  Note that this problem can arise any time a class and its
subclass share the same name.

If you have to say something about double-underscores, I'd prefer
something like:
"""
If you're concerned about name conflicts between a non-public
attribute of a class and the non-public attributes of its subclasses,
some of these can be prevented by using two leading and no trailing
underscores.  This will not work in all cases however, so sublcasses
still cannot be completely ignorant of the non-public attributes of
the superclass.
"""

STeVe
--
You can wordify anything if you just verb it.
        --- Bucky Katt, Get Fuzzy

From barry at python.org  Sat Dec 10 05:35:28 2005
From: barry at python.org (Barry Warsaw)
Date: Fri, 09 Dec 2005 23:35:28 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <A77618B80CDD2543B705C82B7665D9F9032F2997@ex9.hostedexchange.local>
References: <A77618B80CDD2543B705C82B7665D9F9032F2997@ex9.hostedexchange.local>
Message-ID: <1134189328.14849.36.camel@geddy.wooz.org>

On Fri, 2005-12-09 at 16:23 -0800, Robert Brewer wrote:
> Barry Warsaw wrote:
> > Again, I'd say something like: Since your exceptions
> > will be classes, use the CapWord naming convention for
> > classes to name your exceptions. It is recommended
> > that your exception class end in the word "Error".
> 
> Unless, of course, your exception is not an error (like the
> aforementioned HTTPRedirect). ;)

Good point!
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051209/3da1efd5/attachment.pgp

From steve at holdenweb.com  Sat Dec 10 10:20:14 2005
From: steve at holdenweb.com (Steve Holden)
Date: Sat, 10 Dec 2005 09:20:14 +0000
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <A77618B80CDD2543B705C82B7665D9F9032F2997@ex9.hostedexchange.local>
References: <A77618B80CDD2543B705C82B7665D9F9032F2997@ex9.hostedexchange.local>
Message-ID: <439A9DCE.5050301@holdenweb.com>

Robert Brewer wrote:
> Barry Warsaw wrote:
> 
>>Again, I'd say something like: Since your exceptions
>>will be classes, use the CapWord naming convention for
>>classes to name your exceptions. It is recommended
>>that your exception class end in the word "Error".
> 
> 
> Unless, of course, your exception is not an error (like the
> aforementioned HTTPRedirect). ;)
> 
For library modules I'd recommend that the exception hierarchy be rooted 
at either BaseError (for modules not designed from import-*) or 
ModulenameBaseError (for those that are).

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From mwh at python.net  Sat Dec 10 10:53:07 2005
From: mwh at python.net (Michael Hudson)
Date: Sat, 10 Dec 2005 09:53:07 +0000
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134168683.19370.29.camel@geddy.wooz.org> (Barry Warsaw's
	message of "Fri, 09 Dec 2005 17:51:23 -0500")
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
Message-ID: <2my82t6zy4.fsf@starship.python.net>

Barry Warsaw <barry at python.org> writes:

> Let's say something like "string-based exceptions are strongly
> discouraged, and in fact may be deprecated or disappear in a future
> Python version.  Use class-based exceptions."

If I have anything to do with it, they _will_ be deprecated in 2.5.
There is simply no excuse for writing new code using string
exceptions.

Cheers,
mwh
(PS: are people still interested in my new-style exceptions patch?
 http://bugs.python.org/1104669)

-- 
  In the 1950s and 60s there was a regular brain drain of young
  Australians from the cities to London, but it was because of 
  money, culture and opportunity, not spiders.
           -- Al Grant, ucam.chat, from Owen Dunn's review of the year

From ncoghlan at gmail.com  Sat Dec 10 12:01:35 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 10 Dec 2005 21:01:35 +1000
Subject: [Python-Dev] New-style exceptions patch (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <2my82t6zy4.fsf@starship.python.net>
References: <4399F967.3080300@colorstudy.com>	<1134168683.19370.29.camel@geddy.wooz.org>
	<2my82t6zy4.fsf@starship.python.net>
Message-ID: <439AB58F.1080909@gmail.com>

Michael Hudson wrote:
> Barry Warsaw <barry at python.org> writes:
> 
>> Let's say something like "string-based exceptions are strongly
>> discouraged, and in fact may be deprecated or disappear in a future
>> Python version.  Use class-based exceptions."
> 
> If I have anything to do with it, they _will_ be deprecated in 2.5.
> There is simply no excuse for writing new code using string
> exceptions.
> 
> Cheers,
> mwh
> (PS: are people still interested in my new-style exceptions patch?
>  http://bugs.python.org/1104669)

Is there a specific concern with it you want people to check out, or just a 
few more "works for me" tests on different platforms?

Cheers.
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From mwh at python.net  Sat Dec 10 14:24:09 2005
From: mwh at python.net (Michael Hudson)
Date: Sat, 10 Dec 2005 13:24:09 +0000
Subject: [Python-Dev] New-style exceptions patch
In-Reply-To: <439AB58F.1080909@gmail.com> (Nick Coghlan's message of "Sat,
	10 Dec 2005 21:01:35 +1000")
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<2my82t6zy4.fsf@starship.python.net> <439AB58F.1080909@gmail.com>
Message-ID: <2mu0dh6q6e.fsf@starship.python.net>

Nick Coghlan <ncoghlan at gmail.com> writes:

> Michael Hudson wrote:
>> (PS: are people still interested in my new-style exceptions patch?
>>  http://bugs.python.org/1104669)
>
> Is there a specific concern with it you want people to check out, or just a 
> few more "works for me" tests on different platforms?

Well, I'd like some more people to vet it for basic sanity and maybe
have a thought about documentation changes.  Beyond checking that it
compiles with various C compilers I hope that there's not that much
chance for x-platform variation here.

Cheers,
mwh

-- 
  > Why are we talking about bricks and concrete in a lisp newsgroup?
  After long experiment it was found preferable to talking about why
  Lisp is slower than C++...
                        -- Duane Rettig & Tim Bradshaw, comp.lang.lisp

From pinard at iro.umontreal.ca  Sat Dec 10 17:45:40 2005
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Sat, 10 Dec 2005 11:45:40 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439A110E.8090605@colorstudy.com>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
Message-ID: <20051210164540.GA15203@phenix.sram.qc.ca>

[Ian Bicking]
>Barry Warsaw wrote:

>Just like I wouldn't like someone using "self" outside of the first 
>argument of instance methods.

A tiny nit.  Within __new__(cls, ...), I find quite legible writing:

    self = BASECLASSE.__new__(cls, ...)

and using it afterwards.

-- 
Fran?ois Pinard   http://pinard.progiciels-bpi.ca

From bcannon at gmail.com  Sat Dec 10 23:27:21 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Sat, 10 Dec 2005 14:27:21 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <2my82t6zy4.fsf@starship.python.net>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<2my82t6zy4.fsf@starship.python.net>
Message-ID: <bbaeab100512101427l77ec5367w67d0aa568356e619@mail.gmail.com>

On 12/10/05, Michael Hudson <mwh at python.net> wrote:
> Barry Warsaw <barry at python.org> writes:
>
> > Let's say something like "string-based exceptions are strongly
> > discouraged, and in fact may be deprecated or disappear in a future
> > Python version.  Use class-based exceptions."
>
> If I have anything to do with it, they _will_ be deprecated in 2.5.
> There is simply no excuse for writing new code using string
> exceptions.
>

PEP 352 will deprecate raising string exceptions in 2.5 .

> Cheers,
> mwh
> (PS: are people still interested in my new-style exceptions patch?
>  http://bugs.python.org/1104669)

I am.  I was planning to use the patch as a base for implementing PEP
352.  Guido is pretty much ready to accept the PEP, he just has not
officially pronounced yet.

-Brett

From jim at zope.com  Sun Dec 11 17:20:24 2005
From: jim at zope.com (Jim Fulton)
Date: Sun, 11 Dec 2005 11:20:24 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <4399F967.3080300@colorstudy.com>
References: <4399F967.3080300@colorstudy.com>
Message-ID: <439C51C8.2010909@zope.com>

Ian Bicking wrote:
 > I was reading through PEP 8, and I think there's a few things that could
 > be clarified or updated:

Good idea.

...

>      Designing for inheritance
> 
>        Always decide whether a class's methods and instance variables
>        should be public or non-public.  In general, never make data
>        variables public unless you're implementing essentially a
>        record.   It's almost always preferrable to give a functional
 >        interface to your class instead (and some Python 2.2
 >        developments will make this much nicer).
 >
 > Yes, Python 2.2 developments have made this better.  Use of property()
 > should be suggested.

This seems outdated.  My impression, in part from time spent
working with the Python Labs guys, is that it is fine to have public
data sttributes even for non-"record" types.  In fact, I would argue that
any time you would be tempted to provide "getFoo()" and "setFoo(v)"
for some "private attribute _foo", it would be better to make it
public.  I certainly find "blah.foo" and "blah.foo = v" to be much
better than "blah.getFoo()" and blah.setFoo(v)".

Certainly, properties provide a safety belt.  I would argue it this
way: Python APIs can include attributes as well as methods.
Exposure of an attribute need not constrain the implementation, thanks
to properties.  OTOH, I wouldn't bother with a property unless it's needed.


> 
>        Also decide whether your attributes should be private or not.
>        The difference between private and non-public is that the former
>        will never be useful for a derived class, while the latter might
>        be.  Yes, you should design your classes with inheritence in
>        mind!
> 
>        Private attributes should have two leading underscores, no
>        trailing underscores.
> 
> This conflicts with a previous suggestion "Generally, double leading 
> underscores should be used only to avoid name conflicts with attributes 
> in classes designed to be subclassed."  Or perhaps "private attributes" 
> needs to be better explained.

While, on some level, private variables seem attractive, I think that
experience (for everyone I know) has shown them to be an attractive
nuisance.  I recommend discouraging them.

I'll note that, IMO:

- If you have to worry about protecting attributes from subclasses,
   maybe should shouldn't be using inheritence.

   (This may be too bold a statement, but perhaps the first
    rule of inheritence should echo Fowler's first rule of Distribution:
    "don't inherit". :)

    Increasingly, I like to use inheritence only to avoid "boiler plate"
    implementations, such as default methods or data implementations that
    almost all implementations of some API are going to do the same way.

    On rare occasions, I find inheritence to be, sadly, unavoidable.

    I should also make a distinction between what I would call "private"
    and "public" inheritence.  Private inheritence is between classes
    that are part of a single implementation unit or having a single
    implementor.  With private inheritence, there is much less danger
    since the same people are responsible for the base classes
    and subclasses.  It is public inheritence, where separate people
    maintain the base and subclasses where I think inhetitence should
    be used sparingly.

    Public inheritence causes too much coupling.
   )

- If you really have to use "public" inheritence, then consider naming
   conventions. I think ZODB's use of the _p_ variables has worked well
   for variables reserved for the base class attributes. (Although, I
   think if I could do it over, I would use _persistent_ rather than
   _p_.)

I'll also note that, when providing "transpatent" facilities, like
persistence or proxies whos functions are orthogonal to subclass
or proxied-object functionality, I've come to prefer the use of external
functions to access provided functionality.  For example, rather than
using something like: "someproxy._proxy_object" to get a proxied object
from a proxy, I use "getProxiedObject(someproxy)".  This allows the
proxies themselves to remain as transparent as possible.  I intend
to take a similar approach with future versions of ZODB's persistence
framework to avoid _p_ attributes and methods.

>        Non-public attributes should have a single leading underscore,
>        no trailing underscores.
> 
>        Public attributes should have no leading or trailing
>        underscores, unless they conflict with reserved words, in which
>        case, a single trailing underscore is preferrable to a leading
>        one, or a corrupted spelling, e.g. class_ rather than klass.
>        (This last point is a bit controversial; if you prefer klass
>        over class_ then just be consistent. :).
> 
> With class methods, this has become a more important.  Can PEP 8 include 
> a preferred name for the class argument to classmethods?  I personally 
> prefer cls, there are some who use klass, and I haven't see class_ used.

FWIW, as a general rule, I like using a single trailing underscore,
especially for keywords.  It allows the use of meaningful and easy
to remember names.  When the name of a variable should be "class" or
"for" or whatever, it's easy, as a Python programmer, to remember that
I need to add a trailing _.  As a reformed abuser of single-character
variable names, I've come to really hate abbreviations.  It's not only
easier to use unabbreviated names, it's easier to remember them when
reading code. (Note that ease of use hinges on editors that automate
typeing of repeated names.)

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From fredrik at pythonware.com  Sun Dec 11 17:48:51 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 11 Dec 2005 17:48:51 +0100
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de>
Message-ID: <dnhl9k$bt6$1@sea.gmane.org>

Martin v. Löwis wrote:

> That's primarily for the author of the software to decide, at this
> point. Fredrik Lundh would have to offer it for contribution first.

I've already done that, as others have noted.  Everything I release
under a Python-compatible license is available for bundling with the
python core.

> I don't know what his current position is, but I think it is unlikely
> that he will contribute it: in the past, he often indicated that he
> a) dislikes the growth of the standard Python library

Yes and no; replacing stale or incomplete parts with better libraries
are usually a very good idea (the subprocess library is a recent example)

But it's correct that I want the core library (the parts that lives in the
python development trunk) to get smaller; that doesn't necessarily mean
that a standard Python distribution should ship with a smaller library.

> b) dislikes forking his own branch for inclusion in another package
>     (which would happen if he contributed one version for the
>      standard library, and would continue to maintain the code
>      outside of Python also).

I want to avoid things like sgmlop (which was forked, and is currently
shipped with broken bindings in a mostly unmaintained library).  I also
want to avoid problems for people who've come to rely on the deve-
lopment and release approach I've used since I started shipping Python
software in 1995.

But if everyone is aware that this is a bundled piece of software, and
the development and maintenance process is updated accordingly, that
shouldn't be a problem.

Here's a plan:

  - I check in an existing elementtree release in a separate location in
    the svn.python.org source tree. e.g.

        svn.python.org/kits/elementtree-1.2.6-20050316

    this will make it clear that this is external software, and it also
    provides a reference point for tracking down local changes

  - we decide what elementtree modules to include, and where to place
    them, and copy them to the python trunk.

        (suggestion: either directly under xml, or under xml.etree)

  - I adapt the elementtree selftest so it runs under Python's test suite

  - I convert the pythondoc pages for the included modules to match the
    library reference format (someone will have to help with the markup
    here)

  - when new stable releases appear upstream, add to kits and copy
    relevant modules.  update/tweak docs as necessary.

  - delegate incoming bug reports / patches to the upstream maintainer.

and, optionally

  - sort out expat bundling issues, and include cElementTree as well
    (using the same approach as above).

whaddya think?

</F>




From skip at pobox.com  Sun Dec 11 18:11:07 2005
From: skip at pobox.com (skip@pobox.com)
Date: Sun, 11 Dec 2005 11:11:07 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439C51C8.2010909@zope.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
Message-ID: <17308.23979.978268.995719@montanaro.dyndns.org>


    Jim> This seems outdated.  My impression, in part from time spent
    Jim> working with the Python Labs guys, is that it is fine to have
    Jim> public data sttributes even for non-"record" types.  In fact, I
    Jim> would argue that any time you would be tempted to provide
    Jim> "getFoo()" and "setFoo(v)" for some "private attribute _foo", it
    Jim> would be better to make it public.  I certainly find "blah.foo" and
    Jim> "blah.foo = v" to be much better than "blah.getFoo()" and
    Jim> blah.setFoo(v)".

Presuming the foo attribute provides some element of the API that you are
willing to support forever.  If it is just an implementation detail you
should use accessor methods or properties.

Skip

From jim at zope.com  Sun Dec 11 18:25:54 2005
From: jim at zope.com (Jim Fulton)
Date: Sun, 11 Dec 2005 12:25:54 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <17308.23979.978268.995719@montanaro.dyndns.org>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<17308.23979.978268.995719@montanaro.dyndns.org>
Message-ID: <439C6122.1020606@zope.com>

skip at pobox.com wrote:
>     Jim> This seems outdated.  My impression, in part from time spent
>     Jim> working with the Python Labs guys, is that it is fine to have
>     Jim> public data sttributes even for non-"record" types.  In fact, I
>     Jim> would argue that any time you would be tempted to provide
>     Jim> "getFoo()" and "setFoo(v)" for some "private attribute _foo", it
>     Jim> would be better to make it public.  I certainly find "blah.foo" and
>     Jim> "blah.foo = v" to be much better than "blah.getFoo()" and
>     Jim> blah.setFoo(v)".
> 
> Presuming the foo attribute provides some element of the API that you are
> willing to support forever.  If it is just an implementation detail you
> should use accessor methods or properties.

If foo is an implementation detail, then it shoudln't be exposed at all, even
with accessors.  Using attribute syntax makes no more of a commitment than
accessor functions.  The decision about wither to implement
foo as a key in the instance dictionary *is* an implementation detail that
can be hidden by a property.  If the initial decision, following the rule
of "do the simplest thing that works", is to use an instance dictionary item,
then I wouldn't bother with a property until you change your mind.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From martin at v.loewis.de  Sun Dec 11 19:43:13 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 11 Dec 2005 19:43:13 +0100
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <dnhl9k$bt6$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de>
	<dnhl9k$bt6$1@sea.gmane.org>
Message-ID: <439C7341.4050300@v.loewis.de>

Fredrik Lundh wrote:
>>That's primarily for the author of the software to decide, at this
>>point. Fredrik Lundh would have to offer it for contribution first.
> 
> 
> I've already done that, as others have noted.  Everything I release
> under a Python-compatible license is available for bundling with the
> python core.

I see a difference here, though, between "offer for contribution"
and "make available for bundling". To my knowledge, you never said
(until now) "I would like to incorporate ElementTree into the Python CVS
repository, and thus have it become part of future Python releases".

> But it's correct that I want the core library (the parts that lives in the
> python development trunk) to get smaller; that doesn't necessarily mean
> that a standard Python distribution should ship with a smaller library.

I know you said this before; I could never understand what you mean
by that. I would always assume that we only ship what is in the source
repository (plus, for the specific case of Windows binaries, what is
documented in PCbuild/readme.txt). So how can the trunk get smaller,
yet the distribution larger?

>   - I check in an existing elementtree release in a separate location in
>     the svn.python.org source tree. e.g.
> 
>         svn.python.org/kits/elementtree-1.2.6-20050316
> 
>     this will make it clear that this is external software, and it also
>     provides a reference point for tracking down local changes

Ah, so you want what CVS calls a "vendor branch": code that is
externally maintained, and imported from time to time.

Clearly, "local" (i.e. python.org) changes are one primary issue,
so we should agree on an update process - I would personally prefer
one that allows for merging (in the "svn merge" sense).

The other issue is, of course, the question whose job it is to actually
perform the updates. Would you expect to do that yourself, or would
you expect somebody else does that?

I'm still troubled that you keep saying that sgmlop "was forked". I
had not been PyXML maintainer long enough to remember the precise
history of things, but it was certainly the case that you could have
updated it all the time - you still have write permission to the
PyXML repository.

>   - we decide what elementtree modules to include, and where to place
>     them, and copy them to the python trunk.
> 
>         (suggestion: either directly under xml, or under xml.etree)

Would there be a reason not to include the entire elementtree package?
Either xml.etree, or xml.tree would be fine with me, -0 for putting
it directly into xml.

>   - I adapt the elementtree selftest so it runs under Python's test suite

Good.

>   - I convert the pythondoc pages for the included modules to match the
>     library reference format (someone will have to help with the markup
>     here)

Would you then start using the tex sources as your primary sources, or
would this conversion need to be done every time the package is updated?

>   - when new stable releases appear upstream, add to kits and copy
>     relevant modules.  update/tweak docs as necessary.

The "tweak docs" part sounds somewhat worrying. Of course, you could run
"svn diff" on the old and new version, to see what doc strings have
changed or appeared - but that might be quite some work.

>   - delegate incoming bug reports / patches to the upstream maintainer.

Would it be sufficient to set you as the "Assigned To" in the SF
tracker? I don't see specific bug reporting instructions on the
elementtree page.

> and, optionally
> 
>   - sort out expat bundling issues, and include cElementTree as well
>     (using the same approach as above).

Not sure what this would be; we probably can look at it again when
we are done with the first part.

> whaddya think?

Overall, sounds like a good plan.

Regards,
Martin

From skip at pobox.com  Sun Dec 11 19:57:14 2005
From: skip at pobox.com (skip@pobox.com)
Date: Sun, 11 Dec 2005 12:57:14 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439C6122.1020606@zope.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<17308.23979.978268.995719@montanaro.dyndns.org>
	<439C6122.1020606@zope.com>
Message-ID: <17308.30346.609682.207832@montanaro.dyndns.org>

>>>>> "Jim" == Jim Fulton <jim at zope.com> writes:

    Jim> The decision about wither to implement foo as a key in the instance
    Jim> dictionary *is* an implementation detail that can be hidden by a
    Jim> property.  

If it's not in the instance dictionary, where is it?  

Skip

From jim at zope.com  Sun Dec 11 23:17:31 2005
From: jim at zope.com (Jim Fulton)
Date: Sun, 11 Dec 2005 17:17:31 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <17308.30346.609682.207832@montanaro.dyndns.org>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<17308.23979.978268.995719@montanaro.dyndns.org>
	<439C6122.1020606@zope.com>
	<17308.30346.609682.207832@montanaro.dyndns.org>
Message-ID: <439CA57B.1010201@zope.com>

skip at pobox.com wrote:
>>>>>>"Jim" == Jim Fulton <jim at zope.com> writes:
> 
> 
>     Jim> The decision about wither to implement foo as a key in the instance
>     Jim> dictionary *is* an implementation detail that can be hidden by a
>     Jim> property.  
> 
> If it's not in the instance dictionary, where is it?  

It could be in a slot. It could be in the instance dictionary
under another name. It could be in a subobject. It could be
computed from other variables...

(in a box, with a fox.... :)

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From fredrik at pythonware.com  Sun Dec 11 23:22:05 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 11 Dec 2005 23:22:05 +0100
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de><dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de>
Message-ID: <dni8qe$5eq$1@sea.gmane.org>

Martin v. Löwis wrote:

> >>That's primarily for the author of the software to decide, at this
> >>point. Fredrik Lundh would have to offer it for contribution first.
> >
> > I've already done that, as others have noted.  Everything I release
> > under a Python-compatible license is available for bundling with the
> > python core.
>
> I see a difference here, though, between "offer for contribution"
> and "make available for bundling". To my knowledge, you never said
> (until now) "I would like to incorporate ElementTree into the Python CVS
> repository, and thus have it become part of future Python releases".

Well, I'm offering you to bundle a well-defined version of ElementTree
with Python.  In practice, the plan I proposed means that we'll be shipping
a *copy* of ET with Python, not that ET development will move over to
python.org.

At any time, it should be possible to say "Python release X.Y.Z includes
ElementTree release A.B.C".

(this doesn't rule out bug fixes in the trunk, of course, but work on new
features should take place elsewhere)

> So how can the trunk get smaller, yet the distribution larger?

That's a separate issue; I'll have to get back to this at a later time.

> >   - I check in an existing elementtree release in a separate location in
> >     the svn.python.org source tree. e.g.
> >
> >         svn.python.org/kits/elementtree-1.2.6-20050316
> >
> >     this will make it clear that this is external software, and it also
> >     provides a reference point for tracking down local changes
>
> Ah, so you want what CVS calls a "vendor branch": code that is
> externally maintained, and imported from time to time.

Exactly.  But I'm not sure "branch" is really accurate here; it's more like
"snapshot".  Stable releases are added to the "vendor" tree, and relevant
files are are then copied to the appropriate location in the release tree.

> The other issue is, of course, the question whose job it is to actually
> perform the updates. Would you expect to do that yourself, or would
> you expect somebody else does that?

I can deal with this.

> I'm still troubled that you keep saying that sgmlop "was forked". I
> had not been PyXML maintainer long enough to remember the precise
> history of things, but it was certainly the case that you could have
> updated it all the time - you still have write permission to the
> PyXML repository.

Perhaps, but there's a limit to how much downstream use you can expect
anyone to monitor (cf. the Seigenthaler story).  But I should point out that
I don't think the forking was intentional; it just happened.

> >   - we decide what elementtree modules to include, and where to place
> >     them, and copy them to the python trunk.
> >
> >         (suggestion: either directly under xml, or under xml.etree)
>
> Would there be a reason not to include the entire elementtree package?
> Either xml.etree, or xml.tree would be fine with me, -0 for putting
> it directly into xml.

Since all the relevant module names start with "Element", putting it directly
under xml wouldn't be too bad.  But an xml subpackage is better, and prior
art says "etree".

I think that limiting this to ElementTree, ElementPath, and perhaps Element-
Include would be a good start.

> >   - I convert the pythondoc pages for the included modules to match the
> >     library reference format (someone will have to help with the markup
> >     here)
>
> Would you then start using the tex sources as your primary sources, or
> would this conversion need to be done every time the package is updated?

The reference documentation is autogenerated from markup in the source
file, so yes, some kind of conversion has to be done for each new release.

> >   - when new stable releases appear upstream, add to kits and copy
> >     relevant modules.  update/tweak docs as necessary.
>
> The "tweak docs" part sounds somewhat worrying. Of course, you could run
> "svn diff" on the old and new version, to see what doc strings have
> changed or appeared - but that might be quite some work.

Luckily, it can be partially automated.  And ET doesn't change very quickly.

> >   - delegate incoming bug reports / patches to the upstream maintainer.
>
> Would it be sufficient to set you as the "Assigned To" in the SF
> tracker?

Sure.

And maybe PEP 291 could be updated to cover both compatibility with older
Python versions and other compatibility issues.

> > and, optionally
> >
> >   - sort out expat bundling issues, and include cElementTree as well
> >     (using the same approach as above).
>
> Not sure what this would be; we probably can look at it again when
> we are done with the first part.

The problem is that cElementTree is, by default, statically linked against
its own (unmodified) copy of expat.  The same applies to pyexpat.  I think
it would be better if there was only one copy of expat in

(one way to do this would be to add an "function pointer table" to pyexpat
that contains pointers to selected portions of the expat API, and then add
an indirection level to cElementTree)

> > whaddya think?
>
> Overall, sounds like a good plan.

Just say "go", and I'll start working on this.

</F>




From ianb at colorstudy.com  Sun Dec 11 23:30:51 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Sun, 11 Dec 2005 16:30:51 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439C51C8.2010909@zope.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
Message-ID: <439CA89B.4030600@colorstudy.com>

Jim Fulton wrote:
>>      Designing for inheritance
>>
>>        Always decide whether a class's methods and instance variables
>>        should be public or non-public.  In general, never make data
>>        variables public unless you're implementing essentially a
>>        record.   It's almost always preferrable to give a functional
> 
>  >        interface to your class instead (and some Python 2.2
>  >        developments will make this much nicer).
>  >
>  > Yes, Python 2.2 developments have made this better.  Use of property()
>  > should be suggested.
> 
> This seems outdated.  My impression, in part from time spent
> working with the Python Labs guys, is that it is fine to have public
> data sttributes even for non-"record" types.  In fact, I would argue that
> any time you would be tempted to provide "getFoo()" and "setFoo(v)"
> for some "private attribute _foo", it would be better to make it
> public.  I certainly find "blah.foo" and "blah.foo = v" to be much
> better than "blah.getFoo()" and blah.setFoo(v)".
> 
> Certainly, properties provide a safety belt.  I would argue it this
> way: Python APIs can include attributes as well as methods.
> Exposure of an attribute need not constrain the implementation, thanks
> to properties.  OTOH, I wouldn't bother with a property unless it's needed.

So, getting back to the original paragraph, perhaps it could say:

Decide whether a class's methods and instance variables should be public 
or non-public.  Non-public methods and variables should start with an 
underscore.

Do not use accessor methods, like ``obj.getFoo()`` and 
``obj.setFoo(v)``, instead just expose a public attribute (``obj.foo``). 
  If necessary you can use ``property`` to implement the same 
functionality that accessor methods would give you.  If you do use 
properties, getting that property should never have a side effect. 
[well... I think that certain side effects like caching and logging are 
okay, but I'm not sure how to make that distinction]


Potentially it could be added that the whole issue can often be avoided 
when an object's methods perform actions instead of returning attributes 
of the object.  It's a long topic; maybe it could even just be a link, 
if someone knows of a good discussion along those lines.  I'm sure 
there's some terminology here that I'm forgetting that describes the 
design pattern.  There's also a point when the style guide becomes an 
API design guide, and I don't know how far it should go in that direction.

>>        Also decide whether your attributes should be private or not.
>>        The difference between private and non-public is that the former
>>        will never be useful for a derived class, while the latter might
>>        be.  Yes, you should design your classes with inheritence in
>>        mind!
>>
>>        Private attributes should have two leading underscores, no
>>        trailing underscores.
>>
>> This conflicts with a previous suggestion "Generally, double leading 
>> underscores should be used only to avoid name conflicts with 
>> attributes in classes designed to be subclassed."  Or perhaps "private 
>> attributes" needs to be better explained.
> 
> 
> While, on some level, private variables seem attractive, I think that
> experience (for everyone I know) has shown them to be an attractive
> nuisance.  I recommend discouraging them.

I really really hate double underscores, but I thought I'd let some 
other people suggest stronger language first.  I prefer explicit name 
mangling for those cases where people justifiably use double underscores 
now, e.g., self._MyPackage_variable instead of self.__variable, which I 
think you also suggest below.  Since it's all name mangling anyway, at 
least explicit is better than implicit, especially when it's something 
one could argue *should* look a little ugly.  Perhaps all the 
non-public/private language should be switched to just "private" (one 
underscore) and "hidden from subclasses" (double underscore).  I don't 
like calling __ private at all, because it's not what people coming from 
other languages think of as private.

> I'll note that, IMO:
> 
> - If you have to worry about protecting attributes from subclasses,
>   maybe should shouldn't be using inheritence.
> 
>   (This may be too bold a statement, but perhaps the first
>    rule of inheritence should echo Fowler's first rule of Distribution:
>    "don't inherit". :)
> 
>    Increasingly, I like to use inheritence only to avoid "boiler plate"
>    implementations, such as default methods or data implementations that
>    almost all implementations of some API are going to do the same way.
> 
>    On rare occasions, I find inheritence to be, sadly, unavoidable.
> 
>    I should also make a distinction between what I would call "private"
>    and "public" inheritence.  Private inheritence is between classes
>    that are part of a single implementation unit or having a single
>    implementor.  With private inheritence, there is much less danger
>    since the same people are responsible for the base classes
>    and subclasses.  It is public inheritence, where separate people
>    maintain the base and subclasses where I think inhetitence should
>    be used sparingly.
> 
>    Public inheritence causes too much coupling.
>   )

I think this is getting more into design, and less style guide.

> - If you really have to use "public" inheritence, then consider naming
>   conventions. I think ZODB's use of the _p_ variables has worked well
>   for variables reserved for the base class attributes. (Although, I
>   think if I could do it over, I would use _persistent_ rather than
>   _p_.)
> 
> I'll also note that, when providing "transpatent" facilities, like
> persistence or proxies whos functions are orthogonal to subclass
> or proxied-object functionality, I've come to prefer the use of external
> functions to access provided functionality.  For example, rather than
> using something like: "someproxy._proxy_object" to get a proxied object
> from a proxy, I use "getProxiedObject(someproxy)".  This allows the
> proxies themselves to remain as transparent as possible.  I intend
> to take a similar approach with future versions of ZODB's persistence
> framework to avoid _p_ attributes and methods.

This fits Python's style as well, i.e., len(obj) instead of obj.len(). 
Well, kind of.  When to use functions instead of methods is a whole 
discussion of its own.

>>        Non-public attributes should have a single leading underscore,
>>        no trailing underscores.
>>
>>        Public attributes should have no leading or trailing
>>        underscores, unless they conflict with reserved words, in which
>>        case, a single trailing underscore is preferrable to a leading
>>        one, or a corrupted spelling, e.g. class_ rather than klass.
>>        (This last point is a bit controversial; if you prefer klass
>>        over class_ then just be consistent. :).
>>
>> With class methods, this has become a more important.  Can PEP 8 
>> include a preferred name for the class argument to classmethods?  I 
>> personally prefer cls, there are some who use klass, and I haven't see 
>> class_ used.
> 
> 
> FWIW, as a general rule, I like using a single trailing underscore,
> especially for keywords.  It allows the use of meaningful and easy
> to remember names.  When the name of a variable should be "class" or
> "for" or whatever, it's easy, as a Python programmer, to remember that
> I need to add a trailing _.  As a reformed abuser of single-character
> variable names, I've come to really hate abbreviations.  It's not only
> easier to use unabbreviated names, it's easier to remember them when
> reading code. (Note that ease of use hinges on editors that automate
> typeing of repeated names.)

What about for class methods in particular; do you use class_ as the 
first argument for those methods?  Also, in the case of builtins, 
trailing _'s are dangerous; unlike keywords you won't get a SyntaxError 
if you leave the _ off, or even a NameError.  As I think about it, I 
should really change my own style to stop using even corruptions like 
lst, but perhaps seq instead.  But that's wandering off in a different 
direction from keywords.

-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org

From jim at zope.com  Sun Dec 11 23:57:58 2005
From: jim at zope.com (Jim Fulton)
Date: Sun, 11 Dec 2005 17:57:58 -0500
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <439CA89B.4030600@colorstudy.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com>
Message-ID: <439CAEF6.4000304@zope.com>

Ian Bicking wrote:
> Jim Fulton wrote:
> 
...
>>>        Also decide whether your attributes should be private or not.
>>>        The difference between private and non-public is that the former
>>>        will never be useful for a derived class, while the latter might
>>>        be.  Yes, you should design your classes with inheritence in
>>>        mind!
>>>
>>>        Private attributes should have two leading underscores, no
>>>        trailing underscores.
>>>
>>> This conflicts with a previous suggestion "Generally, double leading 
>>> underscores should be used only to avoid name conflicts with 
>>> attributes in classes designed to be subclassed."  Or perhaps 
>>> "private attributes" needs to be better explained.
>>
>>
>>
>> While, on some level, private variables seem attractive, I think that
>> experience (for everyone I know) has shown them to be an attractive
>> nuisance.  I recommend discouraging them.
> 
> 
> I really really hate double underscores, but I thought I'd let some 
> other people suggest stronger language first.  I prefer explicit name 
> mangling for those cases where people justifiably use double underscores 
> now, e.g., self._MyPackage_variable instead of self.__variable, which I 
> think you also suggest below.  Since it's all name mangling anyway, at 
> least explicit is better than implicit, especially when it's something 
> one could argue *should* look a little ugly.  Perhaps all the 
> non-public/private language should be switched to just "private" (one 
> underscore) and "hidden from subclasses" (double underscore).  I don't 
> like calling __ private at all, because it's not what people coming from 
> other languages think of as private.

Can we officially mark __private as a mistake. Perhaps:

- Strongly discourage it in the style guide

- Mark it in the language reference as a deprecated feature

- Generate deprecation warnings when it is used?
   (This might be too much.)

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From aahz at pythoncraft.com  Sun Dec 11 23:36:43 2005
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 11 Dec 2005 14:36:43 -0800
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <dnhl9k$bt6$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
Message-ID: <20051211223643.GA3462@panix.com>

On Sun, Dec 11, 2005, Fredrik Lundh wrote:
>
> whaddya think?

Huzzah!

(Not that I've used ElementTree personally, but I think this conversation
is a wonderful example of good Open Source discussion and development
practice.  Everyone involved deserves kudos, but particularly Fredrik for
taking the ball and moving it forward.)
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Don't listen to schmucks on USENET when making legal decisions.  Hire
yourself a competent schmuck."  --USENET schmuck (aka Robert Kern)

From jim at zope.com  Mon Dec 12 00:03:58 2005
From: jim at zope.com (Jim Fulton)
Date: Sun, 11 Dec 2005 18:03:58 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439CA89B.4030600@colorstudy.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com>
Message-ID: <439CB05E.1000804@zope.com>

Ian Bicking wrote:
> Jim Fulton wrote:
> 
>>>      Designing for inheritance
>>>
>>>        Always decide whether a class's methods and instance variables
>>>        should be public or non-public.  In general, never make data
>>>        variables public unless you're implementing essentially a
>>>        record.   It's almost always preferrable to give a functional
>>
>>
>>  >        interface to your class instead (and some Python 2.2
>>  >        developments will make this much nicer).
>>  >
>>  > Yes, Python 2.2 developments have made this better.  Use of property()
>>  > should be suggested.
>>
>> This seems outdated.  My impression, in part from time spent
>> working with the Python Labs guys, is that it is fine to have public
>> data sttributes even for non-"record" types.  In fact, I would argue that
>> any time you would be tempted to provide "getFoo()" and "setFoo(v)"
>> for some "private attribute _foo", it would be better to make it
>> public.  I certainly find "blah.foo" and "blah.foo = v" to be much
>> better than "blah.getFoo()" and blah.setFoo(v)".
>>
>> Certainly, properties provide a safety belt.  I would argue it this
>> way: Python APIs can include attributes as well as methods.
>> Exposure of an attribute need not constrain the implementation, thanks
>> to properties.  OTOH, I wouldn't bother with a property unless it's 
>> needed.
> 
> 
> So, getting back to the original paragraph, perhaps it could say:
> 
> Decide whether a class's methods and instance variables should be public 
> or non-public.  Non-public methods and variables should start with an 
> underscore.
> 
> Do not use accessor methods, like ``obj.getFoo()`` and 
> ``obj.setFoo(v)``, instead just expose a public attribute (``obj.foo``). 
>  If necessary you can use ``property`` to implement the same 
> functionality that accessor methods would give you.  If you do use 
> properties, getting that property should never have a side effect. 
> [well... I think that certain side effects like caching and logging are 
> okay, but I'm not sure how to make that distinction]
> 
> Potentially it could be added that the whole issue can often be avoided 
> when an object's methods perform actions instead of returning attributes 
> of the object.  It's a long topic; maybe it could even just be a link, 
> if someone knows of a good discussion along those lines.  I'm sure 
> there's some terminology here that I'm forgetting that describes the 
> design pattern.  There's also a point when the style guide becomes an 
> API design guide, and I don't know how far it should go in that direction.

Perhaps something like:

"If you find yourself writing trivial accessor functions like:

    def getFoo(self):
        return self._foo

    def setFoo(self, v):
        self._foo = v

Use attribute accessors instead.  In the example above, just store foo
in an attribute named "foo".  If you need to store foo a different way
later, you can use properties.

On the other hand, if getting or setting a variable has other application-
meaningful effects, then accessor methods might be better, or perhaps it would
be best not to expose the attributes at all.
"

...

>> While, on some level, private variables seem attractive, I think that
>> experience (for everyone I know) has shown them to be an attractive
>> nuisance.  I recommend discouraging them.
> 
> 
> I really really hate double underscores,

Doesn't everyone? :)

 > but I thought I'd let some
> other people suggest stronger language first.  I prefer explicit name 
> mangling for those cases where people justifiably use double underscores 
> now, e.g., self._MyPackage_variable instead of self.__variable, which I 
> think you also suggest below.  Since it's all name mangling anyway, at 
> least explicit is better than implicit, especially when it's something 
> one could argue *should* look a little ugly.  Perhaps all the 
> non-public/private language should be switched to just "private" (one 
> underscore) and "hidden from subclasses" (double underscore).  I don't 
> like calling __ private at all, because it's not what people coming from 
> other languages think of as private.

I think we should strongly discourage it in the style guide.  I think
we should go even further, as I pointed out in another post.

>> I'll note that, IMO:
>>
>> - If you have to worry about protecting attributes from subclasses,
>>   maybe should shouldn't be using inheritence.
>>
>>   (This may be too bold a statement, but perhaps the first
>>    rule of inheritence should echo Fowler's first rule of Distribution:
>>    "don't inherit". :)
>>
>>    Increasingly, I like to use inheritence only to avoid "boiler plate"
>>    implementations, such as default methods or data implementations that
>>    almost all implementations of some API are going to do the same way.
>>
>>    On rare occasions, I find inheritence to be, sadly, unavoidable.
>>
>>    I should also make a distinction between what I would call "private"
>>    and "public" inheritence.  Private inheritence is between classes
>>    that are part of a single implementation unit or having a single
>>    implementor.  With private inheritence, there is much less danger
>>    since the same people are responsible for the base classes
>>    and subclasses.  It is public inheritence, where separate people
>>    maintain the base and subclasses where I think inhetitence should
>>    be used sparingly.
>>
>>    Public inheritence causes too much coupling.
>>   )
> 
> 
> I think this is getting more into design, and less style guide.

Yup. Although the style guide certianly touches design in places.

...

> What about for class methods in particular; do you use class_ as the 
> first argument for those methods? 

It depends on the context.  I prefer self, as, if it's a class method,
it's clear (to me :) that self is a class.  I sometimes use cls to
be consistent with other code, but I don't like it.

 > Also, in the case of builtins,
> trailing _'s are dangerous; unlike keywords you won't get a SyntaxError 
> if you leave the _ off, or even a NameError.

Good point.

 > As I think about it, I
> should really change my own style to stop using even corruptions like 
> lst, but perhaps seq instead.  But that's wandering off in a different 
> direction from keywords.

Yup.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From bcannon at gmail.com  Mon Dec 12 00:12:06 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Sun, 11 Dec 2005 15:12:06 -0800
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <439C7341.4050300@v.loewis.de>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de>
Message-ID: <bbaeab100512111512t7f363cc0tc04733d1364ab9a0@mail.gmail.com>

On 12/11/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Fredrik Lundh wrote:
[SNIP]
> >   - I check in an existing elementtree release in a separate location in
> >     the svn.python.org source tree. e.g.
> >
> >         svn.python.org/kits/elementtree-1.2.6-20050316
> >
> >     this will make it clear that this is external software, and it also
> >     provides a reference point for tracking down local changes
>
> Ah, so you want what CVS calls a "vendor branch": code that is
> externally maintained, and imported from time to time.
>
> Clearly, "local" (i.e. python.org) changes are one primary issue,
> so we should agree on an update process - I would personally prefer
> one that allows for merging (in the "svn merge" sense).
>
> The other issue is, of course, the question whose job it is to actually
> perform the updates. Would you expect to do that yourself, or would
> you expect somebody else does that?
>

I remember Barry saying he wanted to start a branch for work on the
next version of the 'email' package.  And it is possible more and more
modules developed externally will begin to be included in the stdlib. 
Perhaps PEP 2 should be updated with basic guidelines we plan to stick
to
for modules that are externally developed and occasionally synched
with the core.  Basically I think specifying who the code comes from,
having auto-assignment for bug reports in the tracker, and saying that
no updates to the snapshot except for bug fixes once alpha is released
should be enough.  I would assume the snapshot in svn would just be a
direct copy to the core and not require running any special script or
something to generate anything.  If we do go that way, then mentioning
that in the PEP wouldn't hurt either.

-Brett

From steven.bethard at gmail.com  Mon Dec 12 01:30:16 2005
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 11 Dec 2005 17:30:16 -0700
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <439CAEF6.4000304@zope.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
Message-ID: <d11dcfba0512111630s14c353f7me764ec83c7ea26ce@mail.gmail.com>

Jim Fulton wrote:
> Can we officially mark __private as a mistake. Perhaps:
>
> - Strongly discourage it in the style guide

+1

> - Mark it in the language reference as a deprecated feature

+1

> - Generate deprecation warnings when it is used?

-0

I don't see that this gains us much.  It will create annoyances for
people who don't want to update old code, and since most folks have to
search for the "feature" in the first place, if it's documented as
deprecated, hopefully they won't use it.

STeVe
--
You can wordify anything if you just verb it.
        --- Bucky Katt, Get Fuzzy

From ncoghlan at gmail.com  Mon Dec 12 02:16:55 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 12 Dec 2005 11:16:55 +1000
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439C51C8.2010909@zope.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
Message-ID: <439CCF87.9000405@gmail.com>

Jim Fulton wrote:
> FWIW, as a general rule, I like using a single trailing underscore,
> especially for keywords.  It allows the use of meaningful and easy
> to remember names.  When the name of a variable should be "class" or
> "for" or whatever, it's easy, as a Python programmer, to remember that
> I need to add a trailing _.  As a reformed abuser of single-character
> variable names, I've come to really hate abbreviations.  It's not only
> easier to use unabbreviated names, it's easier to remember them when
> reading code. (Note that ease of use hinges on editors that automate
> typeing of repeated names.)

FWIW, I believe scipy uses the trailing underscore to avoid shadowing certain 
builtins (type_, object_, str_, etc).

I thought it was ugly when I first encountered the convention, but the concept 
is growing on me. . .

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From skip at pobox.com  Mon Dec 12 02:53:03 2005
From: skip at pobox.com (skip@pobox.com)
Date: Sun, 11 Dec 2005 19:53:03 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439CA89B.4030600@colorstudy.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com>
Message-ID: <17308.55295.648767.433858@montanaro.dyndns.org>


    Ian> Do not use accessor methods, like ``obj.getFoo()`` and
    Ian> ``obj.setFoo(v)``, instead just expose a public attribute
    Ian> (``obj.foo``).  If necessary you can use ``property`` to implement
    Ian> the same functionality that accessor methods would give you.

Don't properties only work with new-style clsses?  If so, this should
probably be noted.

Skip

From nnorwitz at gmail.com  Mon Dec 12 02:52:36 2005
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sun, 11 Dec 2005 17:52:36 -0800
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <439CAEF6.4000304@zope.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
Message-ID: <ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>

On 12/11/05, Jim Fulton <jim at zope.com> wrote:
>
> Can we officially mark __private as a mistake. Perhaps:
>
> - Strongly discourage it in the style guide

This may be acceptable.

> - Mark it in the language reference as a deprecated feature
>
> - Generate deprecation warnings when it is used?
>    (This might be too much.)

I recently asked Guido about name mangling wrt Py3k.  He definitely
wanted to keep it in.  Unless he changed his mind, I doubt he would
deprecate it.  His rationale was that there needs to be a way to
handle name collision with multiple inheritance.

n

From tim.peters at gmail.com  Mon Dec 12 03:18:04 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 11 Dec 2005 21:18:04 -0500
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
Message-ID: <1f7befae0512111818x13ab3185w48bd365d0b5fe5f4@mail.gmail.com>

[Neal Norwitz]
> I recently asked Guido about name mangling wrt Py3k.  He definitely
> wanted to keep it in.  Unless he changed his mind, I doubt he would
> deprecate it.  His rationale was that there needs to be a way to
> handle name collision with multiple inheritance.

That wasn't quite it.  The original motivation was to help avoid name
collisions under inheritance period, and especially when writing a
base class intended for subclassing by other parties, such as most
mix-in classes.  For example, if your utility or mixin base class `A`
has a data member named `n`, nobody deriving from `A` dare name one of
their data members `n` too, and it's unreasonable to expect everyone
deriving from `A` to learn and avoid all the names `A` uses
internally.  It's even more unreasonable for A's author to have to
promise, after A's first release, never to change the name of, or
introduce any new, attribute (A's author dare not, lest the new name
conflict with a name someone else's subclass used).

If A's author names the attribute `__n` instead, all those problems go
away, provided only that some subclass doesn't also name itself `A`.

That was the only point to `__` name-mangling.  People who think it's
trying to, e.g., emulate C++'s `private` gimmick are enjoying a
semi-private fantasy ;-)  It works fine for its intended use.

From raymond.hettinger at verizon.net  Mon Dec 12 03:19:01 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun, 11 Dec 2005 21:19:01 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439CA89B.4030600@colorstudy.com>
Message-ID: <000d01c5fec2$6b19f160$cf26a044@oemcomputer>

> Do not use accessor methods, like ``obj.getFoo()`` and
> ``obj.setFoo(v)``, instead just expose a public attribute
(``obj.foo``).

This advice is, of course, not appropriate for all users (properties are
not typically in a Python beginner's skill set) or all use cases.  It is
closer to one person's view of the One-Right-Way(tm).  Opinions on
programming best practices vary widely, evolve over time, and may be
context dependent.



> > While, on some level, private variables seem attractive, I think
that
> > experience (for everyone I know) has shown them to be an attractive
> > nuisance.  I recommend discouraging them.
> 
> I really really hate double underscores

FWIW, I think we have no business dictating to others how they should
name their variables.  This is doubly true for a convention that has a
long history and built-in language support.

My preference is to leave PEP 8 for the minimum practices necessary for
one programmer to be able to read and maintain another programmer's
code.



Raymond


From ncoghlan at gmail.com  Mon Dec 12 03:29:01 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 12 Dec 2005 12:29:01 +1000
Subject: [Python-Dev] Deprecate __ private (was Re: PEP
	8	updates/clarifications)
In-Reply-To: <ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com>
	<439C51C8.2010909@zope.com>	<439CA89B.4030600@colorstudy.com>
	<439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
Message-ID: <439CE06D.70600@gmail.com>

Neal Norwitz wrote:
> On 12/11/05, Jim Fulton <jim at zope.com> wrote:
>> Can we officially mark __private as a mistake. Perhaps:
>>
>> - Strongly discourage it in the style guide
> 
> This may be acceptable.
> 
>> - Mark it in the language reference as a deprecated feature
>>
>> - Generate deprecation warnings when it is used?
>>    (This might be too much.)
> 
> I recently asked Guido about name mangling wrt Py3k.  He definitely
> wanted to keep it in.  Unless he changed his mind, I doubt he would
> deprecate it.  His rationale was that there needs to be a way to
> handle name collision with multiple inheritance.

Keeping it for Py3K would be fine, if the mechanism was changed so that it 
actually worked right. That is, the mechanics would be such that any two 
concurrently existing classes would be guaranteed to mangle the names of their 
private variables differently - simply using the class name (as now) doesn't 
guarantee that when inheriting from a class in a different module and reusing 
the name.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From cce at clarkevans.com  Mon Dec 12 03:47:48 2005
From: cce at clarkevans.com (Clark C. Evans)
Date: Sun, 11 Dec 2005 21:47:48 -0500
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <1f7befae0512111818x13ab3185w48bd365d0b5fe5f4@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<1f7befae0512111818x13ab3185w48bd365d0b5fe5f4@mail.gmail.com>
Message-ID: <20051212024748.GA69972@prometheusresearch.com>

Interesting discussion.  I've been thinking the opposite; that I should
start using __attribute more often for "undocumented, private" member
variables that are implementation details and clearly not part of the
public interface.

I'm curious what people have against it?

On Sun, Dec 11, 2005 at 09:18:04PM -0500, Tim Peters wrote:
| That wasn't quite it.  The original motivation was to help avoid name
| collisions under inheritance period, and especially when writing a
| base class intended for subclassing by other parties

...

| It's even more unreasonable for A's author to have to
| promise, after A's first release, never to change the name of, or
| introduce any new, attribute (A's author dare not, lest the new name
| conflict with a name someone else's subclass used).

About one year ago, I was updating a "shared module" that I wrote about
6-9 months prior.  I added a member variable, and a few days later one
of my applications started to mysteriously fail.  This was a bugger to
track down... name collision problem.

I've since become very sensitive about "from xx import *" as well,
for the same reason -- it tends to cause very nasty bugs when 
the module xx changes to introduce a few more methods, etc.

Best,

Clark

From jcarlson at uci.edu  Mon Dec 12 04:59:10 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sun, 11 Dec 2005 19:59:10 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <17308.55295.648767.433858@montanaro.dyndns.org>
References: <439CA89B.4030600@colorstudy.com>
	<17308.55295.648767.433858@montanaro.dyndns.org>
Message-ID: <20051211195752.CB4A.JCARLSON@uci.edu>


skip at pobox.com wrote:
>     Ian> Do not use accessor methods, like ``obj.getFoo()`` and
>     Ian> ``obj.setFoo(v)``, instead just expose a public attribute
>     Ian> (``obj.foo``).  If necessary you can use ``property`` to implement
>     Ian> the same functionality that accessor methods would give you.
> 
> Don't properties only work with new-style clsses?  If so, this should
> probably be noted.

In the future, aren't all classes going to become new-style?  Was it
going to wait until Py3k, or sometime sooner?

 - Josiah


From guido at python.org  Mon Dec 12 04:58:25 2005
From: guido at python.org (Guido van Rossum)
Date: Sun, 11 Dec 2005 19:58:25 -0800
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <439CE06D.70600@gmail.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<439CE06D.70600@gmail.com>
Message-ID: <ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com>

On 12/11/05, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Keeping it for Py3K would be fine, if the mechanism was changed so that it
> actually worked right. That is, the mechanics would be such that any two
> concurrently existing classes would be guaranteed to mangle the names of their
> private variables differently - simply using the class name (as now) doesn't
> guarantee that when inheriting from a class in a different module and reusing
> the name.

I know about the fear of accidental reuse of class names, but I don't
find it a compelling argument. Python encourages shallow class
hierarchies. It's easy to find all the base classes (look at __mro__).
It's unlikely that a hierarchy refactoring will introduce a new name
conflict after the fact.

Also, I like the current, well-defined mangling algorithm; it means
that when I'm in the debugger I can manually mangle or unmangle names
as required.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Mon Dec 12 05:07:32 2005
From: skip at pobox.com (skip@pobox.com)
Date: Sun, 11 Dec 2005 22:07:32 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <20051211195752.CB4A.JCARLSON@uci.edu>
References: <439CA89B.4030600@colorstudy.com>
	<17308.55295.648767.433858@montanaro.dyndns.org>
	<20051211195752.CB4A.JCARLSON@uci.edu>
Message-ID: <17308.63364.7416.216717@montanaro.dyndns.org>


    >> Don't properties only work with new-style clsses?  If so, this should
    >> probably be noted.

    Josiah> In the future, aren't all classes going to become new-style?

Sure, but PEP 8 should be accurate for current Python. <0.5 wink>

    Josiah> Was it going to wait until Py3k, or sometime sooner?

Dunno.

Skip

From ncoghlan at gmail.com  Mon Dec 12 05:06:56 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 12 Dec 2005 14:06:56 +1000
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <20051211195752.CB4A.JCARLSON@uci.edu>
References: <439CA89B.4030600@colorstudy.com>	<17308.55295.648767.433858@montanaro.dyndns.org>
	<20051211195752.CB4A.JCARLSON@uci.edu>
Message-ID: <439CF760.8040408@gmail.com>

Josiah Carlson wrote:
> skip at pobox.com wrote:
>>     Ian> Do not use accessor methods, like ``obj.getFoo()`` and
>>     Ian> ``obj.setFoo(v)``, instead just expose a public attribute
>>     Ian> (``obj.foo``).  If necessary you can use ``property`` to implement
>>     Ian> the same functionality that accessor methods would give you.
>>
>> Don't properties only work with new-style clsses?  If so, this should
>> probably be noted.
> 
> In the future, aren't all classes going to become new-style?  Was it
> going to wait until Py3k, or sometime sooner?

Going the Java route (no implicit base class) would be an interim step along 
that road (i.e., a release or two where there is no default __metaclass__ 
fallback).

Any old code could be fixed by putting "from types import ClassType as 
__metaclass__" at the top of the affected modules.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From guido at python.org  Mon Dec 12 05:15:13 2005
From: guido at python.org (Guido van Rossum)
Date: Sun, 11 Dec 2005 20:15:13 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439CF760.8040408@gmail.com>
References: <439CA89B.4030600@colorstudy.com>
	<17308.55295.648767.433858@montanaro.dyndns.org>
	<20051211195752.CB4A.JCARLSON@uci.edu> <439CF760.8040408@gmail.com>
Message-ID: <ca471dc20512112015r43b24115oa7ac678819c71a15@mail.gmail.com>

On 12/11/05, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Josiah Carlson wrote:
> > skip at pobox.com wrote:
> >>     Ian> Do not use accessor methods, like ``obj.getFoo()`` and
> >>     Ian> ``obj.setFoo(v)``, instead just expose a public attribute
> >>     Ian> (``obj.foo``).  If necessary you can use ``property`` to implement
> >>     Ian> the same functionality that accessor methods would give you.
> >>
> >> Don't properties only work with new-style clsses?  If so, this should
> >> probably be noted.
> >
> > In the future, aren't all classes going to become new-style?  Was it
> > going to wait until Py3k, or sometime sooner?
>
> Going the Java route (no implicit base class) would be an interim step along
> that road (i.e., a release or two where there is no default __metaclass__
> fallback).
>
> Any old code could be fixed by putting "from types import ClassType as
> __metaclass__" at the top of the affected modules.

I'm not sure what you are proposing and I'm not sure what problem you
are trying to solve.

The plan for new-style vs. classic classes is simple and doesn't need
to change (IMO): until Py3k, the status quo will remain; in Py3k,
there is only new-style (except if you use a custom metaclass).

(That said, I'm all for exceptions becoming new-style in 2.5.)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From s.joaopaulo at gmail.com  Mon Dec 12 05:27:31 2005
From: s.joaopaulo at gmail.com (=?ISO-8859-1?Q?Jo=E3o_Paulo_Silva?=)
Date: Mon, 12 Dec 2005 01:27:31 -0300
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <787073ca0512112025x4f1331e9i@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<1f7befae0512111818x13ab3185w48bd365d0b5fe5f4@mail.gmail.com>
	<20051212024748.GA69972@prometheusresearch.com>
	<787073ca0512112025x4f1331e9i@mail.gmail.com>
Message-ID: <787073ca0512112027y28ef5060h@mail.gmail.com>

Hi All,

I think that a big problem is that there isn't an obvious way to say:
self.a is part of the class interface, self.b isn't. Or: you can
override self._c to do that. I believe we really need a way to do
these things more clear.
Currently we can use methods and properties, but even this is not
clear enough in a inheritance tree.

--
At? mais..
Jo?o Paulo da Silva
LinuxUser #355914
ICQ: 265770691 | Jabber: joaopinga at jabber.org

From s.joaopaulo at gmail.com  Mon Dec 12 05:31:45 2005
From: s.joaopaulo at gmail.com (=?ISO-8859-1?Q?Jo=E3o_Paulo_Silva?=)
Date: Mon, 12 Dec 2005 01:31:45 -0300
Subject: [Python-Dev] Exception type on handling closed files
Message-ID: <787073ca0512112031v232a5d12s@mail.gmail.com>

Look:

>>> a = file("dir/foo")
>>> a.close()
>>> a.read()

Traceback (most recent call last):
  File "<pyshell#28>", line 1, in -toplevel-
    a.read()
ValueError: I/O operation on closed file

Shoudn't this raise IOError? Seems more semantically correct to me.

--
Jo?o Paulo da Silva
LinuxUser #355914
ICQ: 265770691 | Jabber: joaopinga at jabber.org

From steven.bethard at gmail.com  Mon Dec 12 05:53:26 2005
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 11 Dec 2005 21:53:26 -0700
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<439CE06D.70600@gmail.com>
	<ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com>
Message-ID: <d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com>

On 12/11/05, Guido van Rossum <guido at python.org> wrote:
> On 12/11/05, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > Keeping it for Py3K would be fine, if the mechanism was changed so that it
> > actually worked right. That is, the mechanics would be such that any two
> > concurrently existing classes would be guaranteed to mangle the names of their
> > private variables differently - simply using the class name (as now) doesn't
> > guarantee that when inheriting from a class in a different module and reusing
> > the name.
>
> I know about the fear of accidental reuse of class names, but I don't
> find it a compelling argument.

FWIW, I know I currently have a number of classes that are potentially
hazardous in this way.  Each of these classes is basically a
substitute class for a third-party API that I have to code to.  The
API is missing a number of convenience methods, and the most
straightforward way to introduce these methods[1] is to create a
subclass of the appropriate class.  Since they are in a different
module, it seems perfectly normal for me to give them the same name
since for all external modules, they should look the same as the
original API (but with the added methods).  So I have a number of
classes that look something like:

class Document(_cdm.Document):
    ...
    # add convenience methods here
    ...

I don't use double-underscore name mangling, but if I did, it would
clearly fail in this case.

[1] I've concluded this after a variety of refactorings.  But perhaps
there is a better way...

> Also, I like the current, well-defined mangling algorithm; it means
> that when I'm in the debugger I can manually mangle or unmangle names
> as required.

Why couldn't the name mangling do something like:

    '_%s_%s__%s' % (cls.__module__, cls.__name__, attrname)

This would still allow manual mangling/unmangling, and it seems like
it would cover most of the same-name different module concerns...

STeVe
--
You can wordify anything if you just verb it.
        --- Bucky Katt, Get Fuzzy

From jeremy at alum.mit.edu  Mon Dec 12 06:17:25 2005
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 12 Dec 2005 00:17:25 -0500
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <dni8qe$5eq$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
Message-ID: <e8bf7a530512112117r6ecd15b1o7efe387633917cff@mail.gmail.com>

On 12/11/05, Fredrik Lundh <fredrik at pythonware.com> wrote:

> > Overall, sounds like a good plan.
>
> Just say "go", and I'll start working on this.

Are you still waiting for someone to say go?  I'm not sure what
responsible party should say it; if I'm not the right person, would
the right person please say "go."

Jeremy

From martin at v.loewis.de  Mon Dec 12 07:40:45 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 12 Dec 2005 07:40:45 +0100
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <dni8qe$5eq$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de><dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de>
	<dni8qe$5eq$1@sea.gmane.org>
Message-ID: <439D1B6D.9080208@v.loewis.de>

Fredrik Lundh wrote:
> Exactly.  But I'm not sure "branch" is really accurate here; it's more like
> "snapshot".  Stable releases are added to the "vendor" tree, and relevant
> files are are then copied to the appropriate location in the release tree.

In practice, it will be a branch - unless you want to completely rule
out modifications (which you didn't).

>>The other issue is, of course, the question whose job it is to actually
>>perform the updates. Would you expect to do that yourself, or would
>>you expect somebody else does that?
> 
> 
> I can deal with this.

Sounds good.

> Since all the relevant module names start with "Element", putting it directly
> under xml wouldn't be too bad.  But an xml subpackage is better, and prior
> art says "etree".

So etree it is.

> I think that limiting this to ElementTree, ElementPath, and perhaps Element-
> Include would be a good start.

Ok.

> And maybe PEP 291 could be updated to cover both compatibility with older
> Python versions and other compatibility issues.

So what would be the minimum Python version you keep compatibility with?

> (one way to do this would be to add an "function pointer table" to pyexpat
> that contains pointers to selected portions of the expat API, and then add
> an indirection level to cElementTree)

Ok, this sounds like a larger piece of work.

> Just say "go", and I'll start working on this.

Not sure if it is me who should say that; as nobody else
has spoken against it: go.

Regards,
Martin

From fredrik at pythonware.com  Mon Dec 12 08:32:47 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 12 Dec 2005 08:32:47 +0100
Subject: [Python-Dev] Exception type on handling closed files
References: <787073ca0512112031v232a5d12s@mail.gmail.com>
Message-ID: <dnj931$eto$1@sea.gmane.org>

João Paulo Silva wrote:

> >>> a = file("dir/foo")
> >>> a.close()
> >>> a.read()
>
> Traceback (most recent call last):
>   File "<pyshell#28>", line 1, in -toplevel-
>     a.read()
> ValueError: I/O operation on closed file
>
> Shoudn't this raise IOError? Seems more semantically correct to me.

IOError is, as the documentation says, used "when an I/O operation fails
for an I/O related reason", while ValueError is used "when an argument has
the right type but an inappropriate value."

</F>




From fredrik at pythonware.com  Mon Dec 12 08:51:27 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 12 Dec 2005 08:51:27 +0100
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de><dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de><dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de>
Message-ID: <dnja60$h84$1@sea.gmane.org>

Martin v. Löwis wrote

> > And maybe PEP 291 could be updated to cover both compatibility with older
> > Python versions and other compatibility issues.
>
> So what would be the minimum Python version you keep compatibility with?

as Brett pointed out, the procedure to use for externally developed and
bundled components should be described in PEP 2.  but we need to list
them somewhere too; PEP 291 is as good as any other place.

> > Just say "go", and I'll start working on this.
>
> Not sure if it is me who should say that; as nobody else
> has spoken against it: go.

just one question: where do you want the "vendor" checkins ?  I'm using
a flat "kits" namespace in my own repositories, e.g.

    http://svn.python.org/kits/elementtree-1.2.6-20050316
    http://svn.python.org/kits/jpeg-6b
    http://svn.python.org/kits/zlib-1.2.1

or, as commands for this specific case:

    $ wget http://effbot.org/downloads/elementtree-1.2.6-20050316.tar.gz
    $ tar xvfz elementtree-1.2.6-20050316.tar.gz
    $ svn import elementtree-1.2.6-20050316
        svn+ssh://pythondev at svn.python.org/kits/elementtree-1.2.6-20050316

anyone has a better name?

</F>




From ncoghlan at gmail.com  Mon Dec 12 09:14:07 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 12 Dec 2005 18:14:07 +1000
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <ca471dc20512112015r43b24115oa7ac678819c71a15@mail.gmail.com>
References: <439CA89B.4030600@colorstudy.com>	
	<17308.55295.648767.433858@montanaro.dyndns.org>	
	<20051211195752.CB4A.JCARLSON@uci.edu> <439CF760.8040408@gmail.com>
	<ca471dc20512112015r43b24115oa7ac678819c71a15@mail.gmail.com>
Message-ID: <439D314F.9070706@gmail.com>

Guido van Rossum wrote:
> On 12/11/05, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Josiah Carlson wrote:
>>> skip at pobox.com wrote:
>>>>     Ian> Do not use accessor methods, like ``obj.getFoo()`` and
>>>>     Ian> ``obj.setFoo(v)``, instead just expose a public attribute
>>>>     Ian> (``obj.foo``).  If necessary you can use ``property`` to implement
>>>>     Ian> the same functionality that accessor methods would give you.
>>>>
>>>> Don't properties only work with new-style clsses?  If so, this should
>>>> probably be noted.
>>> In the future, aren't all classes going to become new-style?  Was it
>>> going to wait until Py3k, or sometime sooner?
>> Going the Java route (no implicit base class) would be an interim step along
>> that road (i.e., a release or two where there is no default __metaclass__
>> fallback).
>>
>> Any old code could be fixed by putting "from types import ClassType as
>> __metaclass__" at the top of the affected modules.
> 
> I'm not sure what you are proposing and I'm not sure what problem you
> are trying to solve.

I'm accustomed to handling major semantic changes in an API by deprecating the 
API first, then later bringing it back with the new semantics. A sharp cutover 
to new semantics (even in a version advertised as backwards incompatible) 
makes me nervous :)

> The plan for new-style vs. classic classes is simple and doesn't need
> to change (IMO): until Py3k, the status quo will remain; in Py3k,
> there is only new-style (except if you use a custom metaclass).

The problem I have with the currently planned sharp cutover is that the errors 
caused by the change are not necessarily easy to predict, causing difficulties 
with managing that transition. Tracking down whether or not the change to 
new-style classes is the cause of a given Py3k migration problem could be 
difficult.

Code can be future-proofed by instituting one of three rules:
   1. Always inherit from something (enforcable via "__metaclass__ = None")
   2. Always use new-style classes by default (via "__metaclass__ = type")
   3. Always use old-style classes by default
       (via "from types import ClassType as __metaclass__")

One way to make this migration easier to manage would be to have the class 
creation code check __builtins__ for a definition of __metaclass__. This would 
make it possible for application developers to determine whether or not their 
application or any of its support libraries are dependent on certain classes 
being old-style (by running the program and changing the default metaclass via 
"__builtins__.__metaclass_ = None" or "__builtins__.__metaclass_ = type").

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From steve at holdenweb.com  Mon Dec 12 09:36:37 2005
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 12 Dec 2005 08:36:37 +0000
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <1f7befae0512111818x13ab3185w48bd365d0b5fe5f4@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com>
	<439C51C8.2010909@zope.com>	<439CA89B.4030600@colorstudy.com>
	<439CAEF6.4000304@zope.com>	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<1f7befae0512111818x13ab3185w48bd365d0b5fe5f4@mail.gmail.com>
Message-ID: <439D3695.3020509@holdenweb.com>

Tim Peters wrote:
> [Neal Norwitz]
> 
>>I recently asked Guido about name mangling wrt Py3k.  He definitely
>>wanted to keep it in.  Unless he changed his mind, I doubt he would
>>deprecate it.  His rationale was that there needs to be a way to
>>handle name collision with multiple inheritance.
> 
> 
> That wasn't quite it.  The original motivation was to help avoid name
> collisions under inheritance period, and especially when writing a
> base class intended for subclassing by other parties, such as most
> mix-in classes.  For example, if your utility or mixin base class `A`
> has a data member named `n`, nobody deriving from `A` dare name one of
> their data members `n` too, and it's unreasonable to expect everyone
> deriving from `A` to learn and avoid all the names `A` uses
> internally.  It's even more unreasonable for A's author to have to
> promise, after A's first release, never to change the name of, or
> introduce any new, attribute (A's author dare not, lest the new name
> conflict with a name someone else's subclass used).
> 
> If A's author names the attribute `__n` instead, all those problems go
> away, provided only that some subclass doesn't also name itself `A`.
> 
> That was the only point to `__` name-mangling.  People who think it's
> trying to, e.g., emulate C++'s `private` gimmick are enjoying a
> semi-private fantasy ;-)  It works fine for its intended use.

In that case it would seem to make even *more* sense, theoretically, to 
replace the class name in mangled names with a GUID, hence avoiding 
collisions in similarly-named subclasses.

Then it would work even finer (though the mangled names would be longer, 
and less meaningful in debugging).

mangling-things-by-typing-them-since-1967-ly y'rs  - steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From fredrik at pythonware.com  Mon Dec 12 11:49:46 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 12 Dec 2005 11:49:46 +0100
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de><dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de><dni8qe$5eq$1@sea.gmane.org><439D1B6D.9080208@v.loewis.de>
	<dnja60$h84$1@sea.gmane.org>
Message-ID: <dnjkkd$efu$1@sea.gmane.org>

Fredrik Lundh wrote:

> just one question: where do you want [to put] the "vendor" checkins ?  I'm using
> a flat "kits" namespace in my own repositories, e.g.

> anyone has a better name?

anyone ?

</F>




From fredrik at pythonware.com  Mon Dec 12 12:14:33 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 12 Dec 2005 12:14:33 +0100
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de><dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de><dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de>
Message-ID: <dnjm2r$j3g$1@sea.gmane.org>

Martin v. Löwis wrote:

> > Since all the relevant module names start with "Element", putting it directly
> > under xml wouldn't be too bad.  But an xml subpackage is better, and prior
> > art says "etree".
>
> So etree it is.

I just realized that the prior art (lxml.etree) uses etree as an alias for the
ElementTree module, not as a package name.  this means that to import the
core Element type, you'd do:

    # classic ET
    from elementtree.ElementTree import Element

or

    # bundled ET
    from xml.etree.ElementTree import Element

or

    # libxml-powered ET
    from lxml.etree import Element

or

    # accelerated ET
    from cElementTree import Element

I'm not sure if this really is a problem.  better explicit than implicit, as PyXML
has shown us.  if people want to be able to rapidly switch between versions,
they can always use from-import or import-as.

</F>




From jim at zope.com  Mon Dec 12 12:24:22 2005
From: jim at zope.com (Jim Fulton)
Date: Mon, 12 Dec 2005 06:24:22 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <17308.55295.648767.433858@montanaro.dyndns.org>
References: <4399F967.3080300@colorstudy.com>
	<439C51C8.2010909@zope.com>	<439CA89B.4030600@colorstudy.com>
	<17308.55295.648767.433858@montanaro.dyndns.org>
Message-ID: <439D5DE6.6090305@zope.com>

skip at pobox.com wrote:
>     Ian> Do not use accessor methods, like ``obj.getFoo()`` and
>     Ian> ``obj.setFoo(v)``, instead just expose a public attribute
>     Ian> (``obj.foo``).  If necessary you can use ``property`` to implement
>     Ian> the same functionality that accessor methods would give you.
> 
> Don't properties only work with new-style clsses?  If so, this should
> probably be noted.

Read properties work with old-style classes.  Write properties require
old-stype classes.  I'm always forgetting this for some reason.  Yes, it
should be noted.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From steve at holdenweb.com  Mon Dec 12 12:22:08 2005
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 12 Dec 2005 11:22:08 +0000
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <dnjkkd$efu$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de><dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de><dni8qe$5eq$1@sea.gmane.org><439D1B6D.9080208@v.loewis.de>	<dnja60$h84$1@sea.gmane.org>
	<dnjkkd$efu$1@sea.gmane.org>
Message-ID: <dnjmh2$k0n$1@sea.gmane.org>

Fredrik Lundh wrote:
> Fredrik Lundh wrote:
> 
> 
>>just one question: where do you want [to put] the "vendor" checkins ?  I'm using
>>a flat "kits" namespace in my own repositories, e.g.
> 
> 
>>anyone has a better name?
> 
> 
> anyone ?
> 
How about "independent" to highlight the fact that development takes 
place elsewhere? Or "external"?

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From noamraph at gmail.com  Mon Dec 12 12:22:52 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Mon, 12 Dec 2005 13:22:52 +0200
Subject: [Python-Dev] A missing piece of information in weakref documentation
Message-ID: <b348a0850512120322q10440029t62ee52df6e0c3c57@mail.gmail.com>

Hello,

I now discovered that a callback registered when creating a weak
reference will be called only if the weak reference object is still
alive. This is not documented in the weakref module documentation.

(It's a good behaviour - it just ought to be documented.)

Have a good day,
Noam

From jim at zope.com  Mon Dec 12 12:33:38 2005
From: jim at zope.com (Jim Fulton)
Date: Mon, 12 Dec 2005 06:33:38 -0500
Subject: [Python-Dev] Deprecate __ private (was Re: PEP
	8	updates/clarifications)
In-Reply-To: <1f7befae0512111818x13ab3185w48bd365d0b5fe5f4@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com>
	<439C51C8.2010909@zope.com>	<439CA89B.4030600@colorstudy.com>
	<439CAEF6.4000304@zope.com>	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<1f7befae0512111818x13ab3185w48bd365d0b5fe5f4@mail.gmail.com>
Message-ID: <439D6012.4060609@zope.com>

Tim Peters wrote:
> [Neal Norwitz]
> 

...

> That was the only point to `__` name-mangling.  People who think it's
> trying to, e.g., emulate C++'s `private` gimmick are enjoying a
> semi-private fantasy ;-)  It works fine for its intended use.

In theory, I agree.

In practice, I don't agree that it works fine.  Inevitably, someone
finds a need to access a "private" variable in a subclass.  Or
even in the original class, you find some need to use something like
__getattr__ where the implicit name mangling doesn't come into play
and you have to emulate the name mangling.  Or perhaps someone wants
to examine the value of one of these variables in the debugger.
In my experience, almost every time someone uses the __private
trick, they or someone else comes to regret it.

OTOH, explicit name mangling provides the benefits of implicit
name mangling without it's drawbacks.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From jim at zope.com  Mon Dec 12 12:41:39 2005
From: jim at zope.com (Jim Fulton)
Date: Mon, 12 Dec 2005 06:41:39 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <000d01c5fec2$6b19f160$cf26a044@oemcomputer>
References: <000d01c5fec2$6b19f160$cf26a044@oemcomputer>
Message-ID: <439D61F3.6010608@zope.com>

Raymond Hettinger wrote:
>>Do not use accessor methods, like ``obj.getFoo()`` and
>>``obj.setFoo(v)``, instead just expose a public attribute
> 
> (``obj.foo``).
> 
> This advice is, of course, not appropriate for all users (properties are
> not typically in a Python beginner's skill set)

Really? In any case, properties are only needed if you change
your mind about the implementation.  In my experience, they are
rarely needed.

 > or all use cases.

I think the advice gave a very narrow case, which was when you were going
to write trivial accessors.

 >  It is
> closer to one person's view of the One-Right-Way(tm).  Opinions on
> programming best practices vary widely, evolve over time, and may be
> context dependent.

I thought I was reflecting more than just my opinion.
Also, the original text had just as strong an admonition --
one that, as I mentioned, seem to be out of line with
current thinking.

...

>>>experience (for everyone I know) has shown them to be an attractive
>>>nuisance.  I recommend discouraging them.
>>
>>I really really hate double underscores
> 
> 
> FWIW, I think we have no business dictating to others how they should
> name their variables.  This is doubly true for a convention that has a
> long history and built-in language support.

Even if, experience with a practice has shown it to be highly
problematic?

> My preference is to leave PEP 8 for the minimum practices necessary for
> one programmer to be able to read and maintain another programmer's
> code.

I'm for making the style guide smaller.  I do think it offers too
much advice in places.  Although I'm not sure we could all agree
om what those places are. :)

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From hyeshik at gmail.com  Mon Dec 12 13:08:17 2005
From: hyeshik at gmail.com (Hye-Shik Chang)
Date: Mon, 12 Dec 2005 21:08:17 +0900
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <dnjkkd$efu$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de> <dnja60$h84$1@sea.gmane.org>
	<dnjkkd$efu$1@sea.gmane.org>
Message-ID: <4f0b69dc0512120408p21a0c194r41e6d1318764422d@mail.gmail.com>

On 12/12/05, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Fredrik Lundh wrote:
>
> > just one question: where do you want [to put] the "vendor" checkins ?  I'm using
> > a flat "kits" namespace in my own repositories, e.g.
>
> > anyone has a better name?
>
> anyone ?
>

I think "contrib" is somewhat conventional for the purpose.

Hye-Shik

From steve at holdenweb.com  Mon Dec 12 13:39:08 2005
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 12 Dec 2005 12:39:08 +0000
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <4f0b69dc0512120408p21a0c194r41e6d1318764422d@mail.gmail.com>
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de>
	<dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de>
	<dni8qe$5eq$1@sea.gmane.org>	<439D1B6D.9080208@v.loewis.de>
	<dnja60$h84$1@sea.gmane.org>	<dnjkkd$efu$1@sea.gmane.org>
	<4f0b69dc0512120408p21a0c194r41e6d1318764422d@mail.gmail.com>
Message-ID: <439D6F6C.8050903@holdenweb.com>

Hye-Shik Chang wrote:
> On 12/12/05, Fredrik Lundh <fredrik at pythonware.com> wrote:
> 
>>Fredrik Lundh wrote:
>>
>>
>>>just one question: where do you want [to put] the "vendor" checkins ?  I'm using
>>>a flat "kits" namespace in my own repositories, e.g.
>>
>>>anyone has a better name?
>>
>>anyone ?
>>
> 
> 
> I think "contrib" is somewhat conventional for the purpose.
> 
Indeed, but conventionally *all* code in the Python core is contributed, 
and I think we need a name that differentiates externally-maintained 
packages from the contributions that are integrated into the core and 
maintained as part of it.

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From fredrik at pythonware.com  Mon Dec 12 13:54:30 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 12 Dec 2005 13:54:30 +0100
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de><dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de><dni8qe$5eq$1@sea.gmane.org>	<439D1B6D.9080208@v.loewis.de><dnja60$h84$1@sea.gmane.org>	<dnjkkd$efu$1@sea.gmane.org><4f0b69dc0512120408p21a0c194r41e6d1318764422d@mail.gmail.com>
	<439D6F6C.8050903@holdenweb.com>
Message-ID: <dnjru6$5pq$1@sea.gmane.org>

Steve Holden wrote:

>>>anyone ?
>>
>> I think "contrib" is somewhat conventional for the purpose.
>>
> Indeed, but conventionally *all* code in the Python core is contributed,
> and I think we need a name that differentiates externally-maintained
> packages from the contributions that are integrated into the core and
> maintained as part of it.

I'm leaning towards a flat "external" directory at the top of the SVN tree.  no tags or
trunk stuff; just the snapshots (plus a README file that explains what's in there).

If nobody stops me (Martin?), I'll set this up later today...

</F> 




From rhamph at gmail.com  Mon Dec 12 13:49:10 2005
From: rhamph at gmail.com (Adam Olsen)
Date: Mon, 12 Dec 2005 05:49:10 -0700
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<439CE06D.70600@gmail.com>
	<ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com>
	<d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com>
Message-ID: <aac2c7cb0512120449g1954e99fk65d24f3bc51cd428@mail.gmail.com>

[Quotations deleted since I'm not replying to anything directly]

When I need an identifier unique to a class I just use a reference to
the class itself.  As such I'd like to suggest that
obj.__private
be converted to
obj.__dict__[(type(obj), '__private')]

Note that I'm accessing __dict__ directly so as to avoid getattr's
requirement for attribute names to be strings.

Obviously it doesn't handle backwards compatibility, so it's more of a
"if I could do it again.." suggestion.

--
Adam Olsen, aka Rhamphoryncus

From hoffman at ebi.ac.uk  Mon Dec 12 14:32:31 2005
From: hoffman at ebi.ac.uk (Michael Hoffman)
Date: Mon, 12 Dec 2005 13:32:31 +0000
Subject: [Python-Dev] Directory for packages maintained outside the core
 (was Re: ElementTree - Why not part of the core?)
In-Reply-To: <439D6F6C.8050903@holdenweb.com>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de> <dnja60$h84$1@sea.gmane.org>
	<dnjkkd$efu$1@sea.gmane.org>
	<4f0b69dc0512120408p21a0c194r41e6d1318764422d@mail.gmail.com>
	<439D6F6C.8050903@holdenweb.com>
Message-ID: <Pine.LNX.4.62.0512121329490.23169@qnzvnan.rov.np.hx>

[Hye-Shik Chang]
>> I think "contrib" is somewhat conventional for the purpose.

[Steve Holden]
> Indeed, but conventionally *all* code in the Python core is contributed,
> and I think we need a name that differentiates externally-maintained
> packages from the contributions that are integrated into the core and
> maintained as part of it.

The same could be said of a lot of other projects that use the
"contrib" convention. I have a much better idea of what "contrib"
means than "kits" or "external."
-- 
Michael Hoffman <hoffman at ebi.ac.uk>
European Bioinformatics Institute


From andrew-pythondev at puzzling.org  Mon Dec 12 14:48:23 2005
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Tue, 13 Dec 2005 00:48:23 +1100
Subject: [Python-Dev] Directory for packages maintained outside the core
	(was Re: ElementTree - Why not part of the core?)
In-Reply-To: <Pine.LNX.4.62.0512121329490.23169@qnzvnan.rov.np.hx>
References: <43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de> <dnja60$h84$1@sea.gmane.org>
	<dnjkkd$efu$1@sea.gmane.org>
	<4f0b69dc0512120408p21a0c194r41e6d1318764422d@mail.gmail.com>
	<439D6F6C.8050903@holdenweb.com>
	<Pine.LNX.4.62.0512121329490.23169@qnzvnan.rov.np.hx>
Message-ID: <20051212134823.GA29948@home.puzzling.org>

On Mon, Dec 12, 2005 at 01:32:31PM +0000, Michael Hoffman wrote:
> [Hye-Shik Chang]
> >> I think "contrib" is somewhat conventional for the purpose.
> 
> [Steve Holden]
> > Indeed, but conventionally *all* code in the Python core is contributed,
> > and I think we need a name that differentiates externally-maintained
> > packages from the contributions that are integrated into the core and
> > maintained as part of it.
> 
> The same could be said of a lot of other projects that use the
> "contrib" convention. I have a much better idea of what "contrib"
> means than "kits" or "external."

I have a much better idea of what "contrib" means than "external", but it's the
wrong idea :)

"contrib" implies to me things that are not really a core part of the project
(just extras that may perhaps be of use to someone), and so they haven't
received the same quality control or integration (e.g. I wouldn't expect to find
documentation for it in the standard library reference).  Of course, I'm
thinking of "contrib" directories in tar.gz files when I think this, but if I
saw a contrib directory in SVN (without having seen this mailing list thread),
I'd probably assume the same of it.

"external" is much clearer to me, and has a clear parallel with SVN's
"svn:external" feature.  Either way, a simple README.txt in the directory could
explain things adequately.

-Andrew.


From Scott.Daniels at Acm.Org  Mon Dec 12 15:02:31 2005
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Mon, 12 Dec 2005 06:02:31 -0800
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <439CAEF6.4000304@zope.com>
References: <4399F967.3080300@colorstudy.com>
	<439C51C8.2010909@zope.com>	<439CA89B.4030600@colorstudy.com>
	<439CAEF6.4000304@zope.com>
Message-ID: <dnjvta$jem$1@sea.gmane.org>

Jim Fulton wrote:
> Ian Bicking wrote:
>> Jim Fulton wrote:
>>> Ian Bicking wrote:
>>>>        Private attributes should have two leading underscores, no
>>>>        trailing underscores.
>>>>
>>>> This conflicts with a previous suggestion "Generally, double leading 
>>>> underscores should be used only to avoid name conflicts with 
>>>> attributes in classes designed to be subclassed."  Or perhaps 
>>>> "private attributes" needs to be better explained.
...
>> I really really hate double underscores, but I thought I'd let some 
>> other people suggest stronger language first....
> 
> Can we officially mark __private as a mistake. Perhaps:
> - Strongly discourage it in the style guide
> - Mark it in the language reference as a deprecated feature
> - Generate deprecation warnings when it is used?
>    (This might be too much.)

Perhaps "The __ name convention is designed for 'mixins'; as a means of
enforcing "private" it is both ineffective and annoying.  For example,
distutils.msvccompiler uses a bunch of instance variables which would I
would like to access in a subclass, but are "unavailable" because the
author could not imagine why I would need them.

--Scott David Daniels
Scott.Daniels at Acm.Org


From jim at zope.com  Mon Dec 12 15:28:33 2005
From: jim at zope.com (Jim Fulton)
Date: Mon, 12 Dec 2005 09:28:33 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134395316.11373.12.camel@localhost>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>	
	<439CA89B.4030600@colorstudy.com>
	<1134395316.11373.12.camel@localhost>
Message-ID: <439D8911.1010505@zope.com>

Gustavo J. A. M. Carneiro wrote:
...

>   IMHO, if getting a property involves a potentially long computation,
> it's better to have an accessor method rather than a property;
> xpto.getFoobar() hints right away the programmer that to access that
> value some code has to be run, so the programmer is more likely to store
> the result in a temp variable for use in the same context, instead of
> calling it multiple times.  Similar reasoning applites for setter vs
> property =.

That's why, in my suggested writeup, I suggested that attributes should
be used if the accessors are trivial.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From jeremy at alum.mit.edu  Mon Dec 12 15:33:54 2005
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 12 Dec 2005 09:33:54 -0500
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <dnjvta$jem$1@sea.gmane.org>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<dnjvta$jem$1@sea.gmane.org>
Message-ID: <e8bf7a530512120633j318bd18ft2d057781534c2f34@mail.gmail.com>

On 12/12/05, Scott David Daniels <Scott.Daniels at acm.org> wrote:
> Jim Fulton wrote:
> > Ian Bicking wrote:
> >> Jim Fulton wrote:
> >>> Ian Bicking wrote:
> >>>>        Private attributes should have two leading underscores, no
> >>>>        trailing underscores.
> >>>>
> >>>> This conflicts with a previous suggestion "Generally, double leading
> >>>> underscores should be used only to avoid name conflicts with
> >>>> attributes in classes designed to be subclassed."  Or perhaps
> >>>> "private attributes" needs to be better explained.
> ...
> >> I really really hate double underscores, but I thought I'd let some
> >> other people suggest stronger language first....
> >
> > Can we officially mark __private as a mistake. Perhaps:
> > - Strongly discourage it in the style guide
> > - Mark it in the language reference as a deprecated feature
> > - Generate deprecation warnings when it is used?
> >    (This might be too much.)
>
> Perhaps "The __ name convention is designed for 'mixins'; as a means of
> enforcing "private" it is both ineffective and annoying.  For example,
> distutils.msvccompiler uses a bunch of instance variables which would I
> would like to access in a subclass, but are "unavailable" because the
> author could not imagine why I would need them.

These are really separate issues, right?  The effect of __ names is to
make a variable private to a class, because it's a right pain to
access it from any other class.  If you design a class for inheritance
and use __ names, you're deciding to keep the details of those names
private.

There is a separate question about whether the designer of
msvccompiler made the right choices about which instance variables
were private.  This issue is really separate from the naming
mechanism.  If the designer of the class didn't intent to make those
instance variables available to you, it's not the language's fault.

There are ways the language and tools could make things easier for
developers.  The debugger could know how to mangle names for us.  It
would be great to have editors/ides that could rename all the
variables if we decide to change the name to make it available to
subclasses.  I think these problems are the primary reasons I dislike
mangled names.  I can't remember how to type the names in pdb and it's
a pain to change every use of the name if I change from __var to _var.
 C++ private variables don't suffer from either of these problems. 
The visibility is separate from the name; if I change an instance
variable from private to protected, I don't have to edit existing code
to track a new name.

Jeremy

From gjc at inescporto.pt  Mon Dec 12 14:48:36 2005
From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro)
Date: Mon, 12 Dec 2005 13:48:36 +0000
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439CA89B.4030600@colorstudy.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com>
Message-ID: <1134395316.11373.12.camel@localhost>

Dom, 2005-12-11 ?s 16:30 -0600, Ian Bicking escreveu:
> Jim Fulton wrote:
> >>      Designing for inheritance
> >>
> >>        Always decide whether a class's methods and instance variables
> >>        should be public or non-public.  In general, never make data
> >>        variables public unless you're implementing essentially a
> >>        record.   It's almost always preferrable to give a functional
> > 
> >  >        interface to your class instead (and some Python 2.2
> >  >        developments will make this much nicer).
> >  >
> >  > Yes, Python 2.2 developments have made this better.  Use of property()
> >  > should be suggested.
> > 
> > This seems outdated.  My impression, in part from time spent
> > working with the Python Labs guys, is that it is fine to have public
> > data sttributes even for non-"record" types.  In fact, I would argue that
> > any time you would be tempted to provide "getFoo()" and "setFoo(v)"
> > for some "private attribute _foo", it would be better to make it
> > public.  I certainly find "blah.foo" and "blah.foo = v" to be much
> > better than "blah.getFoo()" and blah.setFoo(v)".
> > 
> > Certainly, properties provide a safety belt.  I would argue it this
> > way: Python APIs can include attributes as well as methods.
> > Exposure of an attribute need not constrain the implementation, thanks
> > to properties.  OTOH, I wouldn't bother with a property unless it's needed.
> 
> So, getting back to the original paragraph, perhaps it could say:
> 
> Decide whether a class's methods and instance variables should be public 
> or non-public.  Non-public methods and variables should start with an 
> underscore.
> 
> Do not use accessor methods, like ``obj.getFoo()`` and 
> ``obj.setFoo(v)``, instead just expose a public attribute (``obj.foo``). 
>   If necessary you can use ``property`` to implement the same 
> functionality that accessor methods would give you.  If you do use 
> properties, getting that property should never have a side effect. 
> [well... I think that certain side effects like caching and logging are 
> okay, but I'm not sure how to make that distinction]

  IMHO, if getting a property involves a potentially long computation,
it's better to have an accessor method rather than a property;
xpto.getFoobar() hints right away the programmer that to access that
value some code has to be run, so the programmer is more likely to store
the result in a temp variable for use in the same context, instead of
calling it multiple times.  Similar reasoning applites for setter vs
property =.

  Regards,

-- 
Gustavo J. A. M. Carneiro
<gjc at inescporto.pt> <gustavo at users.sourceforge.net>
The universe is always one step beyond logic.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Esta =?ISO-8859-1?Q?=E9?= uma parte de mensagem
	assinada digitalmente
Url : http://mail.python.org/pipermail/python-dev/attachments/20051212/57aad971/attachment.pgp

From aahz at pythoncraft.com  Mon Dec 12 15:52:28 2005
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 12 Dec 2005 06:52:28 -0800
Subject: [Python-Dev] A missing piece of information in weakref
	documentation
In-Reply-To: <b348a0850512120322q10440029t62ee52df6e0c3c57@mail.gmail.com>
References: <b348a0850512120322q10440029t62ee52df6e0c3c57@mail.gmail.com>
Message-ID: <20051212145228.GA25340@panix.com>

On Mon, Dec 12, 2005, Noam Raphael wrote:
> 
> I now discovered that a callback registered when creating a weak
> reference will be called only if the weak reference object is still
> alive. This is not documented in the weakref module documentation.
> 
> (It's a good behaviour - it just ought to be documented.)

Please submit a doc patch to SF (or even just a bug report if you don't
have time).  The patch may be plain text or reST; no need for Latex.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Don't listen to schmucks on USENET when making legal decisions.  Hire
yourself a competent schmuck."  --USENET schmuck (aka Robert Kern)

From skip at pobox.com  Mon Dec 12 16:28:23 2005
From: skip at pobox.com (skip@pobox.com)
Date: Mon, 12 Dec 2005 09:28:23 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439CF760.8040408@gmail.com>
References: <439CA89B.4030600@colorstudy.com>
	<17308.55295.648767.433858@montanaro.dyndns.org>
	<20051211195752.CB4A.JCARLSON@uci.edu> <439CF760.8040408@gmail.com>
Message-ID: <17309.38679.642021.215357@montanaro.dyndns.org>


    Nick> Any old code could be fixed by putting "from types import
    Nick> ClassType as __metaclass__" at the top of the affected modules.

Which would be, what, 90% of all Python code written that defines classes?

Skip

From skip at pobox.com  Mon Dec 12 16:39:13 2005
From: skip at pobox.com (skip@pobox.com)
Date: Mon, 12 Dec 2005 09:39:13 -0600
Subject: [Python-Dev] Incorporating external packages into Python's std
	distribution
In-Reply-To: <e8bf7a530512112117r6ecd15b1o7efe387633917cff@mail.gmail.com>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
	<e8bf7a530512112117r6ecd15b1o7efe387633917cff@mail.gmail.com>
Message-ID: <17309.39329.395921.139413@montanaro.dyndns.org>


    >> Just say "go", and I'll start working on this.

    Jeremy> Are you still waiting for someone to say go?  I'm not sure what
    Jeremy> responsible party should say it; if I'm not the right person,
    Jeremy> would the right person please say "go."

Can we take the absence of an explicit "stop" as an implicit "go"? <wink>

BTW, there is one project I'm theoretically familiar with that attempts to
handle the dual source situation: XEmacs.  I'm still trying to come to terms
with the practical issues involved.  I'm supposed to be updating the
python-mode code, and am only taking baby steps in that direction, so I'm
probably not the best person to describe how it works, but here goes.

For any given externally maintained package you give it a place to live in
the xemacs-packages CVS repository.  Each file gets two versions, e.g.,
python-mode.el and python-mode.el.upstream.  I believe the intent is that
the difference between the two represents XEmacs-specific changes to the
code.  When you import a new version of your code, you're supposed to factor
in the diffs between the upstream version and the XEmacs version.  You could
maintain a context/unified diff instead I suppose, then just update the
.upstream version and patch it to get the candidate version.

Skip


From skip at pobox.com  Mon Dec 12 16:59:54 2005
From: skip at pobox.com (skip@pobox.com)
Date: Mon, 12 Dec 2005 09:59:54 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439D8911.1010505@zope.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com>
	<1134395316.11373.12.camel@localhost> <439D8911.1010505@zope.com>
Message-ID: <17309.40570.699501.68209@montanaro.dyndns.org>


    Jim> That's why, in my suggested writeup, I suggested that attributes
    Jim> should be used if the accessors are trivial.

In my experience it's difficult to find the locations where another module
mucks with your object's state.  Using properties or accessor methods
coupling between modules is reduced and you can be more confident that the
only place an object's state is modified directly is in its own code.

Skip


From pje at telecommunity.com  Mon Dec 12 18:19:24 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 12 Dec 2005 12:19:24 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <17309.40570.699501.68209@montanaro.dyndns.org>
References: <439D8911.1010505@zope.com> <4399F967.3080300@colorstudy.com>
	<439C51C8.2010909@zope.com> <439CA89B.4030600@colorstudy.com>
	<1134395316.11373.12.camel@localhost> <439D8911.1010505@zope.com>
Message-ID: <5.1.1.6.0.20051212120905.0335ac00@mail.telecommunity.com>

At 09:59 AM 12/12/2005 -0600, skip at pobox.com wrote:

>     Jim> That's why, in my suggested writeup, I suggested that attributes
>     Jim> should be used if the accessors are trivial.
>
>In my experience it's difficult to find the locations where another module
>mucks with your object's state.  Using properties or accessor methods
>coupling between modules is reduced and you can be more confident that the
>only place an object's state is modified directly is in its own code.

So?

There is no reason for you to care about this in advance of actual 
requirements.  Normal instance variables should be used for normal instance 
variable things, until you have a need to do something when they 
change.  Then, and only then, is it appropriate to introduce 
properties.  Otherwise, you're just wasting your time with busywork and 
annoying the heck out of people trying to read your code.  Python is not 
Java, and Java's use of getters and setters is a reflection of its 
inadequacies as a programming language, not a badge of strength.  They're a 
bug, not a feature.

What *would* be a nice feature to add to Python would be a descriptor that 
stores the value of the property in the object dictionary, but calls a 
function whenever the attribute is changed.  So then you could do:

      @setter
      def somevar(self, value):
          # update attrs affected by changing self.somevar

This is the shortest upgrade path for the common case of an attribute's 
lifetime.  First, it's just a regular __dict__ attribute, and then you 
maybe want to do something when it changes, but you still want it readable 
and stored normally, without having to have two attribute names (one public 
and one private).


From barry at python.org  Mon Dec 12 17:26:28 2005
From: barry at python.org (Barry Warsaw)
Date: Mon, 12 Dec 2005 11:26:28 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439C51C8.2010909@zope.com>
References: <4399F967.3080300@colorstudy.com>  <439C51C8.2010909@zope.com>
Message-ID: <1134404788.950.24.camel@geddy.wooz.org>

On Sun, 2005-12-11 at 11:20 -0500, Jim Fulton wrote:

> This seems outdated.  My impression, in part from time spent
> working with the Python Labs guys, is that it is fine to have public
> data sttributes even for non-"record" types.  In fact, I would argue that
> any time you would be tempted to provide "getFoo()" and "setFoo(v)"
> for some "private attribute _foo", it would be better to make it
> public.  I certainly find "blah.foo" and "blah.foo = v" to be much
> better than "blah.getFoo()" and blah.setFoo(v)".
> 
> Certainly, properties provide a safety belt.  I would argue it this
> way: Python APIs can include attributes as well as methods.
> Exposure of an attribute need not constrain the implementation, thanks
> to properties.  OTOH, I wouldn't bother with a property unless it's needed.

Let me know what you think about this language (from my in-progress
update of PEP 8):

    Designing for inheritance

      Always decide whether a class's methods and instance variables
      (collectively: "attributes") should be public or non-public.  Public
      attributes are those that you expect unrelated clients of your class to
      use, with your commitment to avoid backward incompatible changes.
      Non-public attributes are those that are not intended to be used by
      third parties; you make no guarantees that non-pubic attributes won't
      change or even be removed.

      We don't use the term "private" here, since no attribute is really
      private in Python (without a generally unnecessary amount of work).
      However, another category of attribute are those which, while not being
      public, are intended for use by subclasses (often called "protected" in
      other languages).  Some classes are designed to be inherited from,
      either to extend or modify aspects of the class's behavior.  When
      designing such a class, take care to make explicit decisions about which
      attributes are public, which are non-public but useful for subclasses, and
      which are truly only to be used by your base class.

      With this in mind, here are the Pythonic guidelines:

      - Public attributes should have no leading underscores.

      - If your public attribute name collides with a reserved keyword, append
        a single trailing underscore to your attribute name.  This is
        preferable to an abbreviation or corrupted spelling.  E.g. "class_"
        is preferable to "cls" or "klass".

        Note 1: See the argument name recommendation above for class methods.

[BAW: I'll include this new text in a later followup]

      - For simple public data attributes, it is fine to expose just the
        attribute name, without complicated accessor/mutator methods.  Keep in
        mind that Python provides an easy path to future enhancement, should
        you find that a simple data attribute needs to grow functional
        behavior.  In that case, use properties to hide functional
        implementation behind simple data attribute access syntax.

        Note 1: Properties only work on new-style classes.

        Note 2: Try to keep the functional behavior side-effect free, although
        side-effects such as caching are generally fine.

      - If your class is intended to be subclassed, and you have attributes
        that you do not want subclasses to use, consider naming them with
        double leading underscores and no trailing underscores.  This invokes
        Python's name mangling algorithm, where the name of the class is
        mangled into the attribute name.  This helps avoid attribute name
        collisions should subclasses inadvertently contain attributes with the
        same name.

        Note 1: Note that only the simple class name is used in the mangled
        name, so if a subclass chooses both the same class name and attribute
        name, you can still get name collisions.

        Note 2: Name mangling can make certain uses, such as debugging, less
        convenient.  However the name mangling algorithm is well documented
        and easy to perform manually.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051212/93228217/attachment.pgp

From skip at pobox.com  Mon Dec 12 18:35:07 2005
From: skip at pobox.com (skip@pobox.com)
Date: Mon, 12 Dec 2005 11:35:07 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <5.1.1.6.0.20051212120905.0335ac00@mail.telecommunity.com>
References: <439D8911.1010505@zope.com> <4399F967.3080300@colorstudy.com>
	<439C51C8.2010909@zope.com> <439CA89B.4030600@colorstudy.com>
	<1134395316.11373.12.camel@localhost>
	<5.1.1.6.0.20051212120905.0335ac00@mail.telecommunity.com>
Message-ID: <17309.46283.452842.97765@montanaro.dyndns.org>

    >> In my experience it's difficult to find the locations where another
    >> module mucks with your object's state.  Using properties or accessor
    >> methods coupling between modules is reduced and you can be more
    >> confident that the only place an object's state is modified directly
    >> is in its own code.

    pje> So?

So I'm saying I encounter it in practice and makes code harder to maintain.
It's not a hypothetical problem for me.

Skip

From Scott.Daniels at Acm.Org  Mon Dec 12 18:34:26 2005
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Mon, 12 Dec 2005 09:34:26 -0800
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <e8bf7a530512120633j318bd18ft2d057781534c2f34@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com>
	<439C51C8.2010909@zope.com>	<439CA89B.4030600@colorstudy.com>
	<439CAEF6.4000304@zope.com>	<dnjvta$jem$1@sea.gmane.org>
	<e8bf7a530512120633j318bd18ft2d057781534c2f34@mail.gmail.com>
Message-ID: <dnkcal$28s$1@sea.gmane.org>

Jeremy Hylton wrote:
> On 12/12/05, Scott David Daniels <Scott.Daniels at acm.org> wrote:
>> Perhaps "The __ name convention is designed for 'mixins'; as a means of
>> enforcing "private" it is both ineffective and annoying.  For example,
>> distutils.msvccompiler uses a bunch of instance variables which would I
>> would like to access in a subclass, but are "unavailable" because the
>> author could not imagine why I would need them.
> 
> These are really separate issues, right?  The effect of __ names is to
> make a variable private to a class, because it's a right pain to
> access it from any other class.  If you design a class for inheritance
> and use __ names, you're deciding to keep the details of those names
> private.
For 'mixins' (or other multi-inheritance schemes) the renaming serves a
useful (and necessary) function -- collision avoidance.

In a hierarchy designed for inheritance, I suspect fewer problems than I
see in the cited code.  For code built with no thought of inheritance,
it will be easier to (re) use parts if non-'__' names are used.  Code
built for inheritance is not responsible for the correctness of
subclasses; I suspect many '__' names are used in a mistaken attempt
to prevent subclasses from making mistakes, rather than attempting
to ease their use.

--Scott David Daniels
Scott.Daniels at Acm.Org


From barry at python.org  Mon Dec 12 19:16:22 2005
From: barry at python.org (Barry Warsaw)
Date: Mon, 12 Dec 2005 13:16:22 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439A110E.8090605@colorstudy.com>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
Message-ID: <1134411382.5676.35.camel@geddy.wooz.org>

On Fri, 2005-12-09 at 17:19 -0600, Ian Bicking wrote:

> I personally feel "cls" should be used for classmethods, and not 
> elsewhere.  Just like I wouldn't like someone using "self" outside of 
> the first argument of instance methods.  So class_ still would be a good 
> spelling elsewhere.

Here's what I've written:

    Function and method arguments

      Always use 'self' for the first argument to instance methods.

      Always use 'cls' for the first argument to class methods.

      If a function argument's name clashes with a reserved keyword, it is
      generally better to append a single trailing underscore rather than use
      an abbreviation or spelling corruption.  Thus "print_" is better than
      "prnt".

> I looked at that too, but most of these didn't jump out at me.  I'll 
> copy in the parts that aren't already in PEP 8 that seem possible:
> 
>    From-imports should follow non-from imports.  Dotted imports should 
> follow
>    non-dotted imports.  Non-dotted imports should be grouped by increasing
>    length, while dotted imports should be grouped roughly alphabetically.
> 
> This seems too complex to me for PEP 8.

Really?  ISTR adopting this convention from Guido, but I'm not 100% sure
about that.  After having used it for several years now, I do really
like this style, but I'm willing to leave the recommendation out of PEP
8.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051212/6211e78a/attachment.pgp

From barry at python.org  Mon Dec 12 19:25:11 2005
From: barry at python.org (Barry Warsaw)
Date: Mon, 12 Dec 2005 13:25:11 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439CA89B.4030600@colorstudy.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com>
Message-ID: <1134411911.950.43.camel@geddy.wooz.org>

On Sun, 2005-12-11 at 16:30 -0600, Ian Bicking wrote:

> Potentially it could be added that the whole issue can often be avoided 
> when an object's methods perform actions instead of returning attributes 
> of the object.  It's a long topic; maybe it could even just be a link, 
> if someone knows of a good discussion along those lines.  I'm sure 
> there's some terminology here that I'm forgetting that describes the 
> design pattern.  There's also a point when the style guide becomes an 
> API design guide, and I don't know how far it should go in that direction.

I'm not exactly sure if this is what you're getting at, but one thing
that bothers me is using data attributes to trigger actions.  Maybe this
gets into the "no side-effects" rule for data attributes, but attributes
that cause an object to perform some action should always be explicit
methods.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051212/56be9cc0/attachment-0001.pgp

From fredrik at pythonware.com  Mon Dec 12 19:27:51 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 12 Dec 2005 19:27:51 +0100
Subject: [Python-Dev] should I really have to install Python before I can
	build it ?
Message-ID: <dnkff9$e3n$1@sea.gmane.org>

looks like you need to have a recent Python version installed
to be able to build the current trunk:

./Parser/asdl_c.py -h ./Include -c ./Python ./Parser/Python.asdl
./Parser/asdl_c.py:150: SyntaxWarning: local name 'self' in 'sum_with_constructors' shadows use of 'self' as global in nested scope
'emit'
  def sum_with_constructors(self, sum, name, depth):
./Parser/asdl_c.py:263: SyntaxWarning: local name 'self' in 'emit_function' shadows use of 'self' as global in nested scope 'emit'
  def emit_function(self, name, ctype, args, attrs, union=1):
./Parser/asdl_c.py:296: SyntaxWarning: local name 'self' in 'emit_body_union' shadows use of 'self' as global in nested scope 'emit'
  def emit_body_union(self, name, args, attrs):
./Parser/asdl_c.py:305: SyntaxWarning: local name 'self' in 'emit_body_struct' shadows use of 'self' as global in nested scope
'emit'
  def emit_body_struct(self, name, args, attrs):
./Parser/asdl_c.py:444: SyntaxWarning: local name 'self' in 'visitField' shadows use of 'self' as global in nested scope 'emit'
  def visitField(self, field, name, depth, product):
./Parser/asdl_c.py:444: SyntaxWarning: local name 'depth' in 'visitField' shadows use of 'depth' as global in nested scope 'emit'
  def visitField(self, field, name, depth, product):
./Parser/asdl_c.py:605: SyntaxWarning: local name 'self' in 'visitField' shadows use of 'self' as global in nested scope 'emit'
  def visitField(self, field, name, depth, product):
./Parser/asdl_c.py:605: SyntaxWarning: local name 'depth' in 'visitField' shadows use of 'depth' as global in nested scope 'emit'
  def visitField(self, field, name, depth, product):
Traceback (most recent call last):
  File "./Parser/asdl_c.py", line 9, in ?
    import asdl
  File "./Parser/asdl.py", line 53, in ?
    class ASDLScanner(spark.GenericScanner, object):
NameError: name 'object' is not defined
make: *** [Include/Python-ast.h] Error 1

(this machine has Python 2.1)

any reason why the C files are not checked into subversion ?

</F>




From barry at python.org  Mon Dec 12 19:44:49 2005
From: barry at python.org (Barry Warsaw)
Date: Mon, 12 Dec 2005 13:44:49 -0500
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <bbaeab100512111512t7f363cc0tc04733d1364ab9a0@mail.gmail.com>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de>
	<bbaeab100512111512t7f363cc0tc04733d1364ab9a0@mail.gmail.com>
Message-ID: <1134413089.951.48.camel@geddy.wooz.org>

On Sun, 2005-12-11 at 15:12 -0800, Brett Cannon wrote:

> I remember Barry saying he wanted to start a branch for work on the
> next version of the 'email' package.  And it is possible more and more
> modules developed externally will begin to be included in the stdlib. 
> Perhaps PEP 2 should be updated with basic guidelines we plan to stick
> to
> for modules that are externally developed and occasionally synched
> with the core.  Basically I think specifying who the code comes from,
> having auto-assignment for bug reports in the tracker, and saying that
> no updates to the snapshot except for bug fixes once alpha is released
> should be enough.  I would assume the snapshot in svn would just be a
> direct copy to the core and not require running any special script or
> something to generate anything.  If we do go that way, then mentioning
> that in the PEP wouldn't hurt either.

Which reminds me.  I think it may make sense to offer svn.python.org to
other contrib projects that may or are included in the stdlib.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051212/826b4d30/attachment.pgp

From jim at zope.com  Mon Dec 12 19:52:28 2005
From: jim at zope.com (Jim Fulton)
Date: Mon, 12 Dec 2005 13:52:28 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <17309.40570.699501.68209@montanaro.dyndns.org>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com>
	<1134395316.11373.12.camel@localhost> <439D8911.1010505@zope.com>
	<17309.40570.699501.68209@montanaro.dyndns.org>
Message-ID: <439DC6EC.60205@zope.com>

skip at pobox.com wrote:
>     Jim> That's why, in my suggested writeup, I suggested that attributes
>     Jim> should be used if the accessors are trivial.
> 
> In my experience it's difficult to find the locations where another module
> mucks with your object's state.  Using properties or accessor methods
> coupling between modules is reduced and you can be more confident that the
> only place an object's state is modified directly is in its own code.

I don't understand this argument.  Any mutating method or property invoked by
foreign code changes an object's state.  If you provide a property or a pair
if accessors that just sets and gets an attribute with a slightly different name,
that affords no more protection than if people were setting the attribute directly.

If you don't want external code to change an attribute, don't expose it through
a public API.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From pje at telecommunity.com  Mon Dec 12 20:01:34 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 12 Dec 2005 14:01:34 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <17309.46283.452842.97765@montanaro.dyndns.org>
References: <5.1.1.6.0.20051212120905.0335ac00@mail.telecommunity.com>
	<439D8911.1010505@zope.com> <4399F967.3080300@colorstudy.com>
	<439C51C8.2010909@zope.com> <439CA89B.4030600@colorstudy.com>
	<1134395316.11373.12.camel@localhost>
	<5.1.1.6.0.20051212120905.0335ac00@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051212140008.01f2c6b8@mail.telecommunity.com>

At 11:35 AM 12/12/2005 -0600, skip at pobox.com wrote:
>     >> In my experience it's difficult to find the locations where another
>     >> module mucks with your object's state.  Using properties or accessor
>     >> methods coupling between modules is reduced and you can be more
>     >> confident that the only place an object's state is modified directly
>     >> is in its own code.
>
>     pje> So?
>
>So I'm saying I encounter it in practice and makes code harder to maintain.
>It's not a hypothetical problem for me.

I don't understand what part is the "problem".  Why do you care what other 
code does to your object's state?  If you need to maintain your own state 
when an attribute changes, change the attribute to a property.  Where's the 
"problem"?


From guido at python.org  Mon Dec 12 20:11:42 2005
From: guido at python.org (Guido van Rossum)
Date: Mon, 12 Dec 2005 11:11:42 -0800
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<439CE06D.70600@gmail.com>
	<ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com>
	<d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com>
Message-ID: <ca471dc20512121111i2596a04dlbe2e151780829ec3@mail.gmail.com>

On 12/11/05, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 12/11/05, Guido van Rossum <guido at python.org> wrote:
> > I know about the fear of accidental reuse of class names, but I don't
> > find it a compelling argument.
>
> FWIW, I know I currently have a number of classes that are potentially
> hazardous in this way.  Each of these classes is basically a
> substitute class for a third-party API that I have to code to.  The
> API is missing a number of convenience methods, and the most
> straightforward way to introduce these methods[1] is to create a
> subclass of the appropriate class.  Since they are in a different
> module, it seems perfectly normal for me to give them the same name
> since for all external modules, they should look the same as the
> original API (but with the added methods).  So I have a number of
> classes that look something like:
>
> class Document(_cdm.Document):
>     ...
>     # add convenience methods here
>     ...

Personally, I find that naming convention a mistake. Call it
MyDocument or EnhancedDocument or DocumentPlusPlus (be creative!) but
don't reuse the original name.

I'm not saying this because it helps the __private argument; I'm
saying this because in lots of contexts we leave out the
package/module path and only use the class name, and added
functionality is a good reason to be able to distinguish between the
original class and the enhanced version.

> > Also, I like the current, well-defined mangling algorithm; it means
> > that when I'm in the debugger I can manually mangle or unmangle names
> > as required.
>
> Why couldn't the name mangling do something like:
>
>     '_%s_%s__%s' % (cls.__module__, cls.__name__, attrname)

Too long, IMO.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 12 20:15:06 2005
From: guido at python.org (Guido van Rossum)
Date: Mon, 12 Dec 2005 11:15:06 -0800
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <aac2c7cb0512120449g1954e99fk65d24f3bc51cd428@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<439CE06D.70600@gmail.com>
	<ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com>
	<d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com>
	<aac2c7cb0512120449g1954e99fk65d24f3bc51cd428@mail.gmail.com>
Message-ID: <ca471dc20512121115l2b98c343q675604bf127e3a@mail.gmail.com>

On 12/12/05, Adam Olsen <rhamph at gmail.com> wrote:
> When I need an identifier unique to a class I just use a reference to
> the class itself.  As such I'd like to suggest that
> obj.__private
> be converted to
> obj.__dict__[(type(obj), '__private')]
>
> Note that I'm accessing __dict__ directly so as to avoid getattr's
> requirement for attribute names to be strings.
>
> Obviously it doesn't handle backwards compatibility, so it's more of a
> "if I could do it again.." suggestion.

but that's not the same at all. The point of __private is that it uses
the *static* scope of the code that contains the reference, not the
(dynamic) type of the object being referenced. With your approach, if
class A defined __private, *anyone* could use A().__private (but not
B().__private where B is a subclass of A). The intention is for
__private to have the right meaning only within the source code for
class A, but it should work even if type(self) is a subclass of A. (Or
even if it's unrelated to A, but that's a separate and weaker use
case.)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 12 20:17:17 2005
From: guido at python.org (Guido van Rossum)
Date: Mon, 12 Dec 2005 11:17:17 -0800
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <439D6012.4060609@zope.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<1f7befae0512111818x13ab3185w48bd365d0b5fe5f4@mail.gmail.com>
	<439D6012.4060609@zope.com>
Message-ID: <ca471dc20512121117p3332c3a1k846deab37e4050c0@mail.gmail.com>

On 12/12/05, Jim Fulton <jim at zope.com> wrote:
> In practice, I don't agree that it works fine.  Inevitably, someone
> finds a need to access a "private" variable in a subclass.  Or
> even in the original class, you find some need to use something like
> __getattr__ where the implicit name mangling doesn't come into play
> and you have to emulate the name mangling.  Or perhaps someone wants
> to examine the value of one of these variables in the debugger.
> In my experience, almost every time someone uses the __private
> trick, they or someone else comes to regret it.
>
> OTOH, explicit name mangling provides the benefits of implicit
> name mangling without it's drawbacks.

I half agree. I've seen many classes overuse __private. But that's a
separate issue from not having the feature at all; you might as well
argue against private in Java or C++.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 12 20:19:34 2005
From: guido at python.org (Guido van Rossum)
Date: Mon, 12 Dec 2005 11:19:34 -0800
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <dnjvta$jem$1@sea.gmane.org>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<dnjvta$jem$1@sea.gmane.org>
Message-ID: <ca471dc20512121119g5089142cj40bbefabe66d1068@mail.gmail.com>

On 12/12/05, Scott David Daniels <Scott.Daniels at acm.org> wrote:
> Perhaps "The __ name convention is designed for 'mixins'; as a means of
> enforcing "private" it is both ineffective and annoying.  For example,
> distutils.msvccompiler uses a bunch of instance variables which would I
> would like to access in a subclass, but are "unavailable" because the
> author could not imagine why I would need them.

But __private's use case is *not* restricted to mixins; this seems to
be a common misconception. It is a tool (not the only one!) for name
conflict avoidance in all inheritance situations, including single
inheritance.

BTW let me note that inheritance is overused. People should get used
to containment patterns (e.g. facade, delegate etc.) in favor of
inheritance patterns.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jeremy at alum.mit.edu  Mon Dec 12 20:22:30 2005
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 12 Dec 2005 14:22:30 -0500
Subject: [Python-Dev] should I really have to install Python before I
	can build it ?
In-Reply-To: <dnkff9$e3n$1@sea.gmane.org>
References: <dnkff9$e3n$1@sea.gmane.org>
Message-ID: <e8bf7a530512121122ie8c8d4av99ed47565e6c91d6@mail.gmail.com>

The C files are checked into subversion.  Perhaps there is some
problem with the timestamps that causes the Makefile to try to rebuild
them anyway?  I have a modern Python and I've been doing a fair amount
of development on these files; as a result, I haven't noticed a
problem.

Jeremy

On 12/12/05, Fredrik Lundh <fredrik at pythonware.com> wrote:
> looks like you need to have a recent Python version installed
> to be able to build the current trunk:
>
> ./Parser/asdl_c.py -h ./Include -c ./Python ./Parser/Python.asdl
> ./Parser/asdl_c.py:150: SyntaxWarning: local name 'self' in 'sum_with_constructors' shadows use of 'self' as global in nested scope
> 'emit'
>   def sum_with_constructors(self, sum, name, depth):
> ./Parser/asdl_c.py:263: SyntaxWarning: local name 'self' in 'emit_function' shadows use of 'self' as global in nested scope 'emit'
>   def emit_function(self, name, ctype, args, attrs, union=1):
> ./Parser/asdl_c.py:296: SyntaxWarning: local name 'self' in 'emit_body_union' shadows use of 'self' as global in nested scope 'emit'
>   def emit_body_union(self, name, args, attrs):
> ./Parser/asdl_c.py:305: SyntaxWarning: local name 'self' in 'emit_body_struct' shadows use of 'self' as global in nested scope
> 'emit'
>   def emit_body_struct(self, name, args, attrs):
> ./Parser/asdl_c.py:444: SyntaxWarning: local name 'self' in 'visitField' shadows use of 'self' as global in nested scope 'emit'
>   def visitField(self, field, name, depth, product):
> ./Parser/asdl_c.py:444: SyntaxWarning: local name 'depth' in 'visitField' shadows use of 'depth' as global in nested scope 'emit'
>   def visitField(self, field, name, depth, product):
> ./Parser/asdl_c.py:605: SyntaxWarning: local name 'self' in 'visitField' shadows use of 'self' as global in nested scope 'emit'
>   def visitField(self, field, name, depth, product):
> ./Parser/asdl_c.py:605: SyntaxWarning: local name 'depth' in 'visitField' shadows use of 'depth' as global in nested scope 'emit'
>   def visitField(self, field, name, depth, product):
> Traceback (most recent call last):
>   File "./Parser/asdl_c.py", line 9, in ?
>     import asdl
>   File "./Parser/asdl.py", line 53, in ?
>     class ASDLScanner(spark.GenericScanner, object):
> NameError: name 'object' is not defined
> make: *** [Include/Python-ast.h] Error 1
>
> (this machine has Python 2.1)
>
> any reason why the C files are not checked into subversion ?
>
> </F>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>

From guido at python.org  Mon Dec 12 20:32:54 2005
From: guido at python.org (Guido van Rossum)
Date: Mon, 12 Dec 2005 11:32:54 -0800
Subject: [Python-Dev] Exception type on handling closed files
In-Reply-To: <dnj931$eto$1@sea.gmane.org>
References: <787073ca0512112031v232a5d12s@mail.gmail.com>
	<dnj931$eto$1@sea.gmane.org>
Message-ID: <ca471dc20512121132u3e1fffe3q136f7b9797ed6df8@mail.gmail.com>

On 12/11/05, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Jo?o Paulo Silva wrote:
>
> > >>> a = file("dir/foo")
> > >>> a.close()
> > >>> a.read()
> >
> > Traceback (most recent call last):
> >   File "<pyshell#28>", line 1, in -toplevel-
> >     a.read()
> > ValueError: I/O operation on closed file
> >
> > Shoudn't this raise IOError? Seems more semantically correct to me.
>
> IOError is, as the documentation says, used "when an I/O operation fails
> for an I/O related reason", while ValueError is used "when an argument has
> the right type but an inappropriate value."

What /F says.

IOError is something you could reasonably catch, log, and ignore
(since I/O devices are known to be fallible). The ValueError (at least
in this case) means there's a logic bug in your program -- you're
trying to use a file that you've already closed. Very important
distinction!

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fredrik at pythonware.com  Mon Dec 12 20:43:48 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 12 Dec 2005 20:43:48 +0100
Subject: [Python-Dev] should I really have to install Python before Ican
	build it ?
References: <dnkff9$e3n$1@sea.gmane.org>
	<e8bf7a530512121122ie8c8d4av99ed47565e6c91d6@mail.gmail.com>
Message-ID: <dnkjtm$umr$1@sea.gmane.org>

Jeremy Hylton wrote:

> The C files are checked into subversion.  Perhaps there is some
> problem with the timestamps that causes the Makefile to try to rebuild
> them anyway?  I have a modern Python and I've been doing a fair amount
> of development on these files; as a result, I haven't noticed a
> problem.

ah, of course.  subversion sets the timestamp to the checkout time for each
file, so things may or may not work after a fresh checkout.

however, adsl_c does use the installed python, rather than the local version:

    #! /usr/bin/env python
    """Generate C code from an ASDL description."""

maybe the right thing here would be to change this to

    #!./python
    """Generate C code from an ASDL description."""

and only run the script if ./python has been built ?

</F>




From ianb at colorstudy.com  Mon Dec 12 20:50:26 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 12 Dec 2005 13:50:26 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <000d01c5fec2$6b19f160$cf26a044@oemcomputer>
References: <000d01c5fec2$6b19f160$cf26a044@oemcomputer>
Message-ID: <439DD482.9000106@colorstudy.com>

Raymond Hettinger wrote:
>>Do not use accessor methods, like ``obj.getFoo()`` and
>>``obj.setFoo(v)``, instead just expose a public attribute
> 
> (``obj.foo``).
> 
> This advice is, of course, not appropriate for all users (properties are
> not typically in a Python beginner's skill set) or all use cases.  It is
> closer to one person's view of the One-Right-Way(tm).  Opinions on
> programming best practices vary widely, evolve over time, and may be
> context dependent.

Beginning programmers do all sorts of things that aren't considered good 
style by more experienced programmers.  That's fine -- but then PEP 8 
should direct them to a better style.  Specifically PEP 8 currently 
suggests that public attributes should be avoided, and this no longer 
needs to be the case.  But at the same time, people are using Java 
conventions of setters and getters (and these conventions exist in older 
code as well), so I think it is helpful to suggest that accessor methods 
should be avoided.  I don't think the suggestion has to be strongly worded.

>>>While, on some level, private variables seem attractive, I think that
>>>experience (for everyone I know) has shown them to be an attractive
>>>nuisance.  I recommend discouraging them.
>>
>>I really really hate double underscores
> 
> 
> FWIW, I think we have no business dictating to others how they should
> name their variables.  This is doubly true for a convention that has a
> long history and built-in language support.

Double underscores aren't just naming, they involve the semantics of 
name mangling.  That's what makes them different than other names, and 
jarring to many programmers (like myself).

Personally I'm happy if we call double underscore attributes "hidden" 
instead of "private", or otherwise help keep people from being 
misdirected into using double underscore as "real" private variables. 
PEP 8 currently gives the impression that they should be used for 
private attributes.

> My preference is to leave PEP 8 for the minimum practices necessary for
> one programmer to be able to read and maintain another programmer's
> code.

There's a couple things I want to use PEP 8 for:

* Deciding on things I don't care that much about, except in terms of 
consistency.  I am happy that PEP 8 was updated to say that underscore 
separated words are preferred, for instance, though I would have been 
just as happy with mixed case.  I just want everyone to at least move 
towards being on the same page.

* When debates on these styles come up, I want to be able to point to 
something somewhat authoritative.  This avoids a lot of pointless 
discussion.

Given these motivations, I guess I don't care that much about how __ is 
presented in PEP 8, except that the current inconsistent messages about 
it is made consistent, and that it isn't misrepresented as being the way 
of indicating "private".  I don't think PEP 8 needs to be a text on good 
API design.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From ianb at colorstudy.com  Mon Dec 12 21:25:54 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 12 Dec 2005 14:25:54 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134411382.5676.35.camel@geddy.wooz.org>
References: <4399F967.3080300@colorstudy.com>	<1134168683.19370.29.camel@geddy.wooz.org>	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
Message-ID: <439DDCD2.10803@colorstudy.com>

Barry Warsaw wrote:
> On Fri, 2005-12-09 at 17:19 -0600, Ian Bicking wrote:
> 
> 
>>I personally feel "cls" should be used for classmethods, and not 
>>elsewhere.  Just like I wouldn't like someone using "self" outside of 
>>the first argument of instance methods.  So class_ still would be a good 
>>spelling elsewhere.
> 
> 
> Here's what I've written:
> 
>     Function and method arguments
> 
>       Always use 'self' for the first argument to instance methods.
> 
>       Always use 'cls' for the first argument to class methods.
> 
>       If a function argument's name clashes with a reserved keyword, it is
>       generally better to append a single trailing underscore rather than use
>       an abbreviation or spelling corruption.  Thus "print_" is better than
>       "prnt".

That looks good to me.  Well, I actually try not to use cls as the first 
argument to metaclass's __new__ method, because there's so many classes 
being tossed about at that point that I try to be more explicit.  But I 
don't consider that a common enough issue to be worth mentioning in PEP 8.

>>I looked at that too, but most of these didn't jump out at me.  I'll 
>>copy in the parts that aren't already in PEP 8 that seem possible:
>>
>>   From-imports should follow non-from imports.  Dotted imports should 
>>follow
>>   non-dotted imports.  Non-dotted imports should be grouped by increasing
>>   length, while dotted imports should be grouped roughly alphabetically.
>>
>>This seems too complex to me for PEP 8.
> 
> 
> Really?  ISTR adopting this convention from Guido, but I'm not 100% sure
> about that.  After having used it for several years now, I do really
> like this style, but I'm willing to leave the recommendation out of PEP
> 8.

It seems so exacting to me; stdlib, external modules, internal modules 
seems like enough ordering to me.  If you want to order things more 
exactly, sure, but I don't really see the point personally.  Since I 
can't assume as a reader that imports are ordered in any way I have to 
search to be sure of what's there.  The grouping help me browse, but I'd 
hope that the import list is short enough that I don't need to use 
alphabetization to scan for a module.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Mon Dec 12 21:40:47 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 12 Dec 2005 15:40:47 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439DDCD2.10803@colorstudy.com>
References: <1134411382.5676.35.camel@geddy.wooz.org>
	<4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
Message-ID: <5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>

At 02:25 PM 12/12/2005 -0600, Ian Bicking wrote:
>That looks good to me.  Well, I actually try not to use cls as the first
>argument to metaclass's __new__ method, because there's so many classes
>being tossed about at that point that I try to be more explicit.  But I
>don't consider that a common enough issue to be worth mentioning in PEP 8.

I usually use 'meta' as the first argument of a metaclass __new__ or a 
metaclass classmethod, to avoid this particular bit of confusion.


From fredrik at pythonware.com  Mon Dec 12 22:03:53 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 12 Dec 2005 22:03:53 +0100
Subject: [Python-Dev] stupid package tricks
Message-ID: <dnkojq$hb9$1@sea.gmane.org>

the xml/__init__.py file contains a cute little hack that overrides
the *entire* xml subtree with stuff from PyXML, if available.

the code basically does

    import _xmlplus
    sys.modules[__name__] = _xmlplus

(exception handling and version checks not shown).

however, this means that as things are right now, xml.etree will
simply disappear if the user has PyXML on the machine.

what's the best way to fix this?  the obvious fix is of course to do
something like

    import _xmlplus
    import xml.etree
    _xmlplus.etree = xml.etree
    sys.modules[__name__] = _xmlplus

but I have to admit that I'm no expert on package internals, so I
might be missing something here.  will the above solution work in
all cases?  is there some better way to do it?

</F>




From bcannon at gmail.com  Mon Dec 12 22:17:32 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Mon, 12 Dec 2005 13:17:32 -0800
Subject: [Python-Dev] should I really have to install Python before Ican
	build it ?
In-Reply-To: <dnkjtm$umr$1@sea.gmane.org>
References: <dnkff9$e3n$1@sea.gmane.org>
	<e8bf7a530512121122ie8c8d4av99ed47565e6c91d6@mail.gmail.com>
	<dnkjtm$umr$1@sea.gmane.org>
Message-ID: <bbaeab100512121317t2fb4de19te0db567af1651615@mail.gmail.com>

On 12/12/05, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Jeremy Hylton wrote:
>
> > The C files are checked into subversion.  Perhaps there is some
> > problem with the timestamps that causes the Makefile to try to rebuild
> > them anyway?  I have a modern Python and I've been doing a fair amount
> > of development on these files; as a result, I haven't noticed a
> > problem.
>
> ah, of course.  subversion sets the timestamp to the checkout time for each
> file, so things may or may not work after a fresh checkout.
>
> however, adsl_c does use the installed python, rather than the local version:
>
>     #! /usr/bin/env python
>     """Generate C code from an ASDL description."""
>
> maybe the right thing here would be to change this to
>
>     #!./python
>     """Generate C code from an ASDL description."""
>
> and only run the script if ./python has been built ?
>

What if you build with a different suffix for the executable?  Or do
different versions of make build different names (e.g., on my OS X
machine the executable is python.exe in my checkout, not python)?

The idea seems fine to me, though, since the generated files are
already checked out.

-Brett

From pje at telecommunity.com  Mon Dec 12 22:29:36 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 12 Dec 2005 16:29:36 -0500
Subject: [Python-Dev] stupid package tricks
In-Reply-To: <dnkojq$hb9$1@sea.gmane.org>
Message-ID: <5.1.1.6.0.20051212162721.020bbbb0@mail.telecommunity.com>

At 10:03 PM 12/12/2005 +0100, Fredrik Lundh wrote:
>the xml/__init__.py file contains a cute little hack that overrides
>the *entire* xml subtree with stuff from PyXML, if available.
>
>the code basically does
>
>     import _xmlplus
>     sys.modules[__name__] = _xmlplus
>
>(exception handling and version checks not shown).
>
>however, this means that as things are right now, xml.etree will
>simply disappear if the user has PyXML on the machine.
>
>what's the best way to fix this?  the obvious fix is of course to do
>something like
>
>     import _xmlplus
>     import xml.etree
>     _xmlplus.etree = xml.etree
>     sys.modules[__name__] = _xmlplus
>
>but I have to admit that I'm no expert on package internals, so I
>might be missing something here.  will the above solution work in
>all cases?  is there some better way to do it?

I'd suggest:

     import _xmlplus
     _xmlplus.__path__.extend(__path__)
     sys.modules[__name__] = _xmlplus

This ensures that any modules or packages inside 'xml' that aren't 
explicitly overridden by _xmlplus will still be available.


From fredrik at pythonware.com  Mon Dec 12 22:23:27 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 12 Dec 2005 22:23:27 +0100
Subject: [Python-Dev] should I really have to install Python before
	Icanbuild it ?
References: <dnkff9$e3n$1@sea.gmane.org><e8bf7a530512121122ie8c8d4av99ed47565e6c91d6@mail.gmail.com><dnkjtm$umr$1@sea.gmane.org>
	<bbaeab100512121317t2fb4de19te0db567af1651615@mail.gmail.com>
Message-ID: <dnkpog$lk9$1@sea.gmane.org>

Brett Cannon wrote:

> > maybe the right thing here would be to change this to
> >
> >     #!./python
> >     """Generate C code from an ASDL description."""
> >
> > and only run the script if ./python has been built ?
>
> What if you build with a different suffix for the executable?  Or do
> different versions of make build different names (e.g., on my OS X
> machine the executable is python.exe in my checkout, not python)?

you're right.  I guess the right thing is to do this in the Makefile, and
use $(PYTHON) to find the appropriate interpreter.

changing the relevant rule to

$(AST_H) $(AST_C): $(AST_ASDL) $(ASDLGEN_FILES)
        -$(PYTHON) $(ASDLGEN) $(AST_ASDL)

might be sufficient.

> The idea seems fine to me, though, since the generated files are
> already checked out.

</F>




From steven.bethard at gmail.com  Mon Dec 12 22:33:39 2005
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 12 Dec 2005 14:33:39 -0700
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <ca471dc20512121111i2596a04dlbe2e151780829ec3@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<439CE06D.70600@gmail.com>
	<ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com>
	<d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com>
	<ca471dc20512121111i2596a04dlbe2e151780829ec3@mail.gmail.com>
Message-ID: <d11dcfba0512121333o4e17b735mcb5f91df4387d380@mail.gmail.com>

On 12/12/05, Guido van Rossum <guido at python.org> wrote:
> On 12/11/05, Steven Bethard <steven.bethard at gmail.com> wrote:
> > class Document(_cdm.Document):
> >     ...
> >     # add convenience methods here
> >     ...
>
> Personally, I find that naming convention a mistake. Call it
> MyDocument or EnhancedDocument or DocumentPlusPlus (be creative!) but
> don't reuse the original name.
>
> I'm not saying this because it helps the __private argument; I'm
> saying this because in lots of contexts we leave out the
> package/module path and only use the class name, and added
> functionality is a good reason to be able to distinguish between the
> original class and the enhanced version.

Ahh.  I never run into this because I never import objects directly
from modules.  So, instead of:

    from elementtree.ElementTree import ElementTree
    ...
    ElementTree(...)

I almost always write something like:

    import elementtree.ElementTree as et
    ...
    et.ElementTree(...)

Thus, all objects that were imported from external modules are always
immediately identifiable as such by their prefixed module name.  I do
see though that if you like to import the objects directly from the
module this could be confusing.

STeVe
--
You can wordify anything if you just verb it.
        --- Bucky Katt, Get Fuzzy

From noamraph at gmail.com  Mon Dec 12 22:37:04 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Mon, 12 Dec 2005 23:37:04 +0200
Subject: [Python-Dev] A missing piece of information in weakref
	documentation
In-Reply-To: <20051212145228.GA25340@panix.com>
References: <b348a0850512120322q10440029t62ee52df6e0c3c57@mail.gmail.com>
	<20051212145228.GA25340@panix.com>
Message-ID: <b348a0850512121337r6e3dfbddge10b5d7f0c077d0e@mail.gmail.com>

On 12/12/05, Aahz <aahz at pythoncraft.com> wrote:
> Please submit a doc patch to SF (or even just a bug report if you don't
> have time).  The patch may be plain text or reST; no need for Latex.

Done - patch number 1379023.

Noam

From dberlin at dberlin.org  Mon Dec 12 22:13:00 2005
From: dberlin at dberlin.org (Daniel Berlin)
Date: Mon, 12 Dec 2005 16:13:00 -0500
Subject: [Python-Dev] should I really have to install Python before	Ican
	build it ?
In-Reply-To: <dnkjtm$umr$1@sea.gmane.org>
References: <dnkff9$e3n$1@sea.gmane.org>
	<e8bf7a530512121122ie8c8d4av99ed47565e6c91d6@mail.gmail.com>
	<dnkjtm$umr$1@sea.gmane.org>
Message-ID: <1134421980.9942.0.camel@linux.site>

On Mon, 2005-12-12 at 20:43 +0100, Fredrik Lundh wrote:
> Jeremy Hylton wrote:
> 
> > The C files are checked into subversion.  Perhaps there is some
> > problem with the timestamps that causes the Makefile to try to rebuild
> > them anyway?  I have a modern Python and I've been doing a fair amount
> > of development on these files; as a result, I haven't noticed a
> > problem.
> 
> ah, of course.  subversion sets the timestamp to the checkout time for each
> file, so things may or may not work after a fresh checkout.
You can change this by setting use-commit-times=true in
~/.subversion/config



From fredrik at pythonware.com  Mon Dec 12 23:12:38 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 12 Dec 2005 23:12:38 +0100
Subject: [Python-Dev] Incorporating external packages into Python's
	stddistribution
References: <17304.33755.693941.811233@montanaro.dyndns.org><43989A99.9000402@v.loewis.de>
	<dnhl9k$bt6$1@sea.gmane.org><439C7341.4050300@v.loewis.de>
	<dni8qe$5eq$1@sea.gmane.org><e8bf7a530512112117r6ecd15b1o7efe387633917cff@mail.gmail.com>
	<17309.39329.395921.139413@montanaro.dyndns.org>
Message-ID: <dnkskp$vsu$1@sea.gmane.org>

skip at pobox.com wrote:

> BTW, there is one project I'm theoretically familiar with that attempts to
> handle the dual source situation: XEmacs.  I'm still trying to come to terms
> with the practical issues involved.  I'm supposed to be updating the
> python-mode code, and am only taking baby steps in that direction, so I'm
> probably not the best person to describe how it works, but here goes.
>
> For any given externally maintained package you give it a place to live in
> the xemacs-packages CVS repository.  Each file gets two versions, e.g.,
> python-mode.el and python-mode.el.upstream.  I believe the intent is that
> the difference between the two represents XEmacs-specific changes to the
> code.  When you import a new version of your code, you're supposed to factor
> in the diffs between the upstream version and the XEmacs version.  You could
> maintain a context/unified diff instead I suppose, then just update the
> .upstream version and patch it to get the candidate version.

in the model I proposed (and just implemented), the "external" repository
corresponds to your "upstream" copy.  you can use

    $ svn log -v --stop-on-copy <file>

to get an overview of all changes since the last upstream copy

$ svn log -v --stop-on-copy Lib/xml/etree/ElementTree.py
------------------------------------------------------------------------
r41651 | fredrik.lundh | 2005-12-12 16:10:44 +0100 (Mon, 12 Dec 2005) | 3 lines
Changed paths:
   A /python/trunk/Lib/xml/etree
   A /python/trunk/Lib/xml/etree/ElementInclude.py
   (from /external/elementtree-1.2.6-20050316/elementtree/ElementInclude.py:41650)
   A /python/trunk/Lib/xml/etree/ElementPath.py
   (from /external/elementtree-1.2.6-20050316/elementtree/ElementPath.py:41650)
   A /python/trunk/Lib/xml/etree/ElementTree.py
   (from /external/elementtree-1.2.6-20050316/elementtree/ElementTree.py:41650)
   A /python/trunk/Lib/xml/etree/__init__.py
   (from /external/elementtree-1.2.6-20050316/elementtree/__init__.py:41650)

added ElementTree core components to xml.etree

and use

    $ svn diff -r <version> <file>

to get a full diff:

$ svn diff -r 41651 Lib/xml/etree/ElementTree.py
$

(nothing has changed yet)

to update to a new upstream release, save the diff somewhere, import
the new release under external, copy relevant files to trunk, commit, merge
in the diff by hand, or using "svn merge".  when you're done, commit again.
that's it.

</F>




From arigo at tunes.org  Mon Dec 12 22:54:52 2005
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 12 Dec 2005 22:54:52 +0100
Subject: [Python-Dev] should I really have to install Python before
	Icanbuild it ?
In-Reply-To: <dnkpog$lk9$1@sea.gmane.org>
References: <bbaeab100512121317t2fb4de19te0db567af1651615@mail.gmail.com>
	<dnkpog$lk9$1@sea.gmane.org>
Message-ID: <20051212215452.GA19322@code1.codespeak.net>

Hi Fredrik,

On Mon, Dec 12, 2005 at 10:23:27PM +0100, Fredrik Lundh wrote:
> $(AST_H) $(AST_C): $(AST_ASDL) $(ASDLGEN_FILES)
>         -$(PYTHON) $(ASDLGEN) $(AST_ASDL)

I suppose that the trick is in the "-" sign here.  If this command fails
for any reason you get warnings and errors but the build still continues
with the current version of the .h/.c files, and we are left with
telling users "no no, ignore this build error, everything is fine".

The same just-ignore-it behavior can bite if the script genuinely fails
after you just made a typo in one of the input files, for example.
Doesn't look particularly clean to me, if you want my opinion.


A bientot,

Armin

From bcannon at gmail.com  Mon Dec 12 23:31:32 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Mon, 12 Dec 2005 14:31:32 -0800
Subject: [Python-Dev] should I really have to install Python before Ican
	build it ?
In-Reply-To: <1134421980.9942.0.camel@linux.site>
References: <dnkff9$e3n$1@sea.gmane.org>
	<e8bf7a530512121122ie8c8d4av99ed47565e6c91d6@mail.gmail.com>
	<dnkjtm$umr$1@sea.gmane.org> <1134421980.9942.0.camel@linux.site>
Message-ID: <bbaeab100512121431i99cc49ep85fc0501e88f8f79@mail.gmail.com>

On 12/12/05, Daniel Berlin <dberlin at dberlin.org> wrote:
> On Mon, 2005-12-12 at 20:43 +0100, Fredrik Lundh wrote:
> > Jeremy Hylton wrote:
> >
> > > The C files are checked into subversion.  Perhaps there is some
> > > problem with the timestamps that causes the Makefile to try to rebuild
> > > them anyway?  I have a modern Python and I've been doing a fair amount
> > > of development on these files; as a result, I haven't noticed a
> > > problem.
> >
> > ah, of course.  subversion sets the timestamp to the checkout time for each
> > file, so things may or may not work after a fresh checkout.
> You can change this by setting use-commit-times=true in
> ~/.subversion/config

What do other people think of this option?  Sounds reasonable to me. 
if people like it I will add this to the suggested config options
specified in the dev FAQ.

-Brett

From martin at v.loewis.de  Mon Dec 12 23:38:13 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 12 Dec 2005 23:38:13 +0100
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <1134413089.951.48.camel@geddy.wooz.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>	
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>	
	<439C7341.4050300@v.loewis.de>	
	<bbaeab100512111512t7f363cc0tc04733d1364ab9a0@mail.gmail.com>
	<1134413089.951.48.camel@geddy.wooz.org>
Message-ID: <439DFBD5.3030000@v.loewis.de>

Barry Warsaw wrote:
> Which reminds me.  I think it may make sense to offer svn.python.org to
> other contrib projects that may or are included in the stdlib.

Sure. Committers should understand what part of the tree they
are supposed to write to.

Regards,
Martin

From aleaxit at gmail.com  Mon Dec 12 23:59:53 2005
From: aleaxit at gmail.com (Alex Martelli)
Date: Mon, 12 Dec 2005 14:59:53 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
Message-ID: <e8a0972d0512121459oecc0f31nf2eb91574387c010@mail.gmail.com>

On 12/12/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 02:25 PM 12/12/2005 -0600, Ian Bicking wrote:
> >That looks good to me.  Well, I actually try not to use cls as the first
> >argument to metaclass's __new__ method, because there's so many classes
> >being tossed about at that point that I try to be more explicit.  But I
> >don't consider that a common enough issue to be worth mentioning in PEP 8.
>
> I usually use 'meta' as the first argument of a metaclass __new__ or a
> metaclass classmethod, to avoid this particular bit of confusion.

...while I use 'mcl' for the same purpose (seems closer to me in
spirit to 'cls' than 'meta' would be); Guido said he liked that, at
the time (a couple of years ago) when he was following a talk of mine
on metaclasses where I introduced this convention.


Alex

From pje at telecommunity.com  Tue Dec 13 00:15:01 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 12 Dec 2005 18:15:01 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <e8a0972d0512121459oecc0f31nf2eb91574387c010@mail.gmail.com
 >
References: <5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>

At 02:59 PM 12/12/2005 -0800, Alex Martelli wrote:
>On 12/12/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> > At 02:25 PM 12/12/2005 -0600, Ian Bicking wrote:
> > >That looks good to me.  Well, I actually try not to use cls as the first
> > >argument to metaclass's __new__ method, because there's so many classes
> > >being tossed about at that point that I try to be more explicit.  But I
> > >don't consider that a common enough issue to be worth mentioning in PEP 8.
> >
> > I usually use 'meta' as the first argument of a metaclass __new__ or a
> > metaclass classmethod, to avoid this particular bit of confusion.
>
>...while I use 'mcl' for the same purpose (seems closer to me in
>spirit to 'cls' than 'meta' would be); Guido said he liked that, at
>the time (a couple of years ago) when he was following a talk of mine
>on metaclasses where I introduced this convention.

I'd rather see 'metaclass' fully spelled out than resort to 'mcl'; 
metaclass code is tricky enough to write without figuring out 
abbreviations.  :)

Indeed, the only reason I use 'cls' is because it was Pronounced the 
standard; before the pronouncement I was using 'klass' as the argument name 
for class methods.


From aleaxit at gmail.com  Tue Dec 13 00:57:48 2005
From: aleaxit at gmail.com (Alex Martelli)
Date: Mon, 12 Dec 2005 15:57:48 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
Message-ID: <e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>

On 12/12/05, Phillip J. Eby <pje at telecommunity.com> wrote:
   ...
> I'd rather see 'metaclass' fully spelled out than resort to 'mcl';
> metaclass code is tricky enough to write without figuring out
> abbreviations.  :)
>
> Indeed, the only reason I use 'cls' is because it was Pronounced the
> standard; before the pronouncement I was using 'klass' as the argument name
> for class methods.

The name choices klass and meta are internally consistent, and so are
cls and mcl.  I just wouldn't like a mixed, and thus
harder-to-remember, pair of choices such as cls and meta.  Perhaps
Guido can Pronounce one way or another and set the subdiscussion to
rest...


Alex

From ianb at colorstudy.com  Tue Dec 13 01:09:27 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 12 Dec 2005 18:09:27 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com>	
	<1134168683.19370.29.camel@geddy.wooz.org>	
	<439A110E.8090605@colorstudy.com>	
	<1134411382.5676.35.camel@geddy.wooz.org>	
	<439DDCD2.10803@colorstudy.com>	
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>	
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
Message-ID: <439E1137.4010403@colorstudy.com>

Alex Martelli wrote:
> On 12/12/05, Phillip J. Eby <pje at telecommunity.com> wrote:
>    ...
> 
>>I'd rather see 'metaclass' fully spelled out than resort to 'mcl';
>>metaclass code is tricky enough to write without figuring out
>>abbreviations.  :)
>>
>>Indeed, the only reason I use 'cls' is because it was Pronounced the
>>standard; before the pronouncement I was using 'klass' as the argument name
>>for class methods.
> 
> 
> The name choices klass and meta are internally consistent, and so are
> cls and mcl.  I just wouldn't like a mixed, and thus
> harder-to-remember, pair of choices such as cls and meta.  Perhaps
> Guido can Pronounce one way or another and set the subdiscussion to
> rest...

I personally happily use "meta", but it doesn't seem that important, 
except insofar as it is reasonable (and perhaps preferred) not to use 
"cls" in that case.  If someone wants to use even more verbose names in 
their metaclass that'd be fine by me -- it's not the kind of code I 
breeze by and expect to instantly understand like I do simple methods. 
I don't think it's that important to include in PEP 8, at least as long 
as no one reads the prescription of "cls" to mean they shouldn't choose 
a better argument name when there's a good reason.  PEP 8 generally 
applies when there isn't a good reason.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From barry at python.org  Tue Dec 13 01:14:20 2005
From: barry at python.org (Barry Warsaw)
Date: Mon, 12 Dec 2005 19:14:20 -0500
Subject: [Python-Dev] should I really have to install Python before
	I	can build it ?
In-Reply-To: <e8bf7a530512121122ie8c8d4av99ed47565e6c91d6@mail.gmail.com>
References: <dnkff9$e3n$1@sea.gmane.org>
	<e8bf7a530512121122ie8c8d4av99ed47565e6c91d6@mail.gmail.com>
Message-ID: <1134432860.11505.4.camel@geddy.wooz.org>

On Mon, 2005-12-12 at 14:22 -0500, Jeremy Hylton wrote:
> The C files are checked into subversion.  Perhaps there is some
> problem with the timestamps that causes the Makefile to try to rebuild
> them anyway?  I have a modern Python and I've been doing a fair amount
> of development on these files; as a result, I haven't noticed a
> problem.

I tried this early today: svn up; make distclean; configure; make

Unfortunately, that requires Python to already exist, so there's
definitely a boostrapping issue in the build process.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051212/b0ac8d40/attachment.pgp

From mike at skew.org  Tue Dec 13 01:01:37 2005
From: mike at skew.org (Mike Brown)
Date: Mon, 12 Dec 2005 17:01:37 -0700 (MST)
Subject: [Python-Dev] ElementTree in stdlib
Message-ID: <200512130001.jBD01bXZ007657@chilled.skew.org>

Catching up on some python-dev email, I was surprised to see that things seem 
to be barrelling ahead with the adding of ElementTree to Python core without 
any discussion on XML-SIG. Sidestepping XML-SIG and the proving grounds of 
PyXML in order to satsify the demand for a Pythonic databinding+API for XML in 
stdlib seems to be a bit of a raised middle finger to those folks who have 
worked hard on competing or differently-scoped APIs, each of which deserves a 
bit more peer review than just a single nomination on python-dev, which seems 
to be all it took to obtain a blessing for ElementTree. I have nothing against 
ElementTree, and would like to see more XML processing options in core, but it 
seems to me like the XML-SIG is being deliberately left out of this process.

Just last month, Guido submitted to XML-SIG a Pythonic XML API that he had 
been tinkering with.[1] I don't think anyone was really bold enough to tell 
him what they really thought of it (other than that it is a lot like XIST), 
but it was admirable that he put it up for peer review rather than just 
dropping it into stdlib. Perhaps more importantly, it prompted some discussion 
that more or less acknowledged that these kinds of APIs do seem to be the 
future of XML in Python, and that we should be thinking about bringing some of 
them into PyXML and, ultimately, stdlib. But the problem of how to choose from 
the many options also became immediately apparent.[2] The discussion stalled, 
but I think it should start up again, in the proper forum, rather than letting 
the first-mentioned API supplant the dozen+ alternatives that could also be 
considered as candidates.[3]

Sorry to be a sourpuss.

Mike
-- 

[1] http://mail.python.org/pipermail/xml-sig/2005-November/011248.html
     (Guido's very civil proposal and request for peer review)
[2] http://mail.python.org/pipermail/xml-sig/2005-November/011252.html (this
     also summarizes the categories of software/approaches that people are
     taking to the general problem of working with XML Pythonically)
[3] http://www.xml.com/pub/a/2004/10/13/py-xml.html (and there are at least
     3 more databinding APIs that have come out since then)

From greg.ewing at canterbury.ac.nz  Tue Dec 13 01:23:07 2005
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 13 Dec 2005 13:23:07 +1300
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
 updates/clarifications)
In-Reply-To: <ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
Message-ID: <439E146B.1000701@canterbury.ac.nz>

Neal Norwitz wrote:

> I recently asked Guido about name mangling wrt Py3k.  He definitely
> wanted to keep it in.  Unless he changed his mind, I doubt he would
> deprecate it.  His rationale was that there needs to be a way to
> handle name collision with multiple inheritance.

Then maybe it should be beefed up to include the
module name somehow, so that it works reliably
(or at least more reliably than now).

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From rhamph at gmail.com  Tue Dec 13 01:24:33 2005
From: rhamph at gmail.com (Adam Olsen)
Date: Mon, 12 Dec 2005 17:24:33 -0700
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <ca471dc20512121115l2b98c343q675604bf127e3a@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<439CE06D.70600@gmail.com>
	<ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com>
	<d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com>
	<aac2c7cb0512120449g1954e99fk65d24f3bc51cd428@mail.gmail.com>
	<ca471dc20512121115l2b98c343q675604bf127e3a@mail.gmail.com>
Message-ID: <aac2c7cb0512121624j3c455645l9dc9c969e72b935e@mail.gmail.com>

On 12/12/05, Guido van Rossum <guido at python.org> wrote:
> but that's not the same at all. The point of __private is that it uses
> the *static* scope of the code that contains the reference, not the
> (dynamic) type of the object being referenced. With your approach, if
> class A defined __private, *anyone* could use A().__private (but not
> B().__private where B is a subclass of A). The intention is for
> __private to have the right meaning only within the source code for
> class A, but it should work even if type(self) is a subclass of A. (Or
> even if it's unrelated to A, but that's a separate and weaker use
> case.)

Err.. you are of course right.  My intent, however, was to use the
static scope of the code, so let me redo my examples:

class ObjClass(object):
    def foo(self):
        return self.__private

becomes

class ObjClass(object):
    def foo(self):
        return object.__getattribute__(self, '__dict__')[(ObjClass,
'__private')]

Hopefully that example does not get bogged down in poor pseudocode.

--
Adam Olsen, aka Rhamphoryncus

From martin at v.loewis.de  Tue Dec 13 01:27:24 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 13 Dec 2005 01:27:24 +0100
Subject: [Python-Dev] should I really have to install Python before Ican
 build it ?
In-Reply-To: <bbaeab100512121431i99cc49ep85fc0501e88f8f79@mail.gmail.com>
References: <dnkff9$e3n$1@sea.gmane.org>	<e8bf7a530512121122ie8c8d4av99ed47565e6c91d6@mail.gmail.com>	<dnkjtm$umr$1@sea.gmane.org>
	<1134421980.9942.0.camel@linux.site>
	<bbaeab100512121431i99cc49ep85fc0501e88f8f79@mail.gmail.com>
Message-ID: <439E156C.3020805@v.loewis.de>

Brett Cannon wrote:
> What do other people think of this option?  Sounds reasonable to me. 
> if people like it I will add this to the suggested config options
> specified in the dev FAQ.

There is a problem with that option, an no good solution. If you
had built the tree before the update, the object files will have
changed recently. If you then update with use-commit-times, some
files might get changes so they are newer than they used to be,
but still older than their object files.

As a result, a rebuilt will fail to pick up the modified sources,
potentially resulting in a broken interpreter (e.g. when a structure
layout changed, yet this change didn't get compiled into all
object files).

CVS tried to tackle this problem with this approach: on update,
touch the updated files so that they are all new, but have the
same relative order in time as the commit times (e.g. spacing
them apart by one second).

Of course, with subversion changesets, this is futile: the
generated files will be in the same changeset as the sources
(e.g. Python-ast committed together with .asdl, configure
committed together with configure.in). As it is the changeset
which has the commit time, all these files have the *same*
commit time. make(1) then decides "not newer".

The common solution is to have an application-specific update
procedure. For example, we might provide a

make update

target, which is defined as

update:
  svn update
  sleep 1
  test ! Python/Python-ast.c -nt Parser/Python.asdl && \
         touch Python/Python-ast.c
  test ! Include/Python-ast.h -nt Parser/Python.asdl && \
         touch Include/Python-ast.h
  test ! configure -nt configure.in && \
         touch configure

This, of course, assumes that the committers of these files
always regenerated them properly before committing.

See

http://gcc.gnu.org/viewcvs/trunk/contrib/gcc_update?rev=106327&view=markup

for gcc's solution to this problem; gcc developers are
expected to invoke contrib/gcc_update, which will automatically
spread the right time stamps after the update completed.

Regards,
Martin

From martin at v.loewis.de  Tue Dec 13 01:38:41 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 13 Dec 2005 01:38:41 +0100
Subject: [Python-Dev] ElementTree - Why not part of the core? (fwd)
In-Reply-To: <dnja60$h84$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de><dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de><dni8qe$5eq$1@sea.gmane.org>	<439D1B6D.9080208@v.loewis.de>
	<dnja60$h84$1@sea.gmane.org>
Message-ID: <439E1811.3000307@v.loewis.de>

Fredrik Lundh wrote:
> just one question: where do you want the "vendor" checkins ? 

external is fine with me.

I think I would have preferred a "real" vendor branch (i.e.
where you do svn merge to integrate the changes, and where
the subsequent external releases all show up in the same
directory, potentially with copies for symbolic release names),
but if you think that manual merging at each external release
is doable/better/simpler, it's fine with me.

Regards,
Martin

From guido at python.org  Tue Dec 13 01:44:37 2005
From: guido at python.org (Guido van Rossum)
Date: Mon, 12 Dec 2005 16:44:37 -0800
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <200512130001.jBD01bXZ007657@chilled.skew.org>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>
Message-ID: <ca471dc20512121644n6813410av5d0c78cca7e33423@mail.gmail.com>

I'm not so surprised that Fredrik chose to bypass XML-SIG. There
doesn't seem to be a lot of decision power there -- in fact it feels a
bit dead, with a weird mix of too-high-level discussions that don't go
anywhere, plus basic beginner's Q+A.

Also, it would seem that /F's ElementTree doesn't need much vetting --
it seems well established and well-known in the XML-SIG (it was listed
in all the overviews of APIs).

Finally, compared offerings based on e.g. 4thought (sp.?), ElementTree
feels much more practical and hence, might I say it, "pythonic".

--Guido

On 12/12/05, Mike Brown <mike at skew.org> wrote:
> Catching up on some python-dev email, I was surprised to see that things seem
> to be barrelling ahead with the adding of ElementTree to Python core without
> any discussion on XML-SIG. Sidestepping XML-SIG and the proving grounds of
> PyXML in order to satsify the demand for a Pythonic databinding+API for XML in
> stdlib seems to be a bit of a raised middle finger to those folks who have
> worked hard on competing or differently-scoped APIs, each of which deserves a
> bit more peer review than just a single nomination on python-dev, which seems
> to be all it took to obtain a blessing for ElementTree. I have nothing against
> ElementTree, and would like to see more XML processing options in core, but it
> seems to me like the XML-SIG is being deliberately left out of this process.
>
> Just last month, Guido submitted to XML-SIG a Pythonic XML API that he had
> been tinkering with.[1] I don't think anyone was really bold enough to tell
> him what they really thought of it (other than that it is a lot like XIST),
> but it was admirable that he put it up for peer review rather than just
> dropping it into stdlib. Perhaps more importantly, it prompted some discussion
> that more or less acknowledged that these kinds of APIs do seem to be the
> future of XML in Python, and that we should be thinking about bringing some of
> them into PyXML and, ultimately, stdlib. But the problem of how to choose from
> the many options also became immediately apparent.[2] The discussion stalled,
> but I think it should start up again, in the proper forum, rather than letting
> the first-mentioned API supplant the dozen+ alternatives that could also be
> considered as candidates.[3]
>
> Sorry to be a sourpuss.
>
> Mike
> --
>
> [1] http://mail.python.org/pipermail/xml-sig/2005-November/011248.html
>      (Guido's very civil proposal and request for peer review)
> [2] http://mail.python.org/pipermail/xml-sig/2005-November/011252.html (this
>      also summarizes the categories of software/approaches that people are
>      taking to the general problem of working with XML Pythonically)
> [3] http://www.xml.com/pub/a/2004/10/13/py-xml.html (and there are at least
>      3 more databinding APIs that have come out since then)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Dec 13 01:48:17 2005
From: guido at python.org (Guido van Rossum)
Date: Mon, 12 Dec 2005 16:48:17 -0800
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <aac2c7cb0512121624j3c455645l9dc9c969e72b935e@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439CA89B.4030600@colorstudy.com>
	<439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<439CE06D.70600@gmail.com>
	<ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com>
	<d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com>
	<aac2c7cb0512120449g1954e99fk65d24f3bc51cd428@mail.gmail.com>
	<ca471dc20512121115l2b98c343q675604bf127e3a@mail.gmail.com>
	<aac2c7cb0512121624j3c455645l9dc9c969e72b935e@mail.gmail.com>
Message-ID: <ca471dc20512121648l10f33f16yb1846bf882375796@mail.gmail.com>

On 12/12/05, Adam Olsen <rhamph at gmail.com> wrote:
> On 12/12/05, Guido van Rossum <guido at python.org> wrote:
> > but that's not the same at all. The point of __private is that it uses
> > the *static* scope of the code that contains the reference, not the
> > (dynamic) type of the object being referenced. With your approach, if
> > class A defined __private, *anyone* could use A().__private (but not
> > B().__private where B is a subclass of A). The intention is for
> > __private to have the right meaning only within the source code for
> > class A, but it should work even if type(self) is a subclass of A. (Or
> > even if it's unrelated to A, but that's a separate and weaker use
> > case.)
>
> Err.. you are of course right.  My intent, however, was to use the
> static scope of the code, so let me redo my examples:
>
> class ObjClass(object):
>     def foo(self):
>         return self.__private
>
> becomes
>
> class ObjClass(object):
>     def foo(self):
>         return object.__getattribute__(self, '__dict__')[(ObjClass,
> '__private')]
>
> Hopefully that example does not get bogged down in poor pseudocode.

Unfortunately that fails one of the other requirements, which (at the
time of implementation) was minimal impact on the rest of the
interpreter. Since __private isn't limited to self, and attribute
lookup doesn't always result in a dict lookup, this would require a
complete overhaul of the getattr API, both at the C and at the Python
level.

But I guess you already said that when you said """Obviously it
doesn't handle backwards compatibility, so it's more of a "if I could
do it again.." suggestion."""

I think all has been said that can be said about this suggestion.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steven.bethard at gmail.com  Tue Dec 13 01:53:32 2005
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 12 Dec 2005 17:53:32 -0700
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <200512130001.jBD01bXZ007657@chilled.skew.org>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>
Message-ID: <d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>

Mike Brown wrote:
> Catching up on some python-dev email, I was surprised to see that things seem
> to be barrelling ahead with the adding of ElementTree to Python core without
> any discussion on XML-SIG. Sidestepping XML-SIG and the proving grounds of
> PyXML in order to satsify the demand for a Pythonic databinding+API for XML in
> stdlib seems to be a bit of a raised middle finger to those folks who have
> worked hard on competing or differently-scoped APIs, each of which deserves a
> bit more peer review than just a single nomination on python-dev, which seems
> to be all it took to obtain a blessing for ElementTree.

I didn't really feel like the proposal was out of the blue.  The
proposal has been brought up before, both on python-dev[1] and the
python-list[2].  ElementTree has a pretty large following - if you
look at XML-based questions on the python-list, I can almost guarantee
you that someone will give an elementtree solution to it (and not just
Fredrik).  I don't know much about any other APIs, so I'm not going to
try to claim it's the best API or anything, but it is the best of what
seems to have any user visibility on the python-list.

[1]http://mail.python.org/pipermail/python-dev/2005-June/054092.html
[2]http://mail.python.org/pipermail/python-list/2005-December/314288.html

STeVe
--
You can wordify anything if you just verb it.
        --- Bucky Katt, Get Fuzzy

From martin at v.loewis.de  Tue Dec 13 02:00:38 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 13 Dec 2005 02:00:38 +0100
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>
	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>
Message-ID: <439E1D36.1090009@v.loewis.de>

Steven Bethard wrote:
> I didn't really feel like the proposal was out of the blue.  The
> proposal has been brought up before, both on python-dev[1] and the
> python-list[2].  ElementTree has a pretty large following - if you
> look at XML-based questions on the python-list, I can almost guarantee
> you that someone will give an elementtree solution to it (and not just
> Fredrik).  I don't know much about any other APIs, so I'm not going to
> try to claim it's the best API or anything, but it is the best of what
> seems to have any user visibility on the python-list.

It's difficult to establish precise numbers, but I would expect that
most readers of xml-sig are well aware of how DOM and SAX work, perhaps
even better than ElementTree.

My main complaint about this was in the past that it is a Python-only
solution, so people working in multiple languages cannot reuse their
knowledge. It seems that this is irrelevant, as people don't work
in multiple languages so much. I still think that Python should continue
to provide standard APIs, for those that know how things are done
in Java.

Regards,
Martin

From martin at v.loewis.de  Tue Dec 13 02:03:48 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 13 Dec 2005 02:03:48 +0100
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <200512130001.jBD01bXZ007657@chilled.skew.org>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>
Message-ID: <439E1DF4.9000204@v.loewis.de>

Mike Brown wrote:
> Catching up on some python-dev email, I was surprised to see that things seem 
> to be barrelling ahead with the adding of ElementTree to Python core without 
> any discussion on XML-SIG. Sidestepping XML-SIG and the proving grounds of 
> PyXML in order to satsify the demand for a Pythonic databinding+API for XML in 
> stdlib seems to be a bit of a raised middle finger to those folks who have 
> worked hard on competing or differently-scoped APIs, each of which deserves a 
> bit more peer review than just a single nomination on python-dev, which seems 
> to be all it took to obtain a blessing for ElementTree. 

That is not true. The single nomination actually triggered a (admittedly
fast) process, where multiple people spoke in favour, not just a single
one. It also followed a discussion on python-list.

> I have nothing against 
> ElementTree, and would like to see more XML processing options in core, but it 
> seems to me like the XML-SIG is being deliberately left out of this process.

I think your impression is wrong (atleast on my part): I did not
deliberately side-step xml-sig; it just didn't occur to me to have the
discussion there also. I implicitly assumed most people on xml-sig would
agree.

> Just last month, Guido submitted to XML-SIG a Pythonic XML API that he had 
> been tinkering with.[1] I don't think anyone was really bold enough to tell 
> him what they really thought of it (other than that it is a lot like XIST), 
> but it was admirable that he put it up for peer review rather than just 
> dropping it into stdlib.

Again, your impression is somewhat wrong: Guido first submitted the code
to the SF bug tracker; there I commented that he should discuss it on
xml-sig. I based this recommendation on my view that any such library
should see a wider audience first before being admitted to the core;
this library of Guido had (to my knowledge) not been seen by a wider
audience.

This is unlike ElementTree, which had existed for quite some time, and
collected a lot of community feedback.

> But the problem of how to choose from 
> the many options also became immediately apparent.[2] The discussion stalled, 
> but I think it should start up again, in the proper forum, rather than letting 
> the first-mentioned API supplant the dozen+ alternatives that could also be 
> considered as candidates.[3]

Well, this is one of the big problems with XML: there are so many
libraries to chose from, for so many different kinds of applications.
It took me some time to understand what kind of application Guido's
library is targetting - and such an analysis always ends up with
saying "It is like X, but has Y instead".

In this setting, how should we chose a library? In the last round,
it was "let's believe in standards". I personally still believe in
standards, but it appears that the Python community views them as too
bloated.

So as that has more-or-less failed, the next natural approach is
"let's believe in the community". For that, two things need to
happen: the author of the package must indicate that he would like
to see it incorporated, and the users must indicate that they like
the package. Both has happened for ElementTree, but I think it
could happen for other packages, as well.

If it is merely the lack of due process you are complaining about,
and you agree with the result, then IMO nothing would need to be
changed about the result. Discussing it post-factum on xml-sig
might still be valuable.

Regards,
Martin


From steven.bethard at gmail.com  Tue Dec 13 02:11:20 2005
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 12 Dec 2005 18:11:20 -0700
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <439E1D36.1090009@v.loewis.de>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>
	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>
	<439E1D36.1090009@v.loewis.de>
Message-ID: <d11dcfba0512121711i7935d0ccob73d1a9097b8e733@mail.gmail.com>

Martin v. L?wis wrote:
> Steven Bethard wrote:
> > I didn't really feel like the proposal was out of the blue.  The
> > proposal has been brought up before, both on python-dev[1] and the
> > python-list[2].  ElementTree has a pretty large following - if you
> > look at XML-based questions on the python-list, I can almost guarantee
> > you that someone will give an elementtree solution to it (and not just
> > Fredrik).  I don't know much about any other APIs, so I'm not going to
> > try to claim it's the best API or anything, but it is the best of what
> > seems to have any user visibility on the python-list.
>
> It's difficult to establish precise numbers, but I would expect that
> most readers of xml-sig are well aware of how DOM and SAX work, perhaps
> even better than ElementTree.

Sorry, I didn't mean to imply that DOM and SAX (though mainly DOM in
my experience) solutions weren't also offered on the python-list. 
It's just that we already have DOM and SAX APIs in the stdlib.  My
point was mainly that elementtree was the xml module that I've seen
most often cited on python-list that isn't already in the stdlib.

STeVe
--
You can wordify anything if you just verb it.
        --- Bucky Katt, Get Fuzzy

From raymond.hettinger at verizon.net  Tue Dec 13 02:11:57 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Mon, 12 Dec 2005 20:11:57 -0500
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <439E1DF4.9000204@v.loewis.de>
Message-ID: <008101c5ff82$37deb220$9418c797@oemcomputer>

> The single nomination actually triggered a (admittedly
> fast) process, where multiple people spoke in favour, not just a
single
> one. It also followed a discussion on python-list.

Also, there were silent +1 votes from people like me who followed all
the posts and saw no need to alter the direction of the discussion.
FWIW, I've been hoping for this for a long time. 

In retrospect, CCing the XML list would have been nice but I don't think
it would have changed the outcome.



Raymond


From mike at skew.org  Tue Dec 13 02:19:29 2005
From: mike at skew.org (Mike Brown)
Date: Mon, 12 Dec 2005 18:19:29 -0700 (MST)
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <439E1C0C.4040601@v.loewis.de>
Message-ID: <200512130119.jBD1JTfB008430@chilled.skew.org>

"Martin v. L> So as that has more-or-less failed, the next natural approach is
> "let's believe in the community". For that, two things need to
> happen: the author of the package must indicate that he would like
> to see it incorporated, and the users must indicate that they like
> the package. Both has happened for ElementTree, but I think it
> could happen for other packages, as well.
> 
> If it is merely the lack of due process you are complaining about,
> and you agree with the result, then IMO nothing would need to be
> changed about the result. Discussing it post-factum on xml-sig
> might still be valuable.

Thanks Martin and others for responding.

I full agree that ElementTree has proven to be useful, popular, and stable, 
and probably no one would object to ElementTree being given the endorsement 
that is implicit in its being made a part of stdlib. 

The lack of due process, given that XML-SIG seems to exist largely to provide 
that very service for all things XML in Python, is indeed all I'm complaining 
about. I am happy that for once, there is momentum behind this sort of thing,
and more power to you for that.

My fears are just that 1. XML-SIG is being seen as either irrelevant or as an 
obstacle (perhaps due to the friction between Fredrik and Uche) and are thus 
being sidestepped, and 2. other libs that could/should be contenders (Amara 
and 4Suite are not in this list, by the way) are going to become further 
marginalized by virtue of the fact that people will say "well, we have 
ElementTree in stdlib already, why do we need (fill in the blank)?"

I suppose the same kind of implicit endorsements were given to minidom and 
SAX, and that obviously hasn't prevented people from going out and using 
ElementTree, lxml, etc., so I don't know... I can't predict the future. I'd 
just feel better about it if everyone on XML-SIG, where people hang out 
because they have a definite interest in this kind of thing, knew what was 
going on. Some authors of other libs may not even be aware that they could so 
easily have their code whisked into stdlib, if it's solid enough.

Mike

From pje at telecommunity.com  Tue Dec 13 03:21:06 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 12 Dec 2005 21:21:06 -0500
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <200512130119.jBD1JTfB008430@chilled.skew.org>
References: <439E1C0C.4040601@v.loewis.de>
Message-ID: <5.1.1.6.0.20051212211429.01f59d60@mail.telecommunity.com>

At 06:19 PM 12/12/2005 -0700, Mike Brown wrote:
>Some authors of other libs may not even be aware that they could so
>easily have their code whisked into stdlib, if it's solid enough.

But here the definition of "solid enough" includes such credits as being 
written by the primary author of CPython's implementations of Unicode and 
regular expressions, and who can be reasonably be believed to be around to 
support and maintain the package for some time.  I don't know who the "some 
authors" you mention are, but those are pretty tough credentials to match, 
as are the apparent popularity, Pythonicness, and performance of ElementTree.

I find it rather hard to believe that there's another XML library that 
could have gotten through the approval process anywhere near as easily.


From nnorwitz at gmail.com  Tue Dec 13 04:38:39 2005
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 12 Dec 2005 19:38:39 -0800
Subject: [Python-Dev] __builtin__ vs __builtins__
Message-ID: <ee2a432c0512121938o1e86eab2ub58d46c8b913bf18@mail.gmail.com>

Below is Guido's response to my question: Can we start getting rid of
__builtins__ (or __builtin__) at least for py3k?

Having both builtin versions is confusing, how can we improve the situation?

n

---------- Forwarded message ----------
From: Guido van Rossum <guido at python.org>


Couple of loose thoughts:

- Having __builtins__ and __builtin__ both is clearly a bad idea.

- Long ago, __builtin__ was just called builtin; I'm not sure I still
agree with the reasoning that made me change it. After all, we don't
have __sys__. But we *do* have __main__, and __builtin__ is special at
least in the sense that modifying it has a global effect. (But then
again, so does modifying sys.) I still think the case for __main__ is
much stronger than for __builtin__ and wouldn't mind renaming the
latter back to builtin.

- Making __builtins__ always be a dict would simplify some code; but
it really means that vars() must be hacked to suppress it in
interactive mode; I really wouldn't like to see the output of vars()
include the entire __builtins__ dict.

- Another alternative might be to merge the __builtin__ and
__builtins__ functionality (and call it __builtin__). This would slow
down some stuff (always one extra indirection to get from __builtin__
to __builtin__.__dict__ which is where the built-ins are looked up)
but that could be fixed by caching __builtin__.__dict__ in the C frame
(I'm fine with the rule that you can't modify your own __builtin__; I
think that rule already exists).

- This is probably worth a few smart people mulling it over some
more... Python-dev?

From skip at pobox.com  Tue Dec 13 05:22:33 2005
From: skip at pobox.com (skip@pobox.com)
Date: Mon, 12 Dec 2005 22:22:33 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134411382.5676.35.camel@geddy.wooz.org>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
Message-ID: <17310.19593.900320.428826@montanaro.dyndns.org>


    >> I looked at that too, but most of these didn't jump out at me.  I'll 
    >> copy in the parts that aren't already in PEP 8 that seem possible:
    >> 
    >> From-imports should follow non-from imports.  Dotted imports should
    >> follow non-dotted imports.  Non-dotted imports should be grouped by
    >> increasing length, while dotted imports should be grouped roughly
    >> alphabetically.
    >> 
    >> This seems too complex to me for PEP 8.

    Barry> Really?  ISTR adopting this convention from Guido, but I'm not
    Barry> 100% sure about that.  After having used it for several years
    Barry> now, I do really like this style, but I'm willing to leave the
    Barry> recommendation out of PEP 8.

This is subjective enough that I would think some rationale explaining this
convention should be given.  Personally, I group imports into three sections
as follows:

    * Python core modules/packages

    * Third-party modules/packages

    * Local modules/packages

I can't explain why I do it that way.  I guess it just satisfies some inner
hobgoblin.

Skip


From skip at pobox.com  Tue Dec 13 05:38:26 2005
From: skip at pobox.com (skip@pobox.com)
Date: Mon, 12 Dec 2005 22:38:26 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439DC6EC.60205@zope.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com>
	<1134395316.11373.12.camel@localhost> <439D8911.1010505@zope.com>
	<17309.40570.699501.68209@montanaro.dyndns.org>
	<439DC6EC.60205@zope.com>
Message-ID: <17310.20546.584413.501771@montanaro.dyndns.org>


    Jim> I don't understand this argument.  Any mutating method or property
    Jim> invoked by foreign code changes an object's state.  

Sure, but the only place I need to look for direct changes to the object's
state are in the object's own code.

    Jim> If you provide a property or a pair if accessors that just sets and
    Jim> gets an attribute with a slightly different name, that affords no
    Jim> more protection than if people were setting the attribute directly.

Sure it does.  Suppose I get an exception in my code because some bit of
code somewhere broke my assumptions about the values an attribute could
assume.  If that attribute is only set by the object's own code, I can more
easily debug it (stick in a print or an assert in the places where the
attribute changes, etc).  If some external bit of code does something like

    self.foo = Foo()
    ...
    self.foo.attr = None

then later in Foo's code I have something like

    self.attr.callme()

The first thing I need to do is figure out who stomped on self.attr.  That
can be time-consuming if I don't necessarily know where the stomping
occurred.

At work we use Python for very rapid development of trading applications.
Today I think we made about a half-dozen micro releases fixing bugs and our
traders tried each one immediately live.  Much of the design is carried
around in our heads or consists of a few equations scribbled on sheets of
paper.  As you might imagine, it's a very lively environment.  Localizing
attribute modifications is a very good thing for us, even if they are simply
one-line set methods.

    Jim> If you don't want external code to change an attribute, don't
    Jim> expose it through a public API.

I suppose "public" is subject to some interpretation.  Just because I don't
prefix an attribute with an underscore doesn't mean I've implicitly declared
it public.  I assume that people will familiarize themselves with the
callable methods of an object and only use direct attribute access if I
haven't provided the necessary methods.

Skip

From guido at python.org  Tue Dec 13 05:43:14 2005
From: guido at python.org (Guido van Rossum)
Date: Mon, 12 Dec 2005 20:43:14 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <17310.19593.900320.428826@montanaro.dyndns.org>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<17310.19593.900320.428826@montanaro.dyndns.org>
Message-ID: <ca471dc20512122043u4db11498h6b407a8500808051@mail.gmail.com>

On 12/12/05, skip at pobox.com <skip at pobox.com> wrote:
>
>     >> I looked at that too, but most of these didn't jump out at me.  I'll
>     >> copy in the parts that aren't already in PEP 8 that seem possible:
>     >>
>     >> From-imports should follow non-from imports.  Dotted imports should
>     >> follow non-dotted imports.  Non-dotted imports should be grouped by
>     >> increasing length, while dotted imports should be grouped roughly
>     >> alphabetically.
>     >> This seems too complex to me for PEP 8.
>
>     Barry> Really?  ISTR adopting this convention from Guido, but I'm not
>     Barry> 100% sure about that.  After having used it for several years
>     Barry> now, I do really like this style, but I'm willing to leave the
>     Barry> recommendation out of PEP 8.
>
> This is subjective enough that I would think some rationale explaining this
> convention should be given.  Personally, I group imports into three sections
> as follows:
>
>     * Python core modules/packages
>
>     * Third-party modules/packages
>
>     * Local modules/packages
>
> I can't explain why I do it that way.  I guess it just satisfies some inner
> hobgoblin.

This is what I recommend too, and PEP 8 should recommend this.

While I admit to a kind of secret enjoyment when I see the standard
library module imports arranged by increasing length, I don't think
that ought to be put in the PEP. (I remember once seeing a friend's
books arranged by size on their shelves and finding it bizarre. You
should have the same feeling when you see imports arranged that way.)
A more rational approach would be to do them alphabetically.

Putting the from...import ones last makes sense if only because it's
not obvious where they fit in the alphabetization.

Dotted non-from imports (e.g. import test.pystone) are rare enough
that they don't deserve a special rule; if you want me to give a rule,
I think they should be mixed in with the undotted ones,
alphabetically.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Tue Dec 13 05:48:36 2005
From: skip at pobox.com (skip@pobox.com)
Date: Mon, 12 Dec 2005 22:48:36 -0600
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <439E1D36.1090009@v.loewis.de>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>
	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>
	<439E1D36.1090009@v.loewis.de>
Message-ID: <17310.21156.907292.278567@montanaro.dyndns.org>


    Martin> It's difficult to establish precise numbers, but I would expect
    Martin> that most readers of xml-sig are well aware of how DOM and SAX
    Martin> work, perhaps even better than ElementTree.

Perhaps the corollary is that people who are not xml-sig readers will likely
be put off by DOM and SAX.  I couldn't tell you what they do, just that they
were Too Hard (tm) for me to bother with XML in most situations.  Then
ElementTree came along.

Skip

From ianb at colorstudy.com  Tue Dec 13 05:47:12 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 12 Dec 2005 22:47:12 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <17310.19593.900320.428826@montanaro.dyndns.org>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<17310.19593.900320.428826@montanaro.dyndns.org>
Message-ID: <439E5250.7050004@colorstudy.com>

skip at pobox.com wrote:
> This is subjective enough that I would think some rationale explaining this
> convention should be given.  Personally, I group imports into three sections
> as follows:
> 
>     * Python core modules/packages
> 
>     * Third-party modules/packages
> 
>     * Local modules/packages

This is already in PEP 8:

     - Imports are always put at the top of the file, just after any
       module comments and docstrings, and before module globals and
       constants.  Imports should be grouped, with the order being

       1. standard library imports
       2. related major package imports (i.e. all email package imports 
next)
       3. application specific imports

       You should put a blank line between each group of imports.

I would suggest that it should also say that __all__ goes after imports. 
  But otherwise it's all good; the Mailman style guide just goes into 
greater detail.

-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org

From guido at python.org  Tue Dec 13 05:52:55 2005
From: guido at python.org (Guido van Rossum)
Date: Mon, 12 Dec 2005 20:52:55 -0800
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <17310.21156.907292.278567@montanaro.dyndns.org>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>
	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>
	<439E1D36.1090009@v.loewis.de>
	<17310.21156.907292.278567@montanaro.dyndns.org>
Message-ID: <ca471dc20512122052k4b985d98m9ce1662858f2d1bc@mail.gmail.com>

On 12/12/05, skip at pobox.com <skip at pobox.com> wrote:
>
>     Martin> It's difficult to establish precise numbers, but I would expect
>     Martin> that most readers of xml-sig are well aware of how DOM and SAX
>     Martin> work, perhaps even better than ElementTree.
>
> Perhaps the corollary is that people who are not xml-sig readers will likely
> be put off by DOM and SAX.  I couldn't tell you what they do, just that they
> were Too Hard (tm) for me to bother with XML in most situations.  Then
> ElementTree came along.

It seems pretty clear why DOM isn't Pythonic: it doesn't use Python's
standard APIs for things that conceptually are "just" lists and dicts,
or at least sequences and mappings. Also, the memory footprint is a
bit outlandish.

I don't think that SAX is unpythonic, but it's pretty low-level and
mostly of use to people writing higher-level XML parsers (my parsexml
module uses it).

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Dec 13 05:56:15 2005
From: guido at python.org (Guido van Rossum)
Date: Mon, 12 Dec 2005 20:56:15 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439E5250.7050004@colorstudy.com>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<17310.19593.900320.428826@montanaro.dyndns.org>
	<439E5250.7050004@colorstudy.com>
Message-ID: <ca471dc20512122056o338762a9u9421f1cb66917ede@mail.gmail.com>

On 12/12/05, Ian Bicking <ianb at colorstudy.com> wrote:
> skip at pobox.com wrote:
> > This is subjective enough that I would think some rationale explaining this
> > convention should be given.  Personally, I group imports into three sections
> > as follows:
> >
> >     * Python core modules/packages
> >
> >     * Third-party modules/packages
> >
> >     * Local modules/packages
>
> This is already in PEP 8:
>
>      - Imports are always put at the top of the file, just after any
>        module comments and docstrings, and before module globals and
>        constants.  Imports should be grouped, with the order being
>
>        1. standard library imports
>        2. related major package imports (i.e. all email package imports next)
>        3. application specific imports

Hm. I like Skip's list better; "related major package imports" is a
bit vague and ambiguous. It seems to have been written before email
became a standard library module; also it clearly meant to say "e.g."
instead of "i.e.".

>        You should put a blank line between each group of imports.
>
> I would suggest that it should also say that __all__ goes after imports.

+1

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From t-meyer at ihug.co.nz  Tue Dec 13 05:49:30 2005
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Tue, 13 Dec 2005 17:49:30 +1300
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439E5250.7050004@colorstudy.com>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<17310.19593.900320.428826@montanaro.dyndns.org>
	<439E5250.7050004@colorstudy.com>
Message-ID: <D8F3389A-C27A-4AA2-A1DD-755E68FCE7C9@ihug.co.nz>

>>     * Python core modules/packages
>>
>>     * Third-party modules/packages
>>
>>     * Local modules/packages
>
> This is already in PEP 8:
[...]
>        1. standard library imports
>        2. related major package imports (i.e. all email package  
> imports
> next)
>        3. application specific imports
>
>        You should put a blank line between each group of imports.

Does this pre-date the email package being included in the standard  
library?  As it is, asterisk 2 and #2 don't appear to match.  If that  
is the case, then perhaps something else should be chosen?

  =Tony.Meyer 

From martin at v.loewis.de  Tue Dec 13 09:59:41 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 13 Dec 2005 09:59:41 +0100
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <200512130119.jBD1JTfB008430@chilled.skew.org>
References: <200512130119.jBD1JTfB008430@chilled.skew.org>
Message-ID: <439E8D7D.9010609@v.loewis.de>

Mike Brown wrote:
> My fears are just that 1. XML-SIG is being seen as either irrelevant or as an 
> obstacle (perhaps due to the friction between Fredrik and Uche) and are thus 
> being sidestepped, and 2. other libs that could/should be contenders (Amara 
> and 4Suite are not in this list, by the way) are going to become further 
> marginalized by virtue of the fact that people will say "well, we have 
> ElementTree in stdlib already, why do we need (fill in the blank)?"

And if they say so, they might be right! I firmly believe that the
standard library should be a community-driven thing, not a
committee-driven one. For that, two things need to happen for a library
to become included:
1. the author of the library must explicitly offer it for inclusion.
   there is no point in "hijacking" the package into the library,
   even if the package license would allow to do so (factually, it
   typically doesn't, because it typically doesn't allow redistribution
   under a different license).
   So without the author's explicit endorsement, and promise to
   maintain it for some time (or some other set of people offering
   that), nothing will happen to (fill in the blank).
2. the users must indicate that they want to see the package as part
   of the library. Again, just that the author would like to contribute
   it isn't enough - there must be people supporting the inclusion of
   the package.

Traditionally, we had a third step:
3. The BDFL must pronounce inclusion of the package. Now, while Guido
   has a firm vision for how the language proper should evolve, he
   often indicated that he can't really comment on some specific library
   because he doesn't know anything about the functionality it provides.
   So in the case of libraries, this requirement often is waived.

> I suppose the same kind of implicit endorsements were given to minidom and 
> SAX, and that obviously hasn't prevented people from going out and using 
> ElementTree, lxml, etc., so I don't know... I can't predict the future. I'd 
> just feel better about it if everyone on XML-SIG, where people hang out 
> because they have a definite interest in this kind of thing, knew what was 
> going on. Some authors of other libs may not even be aware that they could so 
> easily have their code whisked into stdlib, if it's solid enough.

That's part of the process. They could have read PEP 2, so they could
have known to write a PEP and get it discussed. When they don't know
that, they fail the basic test of "author support": if the author isn't
really behind the integration of the package, the package really
shouldn't be integrated (this is why I first predicted ElementTree
would never become part of the library, because I assumed /F would
not like the idea).

Regards,
Martin

From steve at holdenweb.com  Tue Dec 13 10:22:45 2005
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 13 Dec 2005 09:22:45 +0000
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <5.1.1.6.0.20051212211429.01f59d60@mail.telecommunity.com>
References: <439E1C0C.4040601@v.loewis.de>
	<200512130119.jBD1JTfB008430@chilled.skew.org>
	<5.1.1.6.0.20051212211429.01f59d60@mail.telecommunity.com>
Message-ID: <dnm3t7$727$1@sea.gmane.org>

Phillip J. Eby wrote:
> At 06:19 PM 12/12/2005 -0700, Mike Brown wrote:
> 
>>Some authors of other libs may not even be aware that they could so
>>easily have their code whisked into stdlib, if it's solid enough.
> 
> 
> But here the definition of "solid enough" includes such credits as being 
> written by the primary author of CPython's implementations of Unicode and 
> regular expressions, and who can be reasonably be believed to be around to 
> support and maintain the package for some time.  I don't know who the "some 
> authors" you mention are, but those are pretty tough credentials to match, 
> as are the apparent popularity, Pythonicness, and performance of ElementTree.
> 
> I find it rather hard to believe that there's another XML library that 
> could have gotten through the approval process anywhere near as easily.
> 
This can be observed simply by looking at who posts to python-dev. 
Certainly we see input from Fredrik on a fairly regular basis, whereas 
others appear infrequently or not at all. Absence from python-dev can't 
really be seen as expressing any keenness at all for one's code to be 
included in the core. If the authors of code aren't bothered about its 
promotion to the core I hardly think anyone else should be.

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From ncoghlan at gmail.com  Tue Dec 13 10:30:31 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 13 Dec 2005 19:30:31 +1000
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <17309.38679.642021.215357@montanaro.dyndns.org>
References: <439CA89B.4030600@colorstudy.com>
	<17308.55295.648767.433858@montanaro.dyndns.org>
	<20051211195752.CB4A.JCARLSON@uci.edu> <439CF760.8040408@gmail.com>
	<17309.38679.642021.215357@montanaro.dyndns.org>
Message-ID: <439E94B7.7010605@gmail.com>

skip at pobox.com wrote:
>     Nick> Any old code could be fixed by putting "from types import
>     Nick> ClassType as __metaclass__" at the top of the affected modules.
> 
> Which would be, what, 90% of all Python code written that defines classes?

I generally don't allow old-style classes in any code I have control over 
(well, aside from exceptions). Having to type '(object)' all the time is 
annoying, but less annoying than trying to figure out which set of semantics a 
given class is using. My interpreter startup script even includes 
"__metaclass__ = None" in order to disable the implicit metaclass.

I think it's an artifact of only seriously starting to use Python with version 
2.2.2 - I don't really understand how old-style classes work, so I try to 
avoid using them.

However, you raise a fair point, which is why I raised the suggestion of 
respecting a "__metaclass__" definition in the builtins, allowing application 
developers to perform their own new-style class smoke test prior to Py3k.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From walter at livinglogic.de  Tue Dec 13 10:56:33 2005
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue, 13 Dec 2005 10:56:33 +0100
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <ca471dc20512122052k4b985d98m9ce1662858f2d1bc@mail.gmail.com>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>	<439E1D36.1090009@v.loewis.de>	<17310.21156.907292.278567@montanaro.dyndns.org>
	<ca471dc20512122052k4b985d98m9ce1662858f2d1bc@mail.gmail.com>
Message-ID: <439E9AD1.9080103@livinglogic.de>

Guido van Rossum wrote:

> On 12/12/05, skip at pobox.com <skip at pobox.com> wrote:
> 
>>    Martin> It's difficult to establish precise numbers, but I would expect
>>    Martin> that most readers of xml-sig are well aware of how DOM and SAX
>>    Martin> work, perhaps even better than ElementTree.
>>
>>Perhaps the corollary is that people who are not xml-sig readers will likely
>>be put off by DOM and SAX.  I couldn't tell you what they do, just that they
>>were Too Hard (tm) for me to bother with XML in most situations.  Then
>>ElementTree came along.
> 
> It seems pretty clear why DOM isn't Pythonic: it doesn't use Python's
> standard APIs for things that conceptually are "just" lists and dicts,
> or at least sequences and mappings. Also, the memory footprint is a
> bit outlandish.
> 
> I don't think that SAX is unpythonic, but it's pretty low-level and
> mostly of use to people writing higher-level XML parsers (my parsexml
> module uses it).

Having to define classes that conform to a certain API and registering 
instances of those classes as callbacks with the parser doesn't look 
that pythonic to me. An iterator API seems much more pythonic.

Then again, pythonic is whatever you say that it is. ;)

Bye,
    Walter D?rwald

From walter at livinglogic.de  Tue Dec 13 10:38:44 2005
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue, 13 Dec 2005 10:38:44 +0100
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <439E1D36.1090009@v.loewis.de>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>
	<439E1D36.1090009@v.loewis.de>
Message-ID: <439E96A4.6030101@livinglogic.de>

Martin v. L?wis wrote:

> [...]
>  
> It's difficult to establish precise numbers, but I would expect that
> most readers of xml-sig are well aware of how DOM and SAX work, perhaps
> even better than ElementTree.
> 
> My main complaint about this was in the past that it is a Python-only
> solution, so people working in multiple languages cannot reuse their
> knowledge. It seems that this is irrelevant, as people don't work
> in multiple languages so much. I still think that Python should continue
> to provide standard APIs, for those that know how things are done
> in Java.

I think there could be a middle ground between one API for all XML 
processors in all languages (SAX+DOM) and every XML package having its 
own custom API. A common tree API for all Python XML processors might be 
beneficial. Maybe ElementTree can become that API? Or maybe a subset of 
the ElementTree API (I don't think the text and trail attributes should 
be in that API).

Bye,
    Walter D?rwald

From ncoghlan at gmail.com  Tue Dec 13 12:09:47 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 13 Dec 2005 21:09:47 +1000
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <439E9AD1.9080103@livinglogic.de>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>	<439E1D36.1090009@v.loewis.de>	<17310.21156.907292.278567@montanaro.dyndns.org>	<ca471dc20512122052k4b985d98m9ce1662858f2d1bc@mail.gmail.com>
	<439E9AD1.9080103@livinglogic.de>
Message-ID: <439EABFB.3020405@gmail.com>

Walter D?rwald wrote:
> Having to define classes that conform to a certain API and registering 
> instances of those classes as callbacks with the parser doesn't look 
> that pythonic to me. An iterator API seems much more pythonic.

If this is a comment on the ElementTree API, then /F must agree with you - 
iterparse [1] was added to the API earlier this year. . .

Cheers,
Nick.

[1] http://effbot.org/zone/element-iterparse.htm

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From gjc at inescporto.pt  Tue Dec 13 12:13:12 2005
From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro)
Date: Tue, 13 Dec 2005 11:13:12 +0000
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <17310.20546.584413.501771@montanaro.dyndns.org>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <1134395316.11373.12.camel@localhost>
	<439D8911.1010505@zope.com>
	<17309.40570.699501.68209@montanaro.dyndns.org>
	<439DC6EC.60205@zope.com>
	<17310.20546.584413.501771@montanaro.dyndns.org>
Message-ID: <1134472392.8104.9.camel@localhost>

Seg, 2005-12-12 ?s 22:38 -0600, skip at pobox.com escreveu:
>     Jim> I don't understand this argument.  Any mutating method or property
>     Jim> invoked by foreign code changes an object's state.  
> 
> Sure, but the only place I need to look for direct changes to the object's
> state are in the object's own code.
> 
>     Jim> If you provide a property or a pair if accessors that just sets and
>     Jim> gets an attribute with a slightly different name, that affords no
>     Jim> more protection than if people were setting the attribute directly.
> 
> Sure it does.  Suppose I get an exception in my code because some bit of
> code somewhere broke my assumptions about the values an attribute could
> assume.  If that attribute is only set by the object's own code, I can more
> easily debug it (stick in a print or an assert in the places where the
> attribute changes, etc).  If some external bit of code does something like
> 
>     self.foo = Foo()
>     ...
>     self.foo.attr = None
> 
> then later in Foo's code I have something like
> 
>     self.attr.callme()
> 
> The first thing I need to do is figure out who stomped on self.attr.

  I have never done this, but in theory you could replace attr with a
property whose getter uses sys._getframe() to log each modification of
the attribute, thus easily find out who did "self.foo.attr = None".
Almost like gdb's 'watch' command.

  Regards.

-- 
Gustavo J. A. M. Carneiro
<gjc at inescporto.pt> <gustavo at users.sourceforge.net>
The universe is always one step beyond logic.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Esta =?ISO-8859-1?Q?=E9?= uma parte de mensagem
	assinada digitalmente
Url : http://mail.python.org/pipermail/python-dev/attachments/20051213/16eeb7eb/attachment.pgp

From fredrik at pythonware.com  Tue Dec 13 12:41:44 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 13 Dec 2005 12:41:44 +0100
Subject: [Python-Dev] ElementTree in stdlib
References: <200512130001.jBD01bXZ007657@chilled.skew.org>	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>	<439E1D36.1090009@v.loewis.de>	<17310.21156.907292.278567@montanaro.dyndns.org>	<ca471dc20512122052k4b985d98m9ce1662858f2d1bc@mail.gmail.com><439E9AD1.9080103@livinglogic.de>
	<439EABFB.3020405@gmail.com>
Message-ID: <dnmc1q$km$1@sea.gmane.org>

Nick Coghlan wrote:

> > Having to define classes that conform to a certain API and registering
> > instances of those classes as callbacks with the parser doesn't look
> > that pythonic to me. An iterator API seems much more pythonic.
>
> If this is a comment on the ElementTree API, then /F must agree with you -
> iterparse was added to the API earlier this year. . .

When xml.sax was added to Python, the standard approach was to create
parsers that implemented the consumer pattern [1] and called methods
either on the parser class itself, or on a target object.  Examples include
sgmllib, htmllib/formatter, and xmllib.

After the discovery of efficient "pull parsing" patterns [2] and "using iterators
to invert program logic" patterns (see e.g. the "anonymous blocks" thread
from april this year [3], which generated a whole bunch of interesting PEPs),
things have changed a bit.

</F>

1) http://effbot.org/zone/consumer.htm

2) http://mail.python.org/pipermail/xml-sig/2000-May/002335.html

(Paul's xml.dom.pulldom module did make it into the standard library,
but it don't seem to be used much, for some unknown reason...)

3) http://mail.python.org/pipermail/python-dev/2005-April/052753.html
(lots of interesting posts here)




From jim at zope.com  Tue Dec 13 12:45:42 2005
From: jim at zope.com (Jim Fulton)
Date: Tue, 13 Dec 2005 06:45:42 -0500
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <ca471dc20512121117p3332c3a1k846deab37e4050c0@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>	
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>	
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>	
	<1f7befae0512111818x13ab3185w48bd365d0b5fe5f4@mail.gmail.com>	
	<439D6012.4060609@zope.com>
	<ca471dc20512121117p3332c3a1k846deab37e4050c0@mail.gmail.com>
Message-ID: <439EB466.9000109@zope.com>

Guido van Rossum wrote:
> On 12/12/05, Jim Fulton <jim at zope.com> wrote:
> 
>>In practice, I don't agree that it works fine.  Inevitably, someone
>>finds a need to access a "private" variable in a subclass.  Or
>>even in the original class, you find some need to use something like
>>__getattr__ where the implicit name mangling doesn't come into play
>>and you have to emulate the name mangling.  Or perhaps someone wants
>>to examine the value of one of these variables in the debugger.
>>In my experience, almost every time someone uses the __private
>>trick, they or someone else comes to regret it.
>>
>>OTOH, explicit name mangling provides the benefits of implicit
>>name mangling without it's drawbacks.
> 
> 
> I half agree. I've seen many classes overuse __private.

As I point out above, it's not just a matter of overuse.  It
is only recognized by the compiler, so it doesn't work with getattr.
And of couuse, there's the debugger issue.  I've often found
cases where, even when I was using it correctly, I had to do manual
name mangling myself.  Anytime one has to perform weird tricks
in Python to work around magic should be a warning sign.

 > But that's a
> separate issue from not having the feature at all; you might as well
> argue against private in Java or C++.

I'm not arguing against the feature but against it's implementation.

My intuition is that the explicit name magling approach is more in
keeping with Python's way of doing things.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From jim at zope.com  Tue Dec 13 12:57:05 2005
From: jim at zope.com (Jim Fulton)
Date: Tue, 13 Dec 2005 06:57:05 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439DDCD2.10803@colorstudy.com>
References: <4399F967.3080300@colorstudy.com>	<1134168683.19370.29.camel@geddy.wooz.org>	<439A110E.8090605@colorstudy.com>	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
Message-ID: <439EB711.5030005@zope.com>

Ian Bicking wrote:
> Barry Warsaw wrote:
...

>>>This seems too complex to me for PEP 8.
>>
>>
>>Really?  ISTR adopting this convention from Guido, but I'm not 100% sure
>>about that.  After having used it for several years now, I do really
>>like this style, but I'm willing to leave the recommendation out of PEP
>>8.
> 
> 
> It seems so exacting to me;

Me too.

> stdlib, external modules, internal modules 
> seems like enough ordering to me.  If you want to order things more 
> exactly, sure, but I don't really see the point personally.  Since I 
> can't assume as a reader that imports are ordered in any way I have to 
> search to be sure of what's there.  The grouping help me browse, but I'd 
> hope that the import list is short enough that I don't need to use 
> alphabetization to scan for a module.

Personally, I don't find the stdlib/external distinction to be useful.

Personally, I'd rather just sort aphabetically based on dotted package
name.  Because packages provide meaningful groupings to begin with,
this approach provides the most meaningful groupings to me.  (All of
my "internal" modules are in packages.)  When scanning imports, I
don't want to have to think about whether a module is internal or
external.  I've got enough to think about without that. :)

Frankly, I'd be as happy t see the PEP be silent on module ordering.

Jim


-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From jim at zope.com  Tue Dec 13 12:59:39 2005
From: jim at zope.com (Jim Fulton)
Date: Tue, 13 Dec 2005 06:59:39 -0500
Subject: [Python-Dev] Import order (was Re:  PEP 8 updates/clarifications)
In-Reply-To: <17310.19593.900320.428826@montanaro.dyndns.org>
References: <4399F967.3080300@colorstudy.com>	<1134168683.19370.29.camel@geddy.wooz.org>	<439A110E.8090605@colorstudy.com>	<1134411382.5676.35.camel@geddy.wooz.org>
	<17310.19593.900320.428826@montanaro.dyndns.org>
Message-ID: <439EB7AB.9040707@zope.com>

skip at pobox.com wrote:
...
> This is subjective enough that I would think some rationale explaining this
> convention should be given.

This is subjective enough that I don't think it should be in the PEP.
Sometimes, less is more.

JIm

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From jim at zope.com  Tue Dec 13 13:14:14 2005
From: jim at zope.com (Jim Fulton)
Date: Tue, 13 Dec 2005 07:14:14 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <17310.20546.584413.501771@montanaro.dyndns.org>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com>
	<1134395316.11373.12.camel@localhost> <439D8911.1010505@zope.com>
	<17309.40570.699501.68209@montanaro.dyndns.org>
	<439DC6EC.60205@zope.com>
	<17310.20546.584413.501771@montanaro.dyndns.org>
Message-ID: <439EBB16.3000704@zope.com>

skip at pobox.com wrote:
>     Jim> I don't understand this argument.  Any mutating method or property
>     Jim> invoked by foreign code changes an object's state.  
> 
> Sure, but the only place I need to look for direct changes to the object's
> state are in the object's own code.
 >
>     Jim> If you provide a property or a pair if accessors that just sets and
>     Jim> gets an attribute with a slightly different name, that affords no
>     Jim> more protection than if people were setting the attribute directly.
> 
> Sure it does.  Suppose I get an exception in my code because some bit of
> code somewhere broke my assumptions about the values an attribute could
> assume.  If that attribute is only set by the object's own code, I can more
> easily debug it (stick in a print or an assert in the places where the
> attribute changes, etc).  If some external bit of code does something like
> 
>     self.foo = Foo()
>     ...
>     self.foo.attr = None
> 
> then later in Foo's code I have something like
> 
>     self.attr.callme()
> 
> The first thing I need to do is figure out who stomped on self.attr.  That
> can be time-consuming if I don't necessarily know where the stomping
> occurred.

I just don't buy this argument. For trivial accessors and properties,
you can't just look at your code to know where the changes are initiated.
For debugging purposes, it's easy to add a property to allow debugging
of attribute assignment.

> At work we use Python for very rapid development of trading applications.
> Today I think we made about a half-dozen micro releases fixing bugs and our
> traders tried each one immediately live.  Much of the design is carried
> around in our heads or consists of a few equations scribbled on sheets of
> paper.  As you might imagine, it's a very lively environment.  Localizing
> attribute modifications is a very good thing for us, even if they are simply
> one-line set methods.

Having to write accessors for all your public methods doesn't seem consistent
with rapid development.  It increases the ceremony of development and adds
lots of meaningless boilerplate that readers of the code have to wade through.
Note that they can't just skip over it, because they can't know if you've slipped
something meaningful into one of these accessors.

>     Jim> If you don't want external code to change an attribute, don't
>     Jim> expose it through a public API.
> 
> I suppose "public" is subject to some interpretation.  Just because I don't
> prefix an attribute with an underscore doesn't mean I've implicitly declared
> it public.  I assume that people will familiarize themselves with the
> callable methods of an object and only use direct attribute access if I
> haven't provided the necessary methods.

A better approach is to document the API for your classes and expect
people to use that API.  Prepending an underscore documents that a
variable or method is internal.  (Of course, there's still the subclassing
API to deal with, if you need one, but that's a separate issue.)

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From jim at zope.com  Tue Dec 13 13:16:44 2005
From: jim at zope.com (Jim Fulton)
Date: Tue, 13 Dec 2005 07:16:44 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134411911.950.43.camel@geddy.wooz.org>
References: <4399F967.3080300@colorstudy.com>
	<439C51C8.2010909@zope.com>	<439CA89B.4030600@colorstudy.com>
	<1134411911.950.43.camel@geddy.wooz.org>
Message-ID: <439EBBAC.6090900@zope.com>

Barry Warsaw wrote:
> On Sun, 2005-12-11 at 16:30 -0600, Ian Bicking wrote:
> 
> 
>>Potentially it could be added that the whole issue can often be avoided 
>>when an object's methods perform actions instead of returning attributes 
>>of the object.  It's a long topic; maybe it could even just be a link, 
>>if someone knows of a good discussion along those lines.  I'm sure 
>>there's some terminology here that I'm forgetting that describes the 
>>design pattern.  There's also a point when the style guide becomes an 
>>API design guide, and I don't know how far it should go in that direction.
> 
> 
> I'm not exactly sure if this is what you're getting at, but one thing
> that bothers me is using data attributes to trigger actions.  Maybe this
> gets into the "no side-effects" rule for data attributes, but attributes
> that cause an object to perform some action should always be explicit
> methods.

Exactly.  That's why I suggested the PEP start with the trivial case,
which, BTW is extremely common.  Let judgement guide when something
is no-longer trivial.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From jim at zope.com  Tue Dec 13 13:26:25 2005
From: jim at zope.com (Jim Fulton)
Date: Tue, 13 Dec 2005 07:26:25 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134404788.950.24.camel@geddy.wooz.org>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<1134404788.950.24.camel@geddy.wooz.org>
Message-ID: <439EBDF1.80209@zope.com>

Barry Warsaw wrote:
> On Sun, 2005-12-11 at 11:20 -0500, Jim Fulton wrote:
> 
> 
>>This seems outdated.  My impression, in part from time spent
>>working with the Python Labs guys, is that it is fine to have public
>>data sttributes even for non-"record" types.  In fact, I would argue that
>>any time you would be tempted to provide "getFoo()" and "setFoo(v)"
>>for some "private attribute _foo", it would be better to make it
>>public.  I certainly find "blah.foo" and "blah.foo = v" to be much
>>better than "blah.getFoo()" and blah.setFoo(v)".
>>
>>Certainly, properties provide a safety belt.  I would argue it this
>>way: Python APIs can include attributes as well as methods.
>>Exposure of an attribute need not constrain the implementation, thanks
>>to properties.  OTOH, I wouldn't bother with a property unless it's needed.
> 
> 
> Let me know what you think about this language (from my in-progress
> update of PEP 8):
> 
>     Designing for inheritance
> 
>       Always decide whether a class's methods and instance variables
>       (collectively: "attributes") should be public or non-public.  Public
>       attributes are those that you expect unrelated clients of your class to
>       use, with your commitment to avoid backward incompatible changes.
>       Non-public attributes are those that are not intended to be used by
>       third parties; you make no guarantees that non-pubic attributes won't
>       change or even be removed.

I'd add somewhere: "If in doubt, chose non-public. You can always change your
mind later."

>       We don't use the term "private" here, since no attribute is really
>       private in Python (without a generally unnecessary amount of work).
>       However, another category of attribute are those which, while not being
>       public, are intended for use by subclasses (often called "protected" in
>       other languages).  Some classes are designed to be inherited from,
>       either to extend or modify aspects of the class's behavior.  When
>       designing such a class, take care to make explicit decisions about which
>       attributes are public, which are non-public but useful for subclasses, and
>       which are truly only to be used by your base class.

A useful term might be "subclass API".  Decide which non-public attributes
are part of the subclass API.

>       With this in mind, here are the Pythonic guidelines:
> 
>       - Public attributes should have no leading underscores.
> 
>       - If your public attribute name collides with a reserved keyword, append
>         a single trailing underscore to your attribute name.  This is
>         preferable to an abbreviation or corrupted spelling.  E.g. "class_"
>         is preferable to "cls" or "klass".
> 
>         Note 1: See the argument name recommendation above for class methods.
> 
> [BAW: I'll include this new text in a later followup]
> 
>       - For simple public data attributes, it is fine to expose just the
>         attribute name, without complicated accessor/mutator methods.  Keep in
>         mind that Python provides an easy path to future enhancement, should
>         you find that a simple data attribute needs to grow functional
>         behavior.  In that case, use properties to hide functional
>         implementation behind simple data attribute access syntax.
> 
>         Note 1: Properties only work on new-style classes.
> 
>         Note 2: Try to keep the functional behavior side-effect free, although
>         side-effects such as caching are generally fine.

Personally, I'd actively discourage use of trivial accessors.  Simple
attribute access is not only "fine", IMO, but it is much better than
trivial accessors.  This is an important point, IMO, because, in my
experience, the vast majority of accessors *are* trivial.

>       - If your class is intended to be subclassed, and you have attributes
>         that you do not want subclasses to use, consider naming them with
>         double leading underscores and no trailing underscores.  This invokes
>         Python's name mangling algorithm, where the name of the class is
>         mangled into the attribute name.  This helps avoid attribute name
>         collisions should subclasses inadvertently contain attributes with the
>         same name.
> 
>         Note 1: Note that only the simple class name is used in the mangled
>         name, so if a subclass chooses both the same class name and attribute
>         name, you can still get name collisions.
> 
>         Note 2: Name mangling can make certain uses, such as debugging, less
>         convenient.  However the name mangling algorithm is well documented
>         and easy to perform manually.

Of course, I disagree with this last one, but I've been overruled.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From mal at egenix.com  Tue Dec 13 13:17:46 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Dec 2005 13:17:46 +0100
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <dnmc1q$km$1@sea.gmane.org>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>	<439E1D36.1090009@v.loewis.de>	<17310.21156.907292.278567@montanaro.dyndns.org>	<ca471dc20512122052k4b985d98m9ce1662858f2d1bc@mail.gmail.com><439E9AD1.9080103@livinglogic.de>	<439EABFB.3020405@gmail.com>
	<dnmc1q$km$1@sea.gmane.org>
Message-ID: <439EBBEA.9030705@egenix.com>

Nice that we now have ElementTree in the stdlib :-)

Some questions:

* Are you going to contribute cElementTree as well ?

* What was the motivation to not include the whole ElementTree
  package ?

* I'm missing the usual "Licensed to PSF under a Contributor Agreement."
  in the copyright notices of the files:

  http://www.python.org/psf/contrib.html

  I assume that you'll add these, right ?

* How should users that want to use the latest and greatest
  (more recent) distribution directly from your site go about in
  their apps ? Using from...as contructs ?

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 13 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From hoffman at ebi.ac.uk  Tue Dec 13 13:59:04 2005
From: hoffman at ebi.ac.uk (Michael Hoffman)
Date: Tue, 13 Dec 2005 12:59:04 +0000
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439EB711.5030005@zope.com>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com> <439EB711.5030005@zope.com>
Message-ID: <Pine.LNX.4.62.0512131255090.26997@qnzvnan.rov.np.hx>

[Ian Bickling]
>> stdlib, external modules, internal modules seems like enough
>> ordering to me.

[Jim Fulton]
> Personally, I don't find the stdlib/external distinction to be useful.

It's useful because it allows one to quickly see all the prerequisites
need to be installed in one place.
-- 
Michael Hoffman <hoffman at ebi.ac.uk>
European Bioinformatics Institute


From mwh at python.net  Tue Dec 13 14:11:59 2005
From: mwh at python.net (Michael Hudson)
Date: Tue, 13 Dec 2005 13:11:59 +0000
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <ca471dc20512122043u4db11498h6b407a8500808051@mail.gmail.com>
	(Guido van Rossum's message of "Mon, 12 Dec 2005 20:43:14 -0800")
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<17310.19593.900320.428826@montanaro.dyndns.org>
	<ca471dc20512122043u4db11498h6b407a8500808051@mail.gmail.com>
Message-ID: <2m64pt6t0g.fsf@starship.python.net>

Guido van Rossum <guido at python.org> writes:

> Dotted non-from imports (e.g. import test.pystone) are rare enough
> that they don't deserve a special rule; if you want me to give a rule,
> I think they should be mixed in with the undotted ones,
> alphabetically.

I actually really hate this style, though I'm at a bit of a loss as to
explain why...

Cheers,
mwh

-- 
  An encyclopedia is about being as accurate as it can, not being
  evenly misinformed.                     -- Coby Beck, comp.lang.lisp

From rhamph at gmail.com  Tue Dec 13 14:14:19 2005
From: rhamph at gmail.com (Adam Olsen)
Date: Tue, 13 Dec 2005 06:14:19 -0700
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <ca471dc20512121648l10f33f16yb1846bf882375796@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<439CE06D.70600@gmail.com>
	<ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com>
	<d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com>
	<aac2c7cb0512120449g1954e99fk65d24f3bc51cd428@mail.gmail.com>
	<ca471dc20512121115l2b98c343q675604bf127e3a@mail.gmail.com>
	<aac2c7cb0512121624j3c455645l9dc9c969e72b935e@mail.gmail.com>
	<ca471dc20512121648l10f33f16yb1846bf882375796@mail.gmail.com>
Message-ID: <aac2c7cb0512130514p1ce486f4g77bec28b3a963478@mail.gmail.com>

On 12/12/05, Guido van Rossum <guido at python.org> wrote:
> Unfortunately that fails one of the other requirements, which (at the
> time of implementation) was minimal impact on the rest of the
> interpreter. Since __private isn't limited to self, and attribute
> lookup doesn't always result in a dict lookup, this would require a
> complete overhaul of the getattr API, both at the C and at the Python
> level.

I hate to flog a dead horse but I feel it's important to clarify my
intentions here.  I don't see why it couldn't require a dict. 
Immutable builtins will fail either way, and so will classes using
slots (unless they hardcode the required private name).  The only
problematic use-case I can think of is a proxy class, but is that
enough to dictate the entire design of the feature?

> But I guess you already said that when you said """Obviously it
> doesn't handle backwards compatibility, so it's more of a "if I could
> do it again.." suggestion."""

I was referring to code which already hardcodes the format of the
current approach, i.e.:
class Foo(object):
    def __init__(self):
        self.__private = 42
f = Foo()
print f._Foo__private

--
Adam Olsen, aka Rhamphoryncus

From fredrik at pythonware.com  Tue Dec 13 14:28:51 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 13 Dec 2005 14:28:51 +0100
Subject: [Python-Dev] ElementTree in stdlib
References: <200512130001.jBD01bXZ007657@chilled.skew.org>	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>	<439E1D36.1090009@v.loewis.de>	<17310.21156.907292.278567@montanaro.dyndns.org>	<ca471dc20512122052k4b985d98m9ce1662858f2d1bc@mail.gmail.com><439E9AD1.9080103@livinglogic.de>	<439EABFB.3020405@gmail.com><dnmc1q$km$1@sea.gmane.org>
	<439EBBEA.9030705@egenix.com>
Message-ID: <dnmiaj$l8l$1@sea.gmane.org>

M.-A. Lemburg wrote:

> Some questions:
>
> * Are you going to contribute cElementTree as well ?

yes, but there are some build issues we need to sort out first (both pyexpat
and cET link to their own copies of expat)

we also need to figure out how to import the bundled version; should it be
cElementTree, xml.etree.cElementTree, or just xml.etree.ElementTree
(which would then fallback on the Python version if cElementTree isn't
built) ?

> * What was the motivation to not include the whole ElementTree
>  package ?

this is a perfect time to get rid of some little-used stuff.  if there's enough user
demand, we can always add a few more modules before 2.5 goes out of the
door...

> * I'm missing the usual "Licensed to PSF under a Contributor Agreement."
>  in the copyright notices of the files:
>
>  http://www.python.org/psf/contrib.html
>
>  I assume that you'll add these, right ?

will fix.

> * How should users that want to use the latest and greatest
>  (more recent) distribution directly from your site go about in
>  their apps ? Using from...as contructs ?

from-import or import-as works fine

</F> 




From fredrik at pythonware.com  Tue Dec 13 15:22:14 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 13 Dec 2005 15:22:14 +0100
Subject: [Python-Dev] Jython and CPython
Message-ID: <dnmlem$1a9$1@sea.gmane.org>

BTW, what's the policy wrt. Jython-specific modules in the standard library?

Expat isn't available under Jython, but I have a Java/Jython-driver for ElementTree
on my disk.  Can / should this go into the CPython standard library ?

</F> 




From jim at zope.com  Tue Dec 13 15:48:33 2005
From: jim at zope.com (Jim Fulton)
Date: Tue, 13 Dec 2005 09:48:33 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <Pine.LNX.4.62.0512131255090.26997@qnzvnan.rov.np.hx>
References: <4399F967.3080300@colorstudy.com>	<1134168683.19370.29.camel@geddy.wooz.org>	<439A110E.8090605@colorstudy.com>	<1134411382.5676.35.camel@geddy.wooz.org>	<439DDCD2.10803@colorstudy.com>
	<439EB711.5030005@zope.com>
	<Pine.LNX.4.62.0512131255090.26997@qnzvnan.rov.np.hx>
Message-ID: <439EDF41.6080101@zope.com>

Michael Hoffman wrote:
> [Ian Bickling]
> 
>>>stdlib, external modules, internal modules seems like enough
>>>ordering to me.
> 
> 
> [Jim Fulton]
> 
>>Personally, I don't find the stdlib/external distinction to be useful.
> 
> 
> It's useful because it allows one to quickly see all the prerequisites
> need to be installed in one place.

Sure, if you only have one module, and if your module doesn't do any
dynamic imports, and if the things your importing don't have dependencies,
and ...

I think it would be simpler to have a formal dependency system.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From pinard at iro.umontreal.ca  Tue Dec 13 15:53:13 2005
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Tue, 13 Dec 2005 09:53:13 -0500
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <d11dcfba0512121333o4e17b735mcb5f91df4387d380@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<439CE06D.70600@gmail.com>
	<ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com>
	<d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com>
	<ca471dc20512121111i2596a04dlbe2e151780829ec3@mail.gmail.com>
	<d11dcfba0512121333o4e17b735mcb5f91df4387d380@mail.gmail.com>
Message-ID: <20051213145313.GA23676@alcyon.progiciels-bpi.ca>

[Steven Bethard]

>Ahh.  I never run into this because I never import objects directly
>from modules.  So, instead of:

>    from elementtree.ElementTree import ElementTree
>    ...
>    ElementTree(...)

>I almost always write something like:

>    import elementtree.ElementTree as et
>    ...
>    et.ElementTree(...)

This is a bit off-topic, but I felt like sharing our experience.  One 
consultant we once hired here was doing exactly that (importing over 
two-letter abbreviations).

>Thus, all objects that were imported from external modules are always
>immediately identifiable as such by their prefixed module name.  I do
>see though that if you like to import the objects directly from the
>module this could be confusing.

Everybody here agrees that this style makes the code much less legible.
Partly because of the constant indirection.  Also because it imposes
learning all those two-letter abbreviations before reading a module, and 
the learning has to be redone on each visit, it just does not stick.

So, we try to routinely replace abbreviations with the real names
whenever we have to play in one module written by this consultant.  But 
it only goes a little bit at a time.  We should probably suffer taking 
the time, dive in it all, and get rid of this style once and for all...

-- 
Fran?ois Pinard   http://pinard.progiciels-bpi.ca

From mcherm at mcherm.com  Tue Dec 13 15:31:18 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue, 13 Dec 2005 06:31:18 -0800
Subject: [Python-Dev] ElementTree in stdlib
Message-ID: <20051213063118.4eivt285x1f4sks0@login.werra.lunarpages.com>

Mike Brown writes:
   [ElementTree was accepted into stdlib immediately without discussion
    on XML-Sig. Seems like a lack of due process.]
> Some authors of other libs may not even be aware that they could so
> easily have their code whisked into stdlib, if it's solid enough.

It's not the solidity of the CODE in ElementTree that secured the
approval. It's not even the pythonicness of the API (although that's
ElementTree's greatest strength). No, the reason for the rapid
acceptance was the solidity of the *community support*. For a long
time, lots of people (users, not just core developers) have been
thinking to themselves "why isn't ElementTree the standard Python
API for XML?". Once it was stated out loud (on c.l.py) and it was
clear that /F supported the idea, there was little to discuss.

Frankly, if at any time in the past several years the XML-SIG had
published their consensus report on the "preferred API for XML"
(or perhaps "preferred small set of APIs, each tuned for a specific
purpose"), I expect it would have been incorporated in the core.
This could have been done long before /F ever wrote ElementTree.
But historically, this isn't what happened.

I look at some other areas and find that Python tends to have one
good (hopefully excellent) implementation of a given feature, and
perhaps a few high-powered 3rd party implementations for special
purposes. For instance, there's the datetime module which satisfies
most users, then there are tools like mxDateTime for specialists.
Most users of high-precision numbers make due with the built-in
long type, but specialists use GMPY. Most users of threading find
that the threading module is sufficient, but those who really
want full co-routines get stackless.

Expressed in this fashion, I have always felt that the XML-SIG was
basically working on developing and standardizing the specialist
tools for XML, with special attention paid to things like very
high performance, very complete implementation of XML features,
cross-language standardization, automatic object serialization,
and other such features far removed from the basic "I want to read
this file and it's in XML." Those are great areas, and there are
people who need them (for some projects, I'm one of those people).
However, ElementTree is one of the few libraries that have struck
me as being canidates for the "one good implementation" that
serves the basic needs of the typical user.

-- Michael Chermside


From skip at pobox.com  Tue Dec 13 16:18:18 2005
From: skip at pobox.com (skip@pobox.com)
Date: Tue, 13 Dec 2005 09:18:18 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439E94B7.7010605@gmail.com>
References: <439CA89B.4030600@colorstudy.com>
	<17308.55295.648767.433858@montanaro.dyndns.org>
	<20051211195752.CB4A.JCARLSON@uci.edu> <439CF760.8040408@gmail.com>
	<17309.38679.642021.215357@montanaro.dyndns.org>
	<439E94B7.7010605@gmail.com>
Message-ID: <17310.58938.347464.97687@montanaro.dyndns.org>


    Nick> Having to type '(object)' all the time is annoying, but less
    Nick> annoying than trying to figure out which set of semantics a given
    Nick> class is using.

Sure.  Since I started writing Python long before new-style classes were
around, I have lots of classic classes.  My default is thus to use classic
classes in preference to new-style classes, for much the same semantic
reasons as you.

    Nick> I think it's an artifact of only seriously starting to use Python
    Nick> with version 2.2.2 - I don't really understand how old-style
    Nick> classes work, so I try to avoid using them.

Again, we're actually thinking along the same lines.  Classic classes work
just fine for me, so I've been slow to let the new-style class meme permeate
through my brain.

Skip

From stephen at xemacs.org  Tue Dec 13 15:52:18 2005
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 13 Dec 2005 23:52:18 +0900
Subject: [Python-Dev] Incorporating external packages into Python's std
 distribution
In-Reply-To: <17309.39329.395921.139413@montanaro.dyndns.org> (skip@pobox.com's
	message of "Mon, 12 Dec 2005 09:39:13 -0600")
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
	<e8bf7a530512112117r6ecd15b1o7efe387633917cff@mail.gmail.com>
	<17309.39329.395921.139413@montanaro.dyndns.org>
Message-ID: <87u0ddt5gd.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "skip" == skip  <skip at pobox.com> writes:

    skip> BTW, there is one project I'm theoretically familiar with
    skip> that attempts to handle the dual source situation: XEmacs.
    skip> I'm still trying to come to terms with the practical issues
    skip> involved.  I'm supposed to be updating the python-mode code,
    skip> and am only taking baby steps in that direction, so I'm
    skip> probably not the best person to describe how it works, but
    skip> here goes.

I'd be happy to make some time to describe the XEmacs scheme and
experience if somebody wants.  However, XEmacs faces language and code
organization constraints that Python does not, and Fredrik's
suggestion looks like a substantial improvement over the system XEmacs
has in place.

Even with its defects, it's been a great success for us.


-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From skip at pobox.com  Tue Dec 13 16:28:59 2005
From: skip at pobox.com (skip@pobox.com)
Date: Tue, 13 Dec 2005 09:28:59 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439EB711.5030005@zope.com>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com> <439EB711.5030005@zope.com>
Message-ID: <17310.59579.633849.324717@montanaro.dyndns.org>


    Jim> Personally, I don't find the stdlib/external distinction to be
    Jim> useful.

For me it's just a "who do I blame for problems" sort of thing.  Most of the
time I know, but others looking at my code might not know that MySQLdb isn't
in the core but that bsddb is.

Skip

From hoffman at ebi.ac.uk  Tue Dec 13 16:25:24 2005
From: hoffman at ebi.ac.uk (Michael Hoffman)
Date: Tue, 13 Dec 2005 15:25:24 +0000
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439EDF41.6080101@zope.com>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com> <439EB711.5030005@zope.com>
	<Pine.LNX.4.62.0512131255090.26997@qnzvnan.rov.np.hx>
	<439EDF41.6080101@zope.com>
Message-ID: <Pine.LNX.4.62.0512131521440.26997@qnzvnan.rov.np.hx>

[Jim Fulton]
>>> Personally, I don't find the stdlib/external distinction to be useful.

[Michael Hoffman]
>> It's useful because it allows one to quickly see all the prerequisites
>> need to be installed in one place.

[Jim Fulton]
> Sure, if you only have one module, and if your module doesn't do any
> dynamic imports, and if the things your importing don't have dependencies,
> and ...
>
> I think it would be simpler to have a formal dependency system.

More useful, yes, for all the reasons you listed. The fact that people
are still working on a formal dependency system, however, indicates
that it is not simpler.
-- 
Michael Hoffman <hoffman at ebi.ac.uk>
European Bioinformatics Institute


From skip at pobox.com  Tue Dec 13 16:39:32 2005
From: skip at pobox.com (skip@pobox.com)
Date: Tue, 13 Dec 2005 09:39:32 -0600
Subject: [Python-Dev] On moving to new-style classes
In-Reply-To: <439EBB16.3000704@zope.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com>
	<1134395316.11373.12.camel@localhost> <439D8911.1010505@zope.com>
	<17309.40570.699501.68209@montanaro.dyndns.org>
	<439DC6EC.60205@zope.com>
	<17310.20546.584413.501771@montanaro.dyndns.org>
	<439EBB16.3000704@zope.com>
Message-ID: <17310.60212.148214.218004@montanaro.dyndns.org>


    Jim> For debugging purposes, it's easy to add a property to allow
    Jim> debugging of attribute assignment.

Assuming you use new-style classes, which I often don't.  The property/debug
idea that you and Gustavo have both now mentioned makes them a bit more
attractive.

Is there a new-style class HOW-TO somewhere?  It would be useful to
summarize the advantages for them.  I still have this thought stuck in my
head (from where, I don't know, probably incorrect) that one of the main
reasons for new-style classes was to get rid of __dict__.

    Jim> Having to write accessors for all your public methods doesn't seem
    Jim> consistent with rapid development.  

I'd rather trade the 30 seconds it takes to write a simple accessor method
when I need it than the minute or two it takes to figure out where my
attribute got stomped.

I guess it mostly boils down to a matter of taste.  Did I also mention that
most of the programmers here are C++ folk?  They have their beloved inline
keyword.  <wink>

Skip

From jim at zope.com  Tue Dec 13 16:45:13 2005
From: jim at zope.com (Jim Fulton)
Date: Tue, 13 Dec 2005 10:45:13 -0500
Subject: [Python-Dev] On moving to new-style classes
In-Reply-To: <17310.60212.148214.218004@montanaro.dyndns.org>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com>
	<1134395316.11373.12.camel@localhost> <439D8911.1010505@zope.com>
	<17309.40570.699501.68209@montanaro.dyndns.org>
	<439DC6EC.60205@zope.com>
	<17310.20546.584413.501771@montanaro.dyndns.org>
	<439EBB16.3000704@zope.com>
	<17310.60212.148214.218004@montanaro.dyndns.org>
Message-ID: <439EEC89.6020704@zope.com>

skip at pobox.com wrote:
>     Jim> For debugging purposes, it's easy to add a property to allow
>     Jim> debugging of attribute assignment.
> 
> Assuming you use new-style classes, which I often don't.  The property/debug
> idea that you and Gustavo have both now mentioned makes them a bit more
> attractive.
 >
> Is there a new-style class HOW-TO somewhere?

See http://www.python.org/doc/newstyle.html

 > It would be useful to
> summarize the advantages for them.  I still have this thought stuck in my
> head (from where, I don't know, probably incorrect) that one of the main
> reasons for new-style classes was to get rid of __dict__.

No, the main benefit is to begin to resolve the class/type dichotomy.
Among other benefits, this allows you to subclass types written in C.

Of course, there are other benefits, most notably descriptors, which make
properties, among other things, possible.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From steven.bethard at gmail.com  Tue Dec 13 17:13:40 2005
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 13 Dec 2005 09:13:40 -0700
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <20051213145313.GA23676@alcyon.progiciels-bpi.ca>
References: <4399F967.3080300@colorstudy.com> <439CA89B.4030600@colorstudy.com>
	<439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<439CE06D.70600@gmail.com>
	<ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com>
	<d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com>
	<ca471dc20512121111i2596a04dlbe2e151780829ec3@mail.gmail.com>
	<d11dcfba0512121333o4e17b735mcb5f91df4387d380@mail.gmail.com>
	<20051213145313.GA23676@alcyon.progiciels-bpi.ca>
Message-ID: <d11dcfba0512130813y24f4908bu139821e8fc3cdb9b@mail.gmail.com>

On 12/13/05, Fran?ois Pinard <pinard at iro.umontreal.ca> wrote:
> [Steven Bethard]
>
> >Ahh.  I never run into this because I never import objects directly
> >from modules.  So, instead of:
>
> >    from elementtree.ElementTree import ElementTree
> >    ...
> >    ElementTree(...)
>
> >I almost always write something like:
>
> >    import elementtree.ElementTree as et
> >    ...
> >    et.ElementTree(...)
>
> This is a bit off-topic, but I felt like sharing our experience.  One
> consultant we once hired here was doing exactly that (importing over
> two-letter abbreviations).
>
> >Thus, all objects that were imported from external modules are always
> >immediately identifiable as such by their prefixed module name.  I do
> >see though that if you like to import the objects directly from the
> >module this could be confusing.
>
> Everybody here agrees that this style makes the code much less legible.
> Partly because of the constant indirection.  Also because it imposes
> learning all those two-letter abbreviations before reading a module, and
> the learning has to be redone on each visit, it just does not stick.

Much less legible than without the namespace?  Or much less legible
than with a non-abbreviated namespace.  FWIW, here's some real
examples from my code:

import ellogon.utils as utils
import ellogon.features.relations as features_relations
import ellogon.chunking as chunking
import ml.classifiers as _ml_classifiers
import ml.data as _ml_data

The only two-letter one was ElementTree, and the vast majority were
unabbreviated, though  as you can see,  some of them drop one of the
items in the import chain.  Do you find imports like the above
problematic?

FWIW, I don't like importing objects from modules directly for the
same reason that when I write Java now, I always use an explicit
"this" for instance variables.  When I see a name which isn't local to
a function, I want to have some idea where it came from...

STeVe
--
You can wordify anything if you just verb it.
        --- Bucky Katt, Get Fuzzy

From ianb at colorstudy.com  Tue Dec 13 17:17:31 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 13 Dec 2005 10:17:31 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439EB711.5030005@zope.com>
References: <4399F967.3080300@colorstudy.com>	<1134168683.19370.29.camel@geddy.wooz.org>	<439A110E.8090605@colorstudy.com>	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com> <439EB711.5030005@zope.com>
Message-ID: <439EF41B.10605@colorstudy.com>

Jim Fulton wrote:
>> stdlib, external modules, internal modules seems like enough ordering 
>> to me.  If you want to order things more exactly, sure, but I don't 
>> really see the point personally.  Since I can't assume as a reader 
>> that imports are ordered in any way I have to search to be sure of 
>> what's there.  The grouping help me browse, but I'd hope that the 
>> import list is short enough that I don't need to use alphabetization 
>> to scan for a module.
> 
> 
> Personally, I don't find the stdlib/external distinction to be useful.

I like the stdlib coming first, because (if it's not using "from") 
stdlib imports are just line noise and I don't pay any attention to 
them.  I care if a module uses an external package, but I don't care 
what stdlib packages it uses (I'd actually like to be able to avoid 
importing them at all, but that's an aside: 
http://blog.ianbicking.org/py-std.html).  So I don't really care what 
comes first, just that stdlib imports are grouped together so I can 
ignore them, and first is as good a place as anything.  "from...import" 
is a different matter of course, because I need to look at those to see 
where names come from.

Additionally, having the local packages grouped together helps make up 
for the non-distinction of absolute and relative imports.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From fredrik at pythonware.com  Tue Dec 13 17:22:18 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 13 Dec 2005 17:22:18 +0100
Subject: [Python-Dev] Sharing expat instances
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de><dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de><dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de>
Message-ID: <dnmsft$t1g$1@sea.gmane.org>

Martin v. Löwis wrote:

> >   - sort out expat bundling issues, and include cElementTree as well
> >     (using the same approach as above).

...

> > (one way to do this would be to add an "function pointer table" to pyexpat
> > that contains pointers to selected portions of the expat API, and then add
> > an indirection level to cElementTree)
>
> Ok, this sounds like a larger piece of work.

here's a plan:

1. add an Include/pyexpat.h header file which contains a structure
similar to the following:

#define PyExpat_DISPATCH_MAGIC  "... some magic string ..."

struct PyExpat_Dispatch {
    int size; /* size of this structure */
    int MAJOR_VERSION;
    int MINOR_VERSION;
    int MICRO_VERSION;
    ... (*ErrorString)(...)
    ... (*GetErrorColumnNumber)(...)
    ... (*GetErrorLineNumber)(...)
    ... (*Parse)(...)
    ... (*ParserCreate_MM)(...)
    ... (*ParserFree)(...)
    ... (*SetCharacterDataHandler)(...)
    ... (*SetCommentHandler)(...)
    ... (*SetDefaultHandlerExpand)(...)
    ... (*SetElementHandler)(...)
    ... (*SetNamespaceDeclHandler)(...)
    ... (*SetProcessingInstructionHandler)(...)
    ... (*SetUserData)(...)
    /* add new stuff to the end */
}

(this is the minimal stuff used by today's cElementTree; it can of course
be extended to cover a larger part of the current expat API)

2. during pyexpat initialization, initialize all members of this structure, and
make it available as a PyCObject:

    static PyExpat_Dispatch dispatch;

    dispatch.size = sizeof(dispatch):
    dispatch.MAJOR_VERSION = XML_MAJOR_VERSION;
    ...

    obj = PyCObject_FromVoidPtrAndDesc(
        &dispatch, PyExpat_DISPATCH_MAGIC, NULL
    );
    ... stuff object into module dictionary ...

3. in cElementTree (or _elementtree, or whatever the python version will
be named), import pyexpat, fetch the object, and verify

    - that the PyExpat_DISPATCH_MAGIC matches
    - that the size field is at least as large as sizeof(struct PyExpat_Dispatch)
    - that the version number matches (at least MAJOR and MINOR; I'm not
    sure under what circumstances they change the MICRO number)

4. in cElementTree (...), do all expat calls via the dispatch table.

comments ?

</F>




From pje at telecommunity.com  Tue Dec 13 17:43:10 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 13 Dec 2005 11:43:10 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439EBDF1.80209@zope.com>
References: <1134404788.950.24.camel@geddy.wooz.org>
	<4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<1134404788.950.24.camel@geddy.wooz.org>
Message-ID: <5.1.1.6.0.20051213114032.01f5c728@mail.telecommunity.com>

At 07:26 AM 12/13/2005 -0500, Jim Fulton wrote:
>Personally, I'd actively discourage use of trivial accessors.  Simple
>attribute access is not only "fine", IMO, but it is much better than
>trivial accessors.  This is an important point, IMO, because, in my
>experience, the vast majority of accessors *are* trivial.

+1000.  Python is not Java.  It's hard enough to get former Java users to 
stop writing getters and setters in the first place, without PEP 8 
providing even a sliver of support for that nonsense.  If anything, PEP 8 
should warn in the strongest possible terms about the wastefulness of this 
practice and its detrimental effects on code size, readability, and 
performance.


From fredrik at pythonware.com  Tue Dec 13 17:26:33 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 13 Dec 2005 17:26:33 +0100
Subject: [Python-Dev] Deprecate __ private (was Re: PEP
	8updates/clarifications)
References: <4399F967.3080300@colorstudy.com>
	<439CA89B.4030600@colorstudy.com><439CAEF6.4000304@zope.com><ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com><439CE06D.70600@gmail.com><ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com><d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com><ca471dc20512121111i2596a04dlbe2e151780829ec3@mail.gmail.com><d11dcfba0512121333o4e17b735mcb5f91df4387d380@mail.gmail.com><20051213145313.GA23676@alcyon.progiciels-bpi.ca>
	<d11dcfba0512130813y24f4908bu139821e8fc3cdb9b@mail.gmail.com>
Message-ID: <dnmsnq$u1m$1@sea.gmane.org>

Steven Bethard wrote:

> > Everybody here agrees that this style makes the code much less legible.
> > Partly because of the constant indirection.  Also because it imposes
> > learning all those two-letter abbreviations before reading a module, and
> > the learning has to be redone on each visit, it just does not stick.
>
> Much less legible than without the namespace?  Or much less legible
> than with a non-abbreviated namespace.

using abbreviations just for the sake of it may be a bad idea, but using
it to able to quickly switch between different drivers works really well.

my code is full of stuff like:

    import sqlite2 as DB

    import wckTkinter as WCK

    # import cElementtree as ET
    import xml.etree.ElementTree as ET

but you sure won't see

    import sys as SY
    import os.path as op

or other gratuitous aliasing.

</F>




From pje at telecommunity.com  Tue Dec 13 17:51:28 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 13 Dec 2005 11:51:28 -0500
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
 updates/clarifications)
In-Reply-To: <20051213145313.GA23676@alcyon.progiciels-bpi.ca>
References: <d11dcfba0512121333o4e17b735mcb5f91df4387d380@mail.gmail.com>
	<4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<439CE06D.70600@gmail.com>
	<ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com>
	<d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com>
	<ca471dc20512121111i2596a04dlbe2e151780829ec3@mail.gmail.com>
	<d11dcfba0512121333o4e17b735mcb5f91df4387d380@mail.gmail.com>
Message-ID: <5.1.1.6.0.20051213114517.01f73c48@mail.telecommunity.com>

At 09:53 AM 12/13/2005 -0500, Fran?ois Pinard wrote:
>Everybody here agrees that this style makes the code much less legible.

I hope you mean, "here at your company or organization", as I disagree.  :)

The current draft API guidelines for Chandler encourage the use of short 
API module names such as 'pim' and 'mail' to allow more clarity as to 
naming, while encouraging import patterns that make module reloading more 
practical during development.  Using a module name as a prefix to a class 
or function name also allows flatter namespaces with fewer imports, 
especially in the case of APIs which contain many features and are 
frequently used.

Of course, we don't do abbreviations or renaming; instead, code like this 
is the common pattern:

     from application import schema

     class Contact(schema.Item):
         name = schema.One(schema.Text)
         # ... etc.


From pje at telecommunity.com  Tue Dec 13 18:00:27 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 13 Dec 2005 12:00:27 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <Pine.LNX.4.62.0512131521440.26997@qnzvnan.rov.np.hx>
References: <439EDF41.6080101@zope.com> <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com> <439EB711.5030005@zope.com>
	<Pine.LNX.4.62.0512131255090.26997@qnzvnan.rov.np.hx>
	<439EDF41.6080101@zope.com>
Message-ID: <5.1.1.6.0.20051213115321.01f79e58@mail.telecommunity.com>

At 03:25 PM 12/13/2005 +0000, Michael Hoffman wrote:
>[Jim Fulton]
> > Sure, if you only have one module, and if your module doesn't do any
> > dynamic imports, and if the things your importing don't have dependencies,
> > and ...
> >
> > I think it would be simpler to have a formal dependency system.
>
>More useful, yes, for all the reasons you listed. The fact that people
>are still working on a formal dependency system, however, indicates
>that it is not simpler.

Depends on your definition of "still working on".  I'd characterize the 
dependency system offered by setuptools as receiving fine-tuning, rather 
than being under design or development.  A few things have been tweaked in 
the last few months according to real-world feedback: breadth-first 
processing worked out to be better than depth-first when complex recursive 
dependencies are involved, and the handling of '-' in version numbers 
needed a minor adjustment.

In any case, the algorithms involved are near-trivial; the most complex 
piece is the processing of complex version specifications like 
"CherryPy>=2.1.0,!=2.1.1-rc2,<2.2a" into a series of version intervals.

The only outstanding feature request for the dependency resolution 
algorithm is supporting optional or replaceable dependencies such as "we 
need either scipy *or* Numeric".


From fdrake at acm.org  Tue Dec 13 18:00:28 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 13 Dec 2005 12:00:28 -0500
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnmsft$t1g$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<439D1B6D.9080208@v.loewis.de> <dnmsft$t1g$1@sea.gmane.org>
Message-ID: <200512131200.28540.fdrake@acm.org>

On Tuesday 13 December 2005 11:22, Fredrik Lundh wrote:
 > here's a plan:
 >
 > 1. add an Include/pyexpat.h header file which contains a structure
 > similar to the following:
...
 > comments ?

+1


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From barry at python.org  Tue Dec 13 19:28:06 2005
From: barry at python.org (Barry Warsaw)
Date: Tue, 13 Dec 2005 13:28:06 -0500
Subject: [Python-Dev] Incorporating external packages into Python's
	std	distribution
In-Reply-To: <87u0ddt5gd.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
	<e8bf7a530512112117r6ecd15b1o7efe387633917cff@mail.gmail.com>
	<17309.39329.395921.139413@montanaro.dyndns.org>
	<87u0ddt5gd.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <1134498486.12098.28.camel@geddy.wooz.org>

On Tue, 2005-12-13 at 23:52 +0900, Stephen J. Turnbull wrote:

> I'd be happy to make some time to describe the XEmacs scheme and
> experience if somebody wants.  However, XEmacs faces language and code
> organization constraints that Python does not, and Fredrik's
> suggestion looks like a substantial improvement over the system XEmacs
> has in place.
> 
> Even with its defects, it's been a great success for us.

I'd love to read about the way XEmacs is doing this.
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051213/0853d146/attachment.pgp

From guido at python.org  Tue Dec 13 20:00:13 2005
From: guido at python.org (Guido van Rossum)
Date: Tue, 13 Dec 2005 11:00:13 -0800
Subject: [Python-Dev] Jython and CPython
In-Reply-To: <dnmlem$1a9$1@sea.gmane.org>
References: <dnmlem$1a9$1@sea.gmane.org>
Message-ID: <ca471dc20512131100p409a0c7bx94e0ad444ebe6eb6@mail.gmail.com>

I'm no expert on policy any more, but if you can reasonably prevent it
from doing any harm (such as failing unit tests) to CPython I don't
see why not. However I believe that traditionally, Jython-specific
code has been checked into Jython's own source control.

--Guido

On 12/13/05, Fredrik Lundh <fredrik at pythonware.com> wrote:
> BTW, what's the policy wrt. Jython-specific modules in the standard library?
>
> Expat isn't available under Jython, but I have a Java/Jython-driver for ElementTree
> on my disk.  Can / should this go into the CPython standard library ?

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Dec 13 20:09:34 2005
From: guido at python.org (Guido van Rossum)
Date: Tue, 13 Dec 2005 11:09:34 -0800
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <439E9AD1.9080103@livinglogic.de>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>
	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>
	<439E1D36.1090009@v.loewis.de>
	<17310.21156.907292.278567@montanaro.dyndns.org>
	<ca471dc20512122052k4b985d98m9ce1662858f2d1bc@mail.gmail.com>
	<439E9AD1.9080103@livinglogic.de>
Message-ID: <ca471dc20512131109y6b02d371u6657be586089987a@mail.gmail.com>

On 12/13/05, Walter D?rwald <walter at livinglogic.de> wrote:
> Guido van Rossum wrote:
> > I don't think that SAX is unpythonic, but it's pretty low-level and
> > mostly of use to people writing higher-level XML parsers (my parsexml
> > module uses it).
>
> Having to define classes that conform to a certain API and registering
> instances of those classes as callbacks with the parser doesn't look
> that pythonic to me. An iterator API seems much more pythonic.

Perhaps. Although the SAX API lets you leave a callback undefined if
you don't have a need to handle those events; that's a bit trickier to
do with an iterator. Also the different callbacks have different
signatures.

But since /F solved this for ElementTree I have to mostly agree with you. :-)

> Then again, pythonic is whatever you say that it is. ;)

Not at all. I will argue but I will also take arguments from others. Seriously.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Dec 13 20:18:05 2005
From: guido at python.org (Guido van Rossum)
Date: Tue, 13 Dec 2005 11:18:05 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439EB711.5030005@zope.com>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com> <439EB711.5030005@zope.com>
Message-ID: <ca471dc20512131118j47e97319l4fb605a78c3460b4@mail.gmail.com>

On 12/13/05, Jim Fulton <jim at zope.com> wrote:
> Personally, I'd rather just sort aphabetically based on dotted package
> name.  Because packages provide meaningful groupings to begin with,
> this approach provides the most meaningful groupings to me.  (All of
> my "internal" modules are in packages.)  When scanning imports, I
> don't want to have to think about whether a module is internal or
> external.  I've got enough to think about without that. :)

Disagree strongly.

The separation into (1) stdlib, (2) thirdparty, (3) internal helps the
reader assess dependencies -- where to look for more docs, what to do
if the import fails, etc.

> Frankly, I'd be as happy t see the PEP be silent on module ordering.

Obviously I disagree here too. :-)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fredrik at pythonware.com  Tue Dec 13 20:52:12 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 13 Dec 2005 20:52:12 +0100
Subject: [Python-Dev] Sharing expat instances
References: <17304.33755.693941.811233@montanaro.dyndns.org><439D1B6D.9080208@v.loewis.de>
	<dnmsft$t1g$1@sea.gmane.org> <200512131200.28540.fdrake@acm.org>
Message-ID: <dnn8pe$de1$1@sea.gmane.org>

Fred L. Drake, Jr. wrote:

>  > 1. add an Include/pyexpat.h header file which contains a structure
>  > similar to the following:
> ...
>  > comments ?
>
> +1

I take that as a "go ahead" ;-)

</F>




From martin at v.loewis.de  Tue Dec 13 21:02:14 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 13 Dec 2005 21:02:14 +0100
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <20051213063118.4eivt285x1f4sks0@login.werra.lunarpages.com>
References: <20051213063118.4eivt285x1f4sks0@login.werra.lunarpages.com>
Message-ID: <439F28C6.5050703@v.loewis.de>

Michael Chermside wrote:
> Frankly, if at any time in the past several years the XML-SIG had
> published their consensus report on the "preferred API for XML"
> (or perhaps "preferred small set of APIs, each tuned for a specific
> purpose"), I expect it would have been incorporated in the core.
> This could have been done long before /F ever wrote ElementTree.
> But historically, this isn't what happened.

That's not true. The current xml package *is* the consensus of
xml-sig.

Regards,
Martin

From martin at v.loewis.de  Tue Dec 13 21:08:31 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 13 Dec 2005 21:08:31 +0100
Subject: [Python-Dev] Jython and CPython
In-Reply-To: <dnmlem$1a9$1@sea.gmane.org>
References: <dnmlem$1a9$1@sea.gmane.org>
Message-ID: <439F2A3F.4000703@v.loewis.de>

Fredrik Lundh wrote:
> BTW, what's the policy wrt. Jython-specific modules in the standard library?

I don't think there is enough precedence to have a policy. So far, the
only places that explicitly support Jython is the test suite, pickle,
and platform (I wouldn't really count in site here).

If the portability problem can be solved by checking things into Jython
instead, I think I would prefer that. Then having in CPython an import
that only succeeds for Jython would be fine.

Regards,
Martin

From martin at v.loewis.de  Tue Dec 13 21:10:31 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 13 Dec 2005 21:10:31 +0100
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnmsft$t1g$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de><dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de><dni8qe$5eq$1@sea.gmane.org>	<439D1B6D.9080208@v.loewis.de>
	<dnmsft$t1g$1@sea.gmane.org>
Message-ID: <439F2AB7.3060708@v.loewis.de>

Fredrik Lundh wrote:
> comments ?

As Fred says: go ahead.

regards,
Martin

From fdrake at acm.org  Tue Dec 13 21:02:12 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 13 Dec 2005 15:02:12 -0500
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnn8pe$de1$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<200512131200.28540.fdrake@acm.org> <dnn8pe$de1$1@sea.gmane.org>
Message-ID: <200512131502.12526.fdrake@acm.org>

On Tuesday 13 December 2005 14:52, Fredrik Lundh wrote:
 > I take that as a "go ahead" ;-)

Good call!  :-)


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From fdrake at acm.org  Tue Dec 13 21:18:38 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 13 Dec 2005 15:18:38 -0500
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <439F28C6.5050703@v.loewis.de>
References: <20051213063118.4eivt285x1f4sks0@login.werra.lunarpages.com>
	<439F28C6.5050703@v.loewis.de>
Message-ID: <200512131518.38810.fdrake@acm.org>

On Tuesday 13 December 2005 15:02, Martin v. L?wis wrote:
 > That's not true. The current xml package *is* the consensus of
 > xml-sig.

It pretty much was at the time, at any rate.  It's not clear to me that the 
xml package shipped in 2.4 and several preceeding versions of Python would 
pass muster in the current XML-SIG.  There's been a lot of evolution in the 
Python APIs for XML since then, and a lot of really interesting things have 
been tried with varying degrees of acceptance.

Unless the XML-SIG wants to figure it out all over again, adding xml.etree to 
the standard library is probably the best near-term improvement that can be 
made.  Speaking just for myself, I think this is fine, though I agree with 
Jim that an easier-to-use package management system would go a long way to 
avoid the issues related to whether something is in the standard library.

Now, just what it means for a package management system to be easier to use 
might be harder to get us to agree on.  :-)


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From fredrik at pythonware.com  Tue Dec 13 21:29:18 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 13 Dec 2005 21:29:18 +0100
Subject: [Python-Dev] ElementTree in stdlib
References: <20051213063118.4eivt285x1f4sks0@login.werra.lunarpages.com><439F28C6.5050703@v.loewis.de>
	<200512131518.38810.fdrake@acm.org>
Message-ID: <dnnau7$ldg$1@sea.gmane.org>

Fred wrote:

> > That's not true. The current xml package *is* the consensus of
> > xml-sig.
>
> It pretty much was at the time, at any rate.  It's not clear to me that the
> xml package shipped in 2.4 and several preceeding versions of Python would
> pass muster in the current XML-SIG.  There's been a lot of evolution in the
> Python APIs for XML since then, and a lot of really interesting things have
> been tried with varying degrees of acceptance.

from what I can tell, most of the stuff under Lib/xml is between two and three
years old.   the last major PyXML sync appears to be against 1.82, in january
2003.  there are a few bug fixes since then, but that's about it.

what's the status of PyXML?  is it time to move it over to svn.python.org and
bring it up to 1.0 (whatever that would mean?)

</F> 




From mcherm at mcherm.com  Tue Dec 13 21:48:44 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue, 13 Dec 2005 12:48:44 -0800
Subject: [Python-Dev] ElementTree in stdlib
Message-ID: <20051213124844.tvwj6obz1u88wcgg@login.werra.lunarpages.com>

I wrote:
> Frankly, if at any time in the past several years the XML-SIG had
> published their consensus report on the "preferred API for XML"
> (or perhaps "preferred small set of APIs, each tuned for a specific
> purpose"), I expect it would have been incorporated in the core.

Martin v. L?wis objected:
 > That's not true. The current xml package *is* the consensus of
 > xml-sig.

Fred Drake clarifies
> It pretty much was at the time, at any rate.  It's not clear to me that the
> xml package shipped in 2.4 and several preceeding versions of Python would
> pass muster in the current XML-SIG.

Yes, I'm sorry about not being clearer, and thanks for correcting me. It
was the more recent work in XML which I was thinking of.

-- Michael Chermside


From fdrake at acm.org  Tue Dec 13 21:54:00 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 13 Dec 2005 15:54:00 -0500
Subject: [Python-Dev] "xml" package in standard library
In-Reply-To: <dnnau7$ldg$1@sea.gmane.org>
References: <20051213063118.4eivt285x1f4sks0@login.werra.lunarpages.com>
	<200512131518.38810.fdrake@acm.org> <dnnau7$ldg$1@sea.gmane.org>
Message-ID: <200512131554.00880.fdrake@acm.org>

I've CC'd the XML-SIG list on this; discussion should probably continue on 
python-dev since we're well into the xml package at this point.

On Tuesday 13 December 2005 15:29, Fredrik Lundh wrote:
 > what's the status of PyXML?  is it time to move it over to svn.python.org
 > and bring it up to 1.0 (whatever that would mean?)

I'm not entirely sure myself; I've had no real time to look at it for a while.

I think before we need to worry about PyXML from the perspective of the 
standard library, we need to work out a better way to deal with the "xml" 
package.  Not only is the current state a source of confusion for users, it's 
a problem for testing the standard library if there's also a PyXML installed 
for the same version of Python (the PyXML modules are imported instead of the 
stdlib modules, but the tests for the standard library may reflect fixed 
bugs).  This has bit me a few times.

I'd like to propose that a new package be created in the standard library: 
xmlcore.  This package should contain what's currently in the "xml" package.  
The xml package should be replaced with a single module that's responsible 
for the magic that xml/__init__.py deals with now.  The tests for the xml 
package will be changed to test the xmlcore package.

Advantages:

- People who specifically want the standard library code can get it without
  having PyXML get in the way if installed.

- Existing code using the xml package will continue to work.

Risks:

- Pickles containing classes from the xml package will break if we're not
  really careful.  But I think they're pretty fragile now.

I'll be glad to make these or similar changes if there's concensus on this.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From ncoghlan at gmail.com  Tue Dec 13 22:56:42 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 14 Dec 2005 07:56:42 +1000
Subject: [Python-Dev] "xml" package in standard library
In-Reply-To: <200512131554.00880.fdrake@acm.org>
References: <20051213063118.4eivt285x1f4sks0@login.werra.lunarpages.com>	<200512131518.38810.fdrake@acm.org>
	<dnnau7$ldg$1@sea.gmane.org> <200512131554.00880.fdrake@acm.org>
Message-ID: <439F439A.7040102@gmail.com>

Fred L. Drake, Jr. wrote:
> I'd like to propose that a new package be created in the standard library: 
> xmlcore.  This package should contain what's currently in the "xml" package.  
> The xml package should be replaced with a single module that's responsible 
> for the magic that xml/__init__.py deals with now.  The tests for the xml 
> package will be changed to test the xmlcore package.
> 
> Advantages:
> 
> - People who specifically want the standard library code can get it without
>   having PyXML get in the way if installed.
> 
> - Existing code using the xml package will continue to work.
> 
> Risks:
> 
> - Pickles containing classes from the xml package will break if we're not
>   really careful.  But I think they're pretty fragile now.
> 
> I'll be glad to make these or similar changes if there's concensus on this.

Doing *something* would be good (and what you suggest sounds reasonable).

I spent far too much time on a couple of occasions figuring out that an 
application was blowing up because it expected the full PyXML installation, 
rather than just the standard lib XML core (I don't know the xml package tree 
well enough to tell from the name whether a given subpackage is part of the 
standard lib or not).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From fredrik at pythonware.com  Tue Dec 13 23:17:26 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 13 Dec 2005 23:17:26 +0100
Subject: [Python-Dev] Sharing expat instances
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de><dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de><dni8qe$5eq$1@sea.gmane.org><439D1B6D.9080208@v.loewis.de>
	<dnmsft$t1g$1@sea.gmane.org>
Message-ID: <dnnh9n$c5j$1@sea.gmane.org>

I wrote:

> 1. add an Include/pyexpat.h header file which contains a structure
> similar to the following:

> 2. during pyexpat initialization, initialize all members of this structure, and
> make it available as a PyCObject:

> 3. in cElementTree (or _elementtree, or whatever the python version will
> be named), import pyexpat, fetch the object, and verify

> 4. in cElementTree (...), do all expat calls via the dispatch table.

I've fixed all this, and checked in 1 and 2.

the remaining issue is how to include cElementTree.  the current stand-
alone distribution consists of a single cElementTree module, which is in-
stalled under site-packages, as usual.

to avoid collisions, it's probably best to install the bundled version under
xml.etree, but how do you do that for a C module ?

my current idea is to

    1.  include it under a different name (_elementtree.so)

    2.  add a cElementTree.py under xml.etree, which simply does

            from _elementtree import *

does anyone have a better idea ?

</F>




From bcannon at gmail.com  Wed Dec 14 00:18:24 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Tue, 13 Dec 2005 15:18:24 -0800
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnnh9n$c5j$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de> <dnmsft$t1g$1@sea.gmane.org>
	<dnnh9n$c5j$1@sea.gmane.org>
Message-ID: <bbaeab100512131518r607d4590l78ad53a5a572dcc2@mail.gmail.com>

On 12/13/05, Fredrik Lundh <fredrik at pythonware.com> wrote:
> I wrote:
>
> > 1. add an Include/pyexpat.h header file which contains a structure
> > similar to the following:
>
> > 2. during pyexpat initialization, initialize all members of this structure, and
> > make it available as a PyCObject:
>
> > 3. in cElementTree (or _elementtree, or whatever the python version will
> > be named), import pyexpat, fetch the object, and verify
>
> > 4. in cElementTree (...), do all expat calls via the dispatch table.
>
> I've fixed all this, and checked in 1 and 2.
>
> the remaining issue is how to include cElementTree.  the current stand-
> alone distribution consists of a single cElementTree module, which is in-
> stalled under site-packages, as usual.
>
> to avoid collisions, it's probably best to install the bundled version under
> xml.etree, but how do you do that for a C module ?
>
> my current idea is to
>
>     1.  include it under a different name (_elementtree.so)
>
>     2.  add a cElementTree.py under xml.etree, which simply does
>
>             from _elementtree import *
>
> does anyone have a better idea ?
>

Too bad you can't do 1 and for 2 add ``import _elementtree as
cElementTree`` in etree/__init__.py .  Unless I am missing something
it won't work since ``import xml.etree.cElementTree`` will fail.

Since the way you outlined is the standard way to do it in the stdlib
I doubt anyone has thought of a better way.

-Brett

From amk at amk.ca  Wed Dec 14 00:40:12 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Tue, 13 Dec 2005 18:40:12 -0500
Subject: [Python-Dev] "xml" package in standard library
In-Reply-To: <200512131554.00880.fdrake@acm.org>
References: <20051213063118.4eivt285x1f4sks0@login.werra.lunarpages.com>
	<200512131518.38810.fdrake@acm.org> <dnnau7$ldg$1@sea.gmane.org>
	<200512131554.00880.fdrake@acm.org>
Message-ID: <20051213234012.GA19680@rogue.amk.ca>

On Tue, Dec 13, 2005 at 03:54:00PM -0500, Fred L. Drake, Jr. wrote:
> I'd like to propose that a new package be created in the standard library: 
> xmlcore.  This package should contain what's currently in the "xml" package.

+1; it's what should have been done in the first place.

--amk

From jason.orendorff at gmail.com  Wed Dec 14 01:27:17 2005
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Tue, 13 Dec 2005 19:27:17 -0500
Subject: [Python-Dev] Jython and CPython
In-Reply-To: <439F2A3F.4000703@v.loewis.de>
References: <dnmlem$1a9$1@sea.gmane.org> <439F2A3F.4000703@v.loewis.de>
Message-ID: <bb8868b90512131627m62a86d3bl11016bc6c66a46e4@mail.gmail.com>

On 12/13/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> Fredrik Lundh wrote:
> > BTW, what's the policy wrt. Jython-specific modules in the standard
> library?
>
> I don't think there is enough precedence to have a policy. So far, the
> only places that explicitly support Jython is the test suite, pickle,
> and platform (I wouldn't really count in site here).
>

Actually there's some Jython-specific code in xml/sax/__init__.py.  Two
places, both questionable.  One of them refers to sys.registry.  The other
appears to be a workaround for Jython not having 4-argument __import__.

> If the portability problem can be solved by checking things into Jython
> instead, I think I would prefer that.

Yes, it can be solved that way: Jython could implement pyexpat.  I don't
know just how crazy that idea is; my impression is that it could be done,
perhaps imperfectly, as a wrapper around SAX.

-j
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20051213/6310398a/attachment.html

From pje at telecommunity.com  Wed Dec 14 01:46:56 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 13 Dec 2005 19:46:56 -0500
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnnh9n$c5j$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de> <dnmsft$t1g$1@sea.gmane.org>
Message-ID: <5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>

At 11:17 PM 12/13/2005 +0100, Fredrik Lundh wrote:
>my current idea is to
>
>     1.  include it under a different name (_elementtree.so)
>
>     2.  add a cElementTree.py under xml.etree, which simply does
>
>             from _elementtree import *
>
>does anyone have a better idea ?

I was under the impression that simply installing cElementTree.so in the 
relevant package directory would work; this is what the distutils do for 
extensions with a package name.


From nas at arctrix.com  Wed Dec 14 03:44:50 2005
From: nas at arctrix.com (Neil Schemenauer)
Date: Wed, 14 Dec 2005 02:44:50 +0000 (UTC)
Subject: [Python-Dev] should I really have to install Python before
	Icanbuild it ?
References: <bbaeab100512121317t2fb4de19te0db567af1651615@mail.gmail.com>
	<dnkpog$lk9$1@sea.gmane.org>
	<20051212215452.GA19322@code1.codespeak.net>
Message-ID: <dno0v2$jqg$1@sea.gmane.org>

Armin Rigo <arigo at tunes.org> wrote:
> On Mon, Dec 12, 2005 at 10:23:27PM +0100, Fredrik Lundh wrote:
>> $(AST_H) $(AST_C): $(AST_ASDL) $(ASDLGEN_FILES)
>>         -$(PYTHON) $(ASDLGEN) $(AST_ASDL)

> The same just-ignore-it behavior can bite if the script genuinely fails
> after you just made a typo in one of the input files, for example.
> Doesn't look particularly clean to me, if you want my opinion.

Perhaps a good solution would be to have a separate make rule for
generating the AST code (i.e. not included in the overall dependancy
graph).  That increases the chance that they don't get regenerated
when they should but people hacking on the AST files should notice
the error pretty easily.  Other people should always be able to
build from the files checked in to SVN and so having the dependancy
there is just a source of trouble.

  Neil


From mmclay at comcast.net  Wed Dec 14 04:18:12 2005
From: mmclay at comcast.net (Michael McLay)
Date: Tue, 13 Dec 2005 22:18:12 -0500
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnnh9n$c5j$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<dnmsft$t1g$1@sea.gmane.org> <dnnh9n$c5j$1@sea.gmane.org>
Message-ID: <200512132218.12749.mmclay@comcast.net>

On Tuesday 13 December 2005 17:17, Fredrik Lundh wrote:
> the remaining issue is how to include cElementTree.  the current stand-
> alone distribution consists of a single cElementTree module, which is in-
> stalled under site-packages, as usual.
>
> to avoid collisions, it's probably best to install the bundled version
> under xml.etree, but how do you do that for a C module ?
>
> my current idea is to
>
>     1.  include it under a different name (_elementtree.so)
>
>     2.  add a cElementTree.py under xml.etree, which simply does
>
>             from _elementtree import *
>
> does anyone have a better idea ?

Avoiding imaginaary name collisions and putting cElementTree into the xml 
package is making the task harder than need be and will perpetuate the 
problems caused by the earlier decision to do magic in the xml import 
statement. The use of magic was a violation of "explicit is better than 
implicit". Forgeting this rule has caused pain to many people who didn't 
expect a magic side effects from simply installing PyXML. Adding cElementTree 
to the mess that already exists doessn't make sense, and the only 
justification for not adding a new top level package seems to be to avoid an 
imaginary potential for name collisions. 

Why repeat the mistake of PyXML simply to have all the xml software in the 
same top level package? Perhaps if this were a common practice in the stdlib 
it might make sense. For instance, if there were a gui.Tkinter and a 
gui.PyGtk and a gui.wxPython it might look more consistent. Who is pushing to 
put all xml software inside the xml package?  Collisions with names of a top 
level import statement are rare and Python has supports renaming in the rare 
event that someone did need to rename the module.

 import etree as stdetree.

I would like the cElementTree version of the package have a shorter name. For 
example etree. The Python implementation could continue to be named 
ElementTree or shorten it to pyetree. The cElementTree version will probably 
be the most frequently used since it is faster, so why not make it's name 
short.

One final addition would be to include a "See Also" reference to the new etree 
and pyetree modules in the introduction of the xml package.


From kbk at shore.net  Wed Dec 14 04:29:36 2005
From: kbk at shore.net (Kurt B. Kaiser)
Date: Tue, 13 Dec 2005 22:29:36 -0500 (EST)
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200512140329.jBE3TaMh020561@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  383 open (+11) /  2990 closed (+10) /  3373 total (+21)
Bugs    :  927 open (+19) /  5415 closed (+20) /  6342 total (+39)
RFE     :  204 open ( +4) /   192 closed ( +1) /   396 total ( +5)

New / Reopened Patches
______________________

use PyOS_ReadlineFunctionPointer in non-interractive input   (2004-05-18)
CLOSED http://python.org/sf/955928  reopened by  loewis

use PyOS_ReadlineFunctionPointer in non-interractive input   (2005-11-27)
       http://python.org/sf/1367628  opened by  Gregory Lielens

Remove usage of UserDict from os.py  (2005-11-27)
       http://python.org/sf/1367711  opened by  Wolfgang Langner

replace usage of UserDict with new dict class  (2005-11-27)
CLOSED http://python.org/sf/1367717  opened by  Wolfgang Langner

email/Charset.py  (2005-11-28)
       http://python.org/sf/1368247  opened by  Damjan Georgievski

UUID module for Python  (2005-11-29)
       http://python.org/sf/1368955  opened by  Ka-Ping Yee

Module fixedlenfields for standard lib  (2005-11-29)
       http://python.org/sf/1369028  opened by  Michael Str?der

Fix of bug 1366000  (2005-11-30)
       http://python.org/sf/1370147  opened by  STINNER Victor

ConfigParser to accept a custom dict to allow ordering  (2005-12-01)
       http://python.org/sf/1371075  opened by  Micah Elliott

fix UnixBrowswer failure when no browser running  (2005-12-02)
       http://python.org/sf/1372125  opened by  Greg Couch

tiny chunk of unused code in cookielib  (2005-12-04)
CLOSED http://python.org/sf/1372836  opened by  John J Lee

Missing \versionadded in urllib2 and cookielib docs  (2005-12-04)
CLOSED http://python.org/sf/1372995  opened by  John J Lee

chunk.py can't handle >2GB chunks  (2005-12-05)
       http://python.org/sf/1373643  opened by  Christer Weinigel

Tweak pprint.PrettyPrinter.format for subclassing  (2005-12-05)
       http://python.org/sf/1373762  opened by  Mark Hirota

Broader iterable support for xmlrpclib  (2005-12-05)
       http://python.org/sf/1374063  opened by  Skip Montanaro

Improper handling of duplicate cookies  (2005-12-06)
       http://python.org/sf/1375011  opened by  Viraj Alankar

LibRef: reworked chapter organization  (2005-12-07)
       http://python.org/sf/1375417  opened by  A.M. Kuchling

subprocess.CalledProcessError uses errno incorrectly  (2005-12-08)
       http://python.org/sf/1376309  opened by  Michael Hoffman

Use 'seealso' to add examples to LibRef  (2005-12-08)
       http://python.org/sf/1376361  opened by  A.M. Kuchling

fix description of format_exc in traceback doc  (2005-12-08)
       http://python.org/sf/1376914  opened by  Ilya Sandler

xml.parsers.expat documentation fix  (2005-12-10)
       http://python.org/sf/1377848  opened by  Ori Avtalion

weakref callbacks are called only if the weakref is alive  (2005-12-12)
       http://python.org/sf/1379023  opened by  Noam Raphael

StreamReader.readline with size reading multiple lines  (2005-12-13)
       http://python.org/sf/1379332  opened by  Matthew Mueller

Patches Closed
______________

use PyOS_ReadlineFunctionPointer in non-interractive input   (2004-05-18)
       http://python.org/sf/955928  closed by  greglielens

use PyOS_ReadlineFunctionPointer in non-interractive input   (2004-05-18)
       http://python.org/sf/955928  closed by  greglielens

EditorWindow's title with non-ASCII chars.  (2005-03-14)
       http://python.org/sf/1162825  closed by  loewis

Fix for signal related abort in Visual Studio 2005  (2005-11-07)
       http://python.org/sf/1350409  closed by  loewis

PyOS_Readline  (2005-07-04)
       http://python.org/sf/1232343  closed by  dalcinl

PyOS_Readline  (2005-07-04)
       http://python.org/sf/1232343  deleted by  dalcinl

cgi: replace usage of UserDict with new dict class  (2005-11-27)
       http://python.org/sf/1367717  closed by  tds33

tiny chunk of unused code in cookielib  (2005-12-04)
       http://python.org/sf/1372836  closed by  akuchling

SimpleXMLRPCServer.py optional allow_none argument  (2004-02-09)
       http://python.org/sf/893642  closed by  akuchling

SimpleXMLRPCServer optional allow_none / encoding arguments  (2004-10-02)
       http://python.org/sf/1039083  closed by  akuchling

Missing \versionadded in urllib2 and cookielib docs  (2005-12-04)
       http://python.org/sf/1372995  closed by  akuchling

Adding new regrtest resource 'urlfetch'  (2005-08-30)
       http://python.org/sf/1276356  closed by  perky

New / Reopened Bugs
___________________

maximum length not enforce in cgi.parse()  (2005-11-27)
       http://python.org/sf/1367631  opened by  Andrew Rogers

loogger module locks  (2005-11-27)
       http://python.org/sf/1367814  opened by  Chris Fuller

fix for scheme identification in urllib2?  (2005-11-28)
       http://python.org/sf/1368312  opened by  Ben Boals

prompt_user_passwd() in FancyURLopener  (2005-11-28)
       http://python.org/sf/1368368  opened by  Bj?rn Lindqvist

python.dir still refers to python-whatsnew23  (2005-11-28)
CLOSED http://python.org/sf/1368481  opened by  Bernhard Herzog

threading.Timer: Constructor does not handle args correctly  (2005-11-28)
CLOSED http://python.org/sf/1368515  opened by  dominikush

clearing up dictionary keys/set member docs  (2005-11-28)
       http://python.org/sf/1368768  opened by  Mike Meyer

bad external link in xmlrpc lib page  (2005-11-28)
CLOSED http://python.org/sf/1368827  opened by  Jeff Bauer

freeze idle-python2.3 on my debia n sarge  (2005-11-29)
       http://python.org/sf/1369116  opened by  Gabriel Santonja

memory leak - ast_error_finish  (2005-11-30)
CLOSED http://python.org/sf/1370197  opened by  Jim Jewett

Bytecode problems with test_colorsys  (2005-11-30)
       http://python.org/sf/1370322  opened by  Reinhold Birkenfeld

asynchat.async_chat.push() function doesnt say when failed  (2005-11-30)
       http://python.org/sf/1370380  opened by  Jan David Mol

locale.windows_locale   (2005-12-01)
       http://python.org/sf/1371247  opened by  Greg Hazel

distutils is silent about multiple -I/-L/-R  (2005-12-02)
       http://python.org/sf/1371826  opened by  Skip Montanaro

minidom namespace problems  (2005-12-02)
       http://python.org/sf/1371937  opened by  A.M. Kuchling

Cookie and multiple names  (2005-12-03)
       http://python.org/sf/1372650  opened by  Viraj Alankar

email.Header should preserve original FWS  (2005-12-04)
       http://python.org/sf/1372770  opened by  Nathan Herring

diffs in working copy after a build  (2005-12-05)
CLOSED http://python.org/sf/1373150  opened by  Michael Hudson

r41552 broke test_file on OS X  (2005-12-05)
       http://python.org/sf/1373161  reopened by  mwh

r41552 broke test_file on OS X  (2005-12-05)
       http://python.org/sf/1373161  opened by  Michael Hudson

os.makedirs fail if path contains os.pardir  (2005-12-05)
       http://python.org/sf/1373197  opened by  Nir Soffer

Collapse distutils docs  (2005-12-07)
       http://python.org/sf/1375258  opened by  Skip Montanaro

Tutorial errors  (2005-12-07)
       http://python.org/sf/1375599  opened by  Glen Kaukola

test_struct crashed, py2.3.5, solaris 10  (2005-12-08)
       http://python.org/sf/1376400  opened by  rrogans

Memory leak in the email package  (2005-12-08)
       http://python.org/sf/1376775  opened by  ken668

read() / readline() blow up if file has even number of char.  (2005-12-09)
CLOSED http://python.org/sf/1377394  opened by  superwesman

segfaults when using __del__ and weakrefs  (2005-12-10)
       http://python.org/sf/1377858  opened by  Carl Friedrich Bolz

Bus error in ast  (2005-12-10)
CLOSED http://python.org/sf/1377897  opened by  Skip Montanaro

source utf8  (2005-12-11)
       http://python.org/sf/1378022  opened by  zhao

Import value 1e400  from pyc fails  (2005-12-11)
       http://python.org/sf/1378305  opened by  Guenter Jantzen

a problem of urllib using open_local_file  (2005-12-12)
       http://python.org/sf/1378455  opened by  Weongyo Jeong

urllib2.HTTPBasicAuthHandler fails on non-default port  (2005-12-12)
       http://python.org/sf/1378679  opened by  Mikhail Gusarov

logging : fileConfig does not check existance of the file  (2005-12-12)
       http://python.org/sf/1378755  opened by  Didrik Pinte

socket.recv(OOB) raises exception on closed socket  (2005-12-12)
       http://python.org/sf/1379209  opened by  Roy Smith

StreamReader.readline doesn't advance on decode errors  (2005-12-13)
       http://python.org/sf/1379393  opened by  Matthew Mueller

email.Header encode() unicode P2.3xP2.4  (2005-12-13)
       http://python.org/sf/1379416  opened by  Jan Novak

HP-UX thread stack size needs to be increased  (2005-12-13)
       http://python.org/sf/1379804  opened by  Blade

HP-UX: Can't shl_load() a library containing Thread Local  (2005-12-14)
       http://python.org/sf/1379984  opened by  Blade

"unicode_escape" and "raw_unicode_escape" encoding is broken  (2005-12-13)
       http://python.org/sf/1379994  opened by  Mark Mc Mahon

Bugs Closed
___________

Fails assertion in winsig.c under VC 8.0  (2005-03-21)
       http://python.org/sf/1167262  closed by  loewis

python.dir still refers to python-whatsnew23  (2005-11-28)
       http://python.org/sf/1368481  closed by  birkenfeld

threading.Timer: Constructor does not handle args correctly  (2005-11-28)
       http://python.org/sf/1368515  closed by  effbot

bad external link in xmlrpc lib page  (2005-11-29)
       http://python.org/sf/1368827  closed by  effbot

memory leak - ast_error_finish  (2005-11-30)
       http://python.org/sf/1370197  closed by  nnorwitz

SVN webbrowser.py fix 41419 didn't  (2005-11-09)
       http://python.org/sf/1352621  closed by  gregcouch

SimpleXMLRPCServer does not set FD_CLOEXEC  (2005-06-17)
       http://python.org/sf/1222790  closed by  akuchling

SimpleXMLRPCServer cannot handle large requests  (2003-08-21)
       http://python.org/sf/792570  closed by  akuchling

SimpleXMLRPCServer example is broken  (2004-10-06)
       http://python.org/sf/1041501  closed by  akuchling

strange behaviour of xmlrpclib.Server proxy  (2004-02-20)
       http://python.org/sf/901198  closed by  akuchling

xmlrpclib does not use http proxy  (2005-11-05)
       http://python.org/sf/1349316  closed by  akuchling

xmlrpclib.DateTime.decode() should stringify argument  (2005-03-16)
       http://python.org/sf/1164912  closed by  akuchling

xml.sax.expatreader doesn't pass encoding to ParserCreate  (2005-09-02)
       http://python.org/sf/1281032  closed by  akuchling

spawnlp is missing  (2005-11-21)
       http://python.org/sf/1363104  closed by  akuchling

diffs in working copy after a build  (2005-12-04)
       http://python.org/sf/1373150  closed by  nnorwitz

r41552 broke test_file on OS X  (2005-12-04)
       http://python.org/sf/1373161  closed by  nnorwitz

Segfaults from unaligned loads in floatobject.c  (2005-11-02)
       http://python.org/sf/1346144  closed by  mwh

read() / readline() blow up if file has even number of char.  (2005-12-09)
       http://python.org/sf/1377394  closed by  lemburg

running test_codecmaps_* takes too much effort  (2004-08-18)
       http://python.org/sf/1010952  closed by  perky

Bus error in ast  (2005-12-10)
       http://python.org/sf/1377897  closed by  nnorwitz

cjkcodec compile error under AIX 5.2 on symbol 100_encode  (2005-09-14)
       http://python.org/sf/1290333  closed by  perky

New / Reopened RFE
__________________

split() string method has two splitting algorithms  (2005-11-28)
       http://python.org/sf/1367936  opened by  crackwitz

shutils cannot copy owner  (2005-11-28)
       http://python.org/sf/1368091  opened by  Pierre Ossman

Start and end parameters for list.count()  (2005-12-01)
       http://python.org/sf/1370948  opened by  Christoph Zwerschke

Write user's version of RefGuide  (2005-12-08)
       http://python.org/sf/1376292  opened by  A.M. Kuchling

python executable optionally should search script on PATH  (2005-12-13)
       http://python.org/sf/1379573  opened by  Christoph Conrad

RFE Closed
__________

request for bugs.python.org  (2003-10-02)
       http://python.org/sf/816628  closed by  akuchling


From jason.orendorff at gmail.com  Wed Dec 14 05:18:24 2005
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Tue, 13 Dec 2005 23:18:24 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439EBDF1.80209@zope.com>
References: <4399F967.3080300@colorstudy.com> <439C51C8.2010909@zope.com>
	<1134404788.950.24.camel@geddy.wooz.org> <439EBDF1.80209@zope.com>
Message-ID: <bb8868b90512132018g34f8ac45t597d954251d112ac@mail.gmail.com>

Barry Warsaw wrote:
>       - If your class is intended to be subclassed, and you have attributes
>         that you do not want subclasses to use, consider naming them with
>         double leading underscores and no trailing underscores.  This invokes
>         Python's name mangling algorithm, where the name of the class is
>         mangled into the attribute name.  This helps avoid attribute name
>         collisions should subclasses inadvertently contain attributes with the
>         same name.
>
>         Note 1: Note that only the simple class name is used in the mangled
>         name, so if a subclass chooses both the same class name and attribute
>         name, you can still get name collisions.
>
>         Note 2: Name mangling can make certain uses, such as debugging, less
>         convenient.  However the name mangling algorithm is well documented
>         and easy to perform manually.

Hmm.  How about just:  "Put two leading underscores on an attribute's
name to strongly discourage code outside the class from accessing it."

-j

From jason.orendorff at gmail.com  Wed Dec 14 05:27:58 2005
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Tue, 13 Dec 2005 23:27:58 -0500
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <439E9AD1.9080103@livinglogic.de>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>
	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>
	<439E1D36.1090009@v.loewis.de>
	<17310.21156.907292.278567@montanaro.dyndns.org>
	<ca471dc20512122052k4b985d98m9ce1662858f2d1bc@mail.gmail.com>
	<439E9AD1.9080103@livinglogic.de>
Message-ID: <bb8868b90512132027s3dd8c12ak9de0e036249886bf@mail.gmail.com>

On 12/13/05, Walter D?rwald <walter at livinglogic.de> wrote:
> Guido van Rossum wrote:
> > I don't think that SAX is unpythonic, but it's pretty low-level and
> > mostly of use to people writing higher-level XML parsers (my parsexml
> > module uses it).
>
> Having to define classes that conform to a certain API and registering
> instances of those classes as callbacks with the parser doesn't look
> that pythonic to me. An iterator API seems much more pythonic.

Strongly agree.  This very morning I wrote a long tirade about how I
wish Python had true coroutines, for the sole reason that I could wrap
SAX in an iterator-based API.

Eventually I decided it was SAX's fault for having such a crummy API,
so I didn't post it.

-j

From bcannon at gmail.com  Wed Dec 14 06:29:43 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Tue, 13 Dec 2005 21:29:43 -0800
Subject: [Python-Dev] should I really have to install Python before
	Icanbuild it ?
In-Reply-To: <dno0v2$jqg$1@sea.gmane.org>
References: <bbaeab100512121317t2fb4de19te0db567af1651615@mail.gmail.com>
	<dnkpog$lk9$1@sea.gmane.org>
	<20051212215452.GA19322@code1.codespeak.net>
	<dno0v2$jqg$1@sea.gmane.org>
Message-ID: <bbaeab100512132129s73a3b9ccu4de78a86fd82653c@mail.gmail.com>

On 12/13/05, Neil Schemenauer <nas at arctrix.com> wrote:
> Armin Rigo <arigo at tunes.org> wrote:
> > On Mon, Dec 12, 2005 at 10:23:27PM +0100, Fredrik Lundh wrote:
> >> $(AST_H) $(AST_C): $(AST_ASDL) $(ASDLGEN_FILES)
> >>         -$(PYTHON) $(ASDLGEN) $(AST_ASDL)
>
> > The same just-ignore-it behavior can bite if the script genuinely fails
> > after you just made a typo in one of the input files, for example.
> > Doesn't look particularly clean to me, if you want my opinion.
>
> Perhaps a good solution would be to have a separate make rule for
> generating the AST code (i.e. not included in the overall dependancy
> graph).  That increases the chance that they don't get regenerated
> when they should but people hacking on the AST files should notice
> the error pretty easily.  Other people should always be able to
> build from the files checked in to SVN and so having the dependancy
> there is just a source of trouble.
>

Sounds reasonable to me.  Would just need to make sure that it is
documented in the proper places so people know to do it.

-Brett

From fredrik at pythonware.com  Wed Dec 14 07:05:32 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 14 Dec 2005 07:05:32 +0100
Subject: [Python-Dev] Sharing expat instances
References: <17304.33755.693941.811233@montanaro.dyndns.org><dnmsft$t1g$1@sea.gmane.org>
	<dnnh9n$c5j$1@sea.gmane.org>
	<200512132218.12749.mmclay@comcast.net>
Message-ID: <dnocne$kt4$1@sea.gmane.org>

Michael McLay wrote:

> Avoiding imaginaary name collisions and putting cElementTree into the xml
> package

there's nothing imaginary here -- cElementTree is an existing and quite popular
module, and will remain available as a separate distribution.

it would be nice if people could install that kit also under 2.5 without risking to
mess up their Python installation.

(another solution would of course to rule out use of cElementTree by modules
shipped with Python...)

</F>




From fdrake at acm.org  Wed Dec 14 08:00:18 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 14 Dec 2005 02:00:18 -0500
Subject: [Python-Dev] "xml" package in standard library
In-Reply-To: <20051213234012.GA19680@rogue.amk.ca>
References: <20051213063118.4eivt285x1f4sks0@login.werra.lunarpages.com>
	<200512131554.00880.fdrake@acm.org>
	<20051213234012.GA19680@rogue.amk.ca>
Message-ID: <200512140200.18421.fdrake@acm.org>

On Tuesday 13 December 2005 18:40, A.M. Kuchling wrote:
 > +1; it's what should have been done in the first place.

If only I'd understood that when I added the xml/PyXML hack to the stdlib 
years ago.  :-(

Fixed now.  I'll deal with the documentation in a few days; I actually expect 
to have some time.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From walter at livinglogic.de  Wed Dec 14 09:04:43 2005
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed, 14 Dec 2005 09:04:43 +0100
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <ca471dc20512131109y6b02d371u6657be586089987a@mail.gmail.com>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>	
	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>	
	<439E1D36.1090009@v.loewis.de>	
	<17310.21156.907292.278567@montanaro.dyndns.org>	
	<ca471dc20512122052k4b985d98m9ce1662858f2d1bc@mail.gmail.com>	
	<439E9AD1.9080103@livinglogic.de>
	<ca471dc20512131109y6b02d371u6657be586089987a@mail.gmail.com>
Message-ID: <439FD21B.5010306@livinglogic.de>

Guido van Rossum wrote:

> On 12/13/05, Walter D?rwald <walter at livinglogic.de> wrote:
>> Guido van Rossum wrote:
>>> I don't think that SAX is unpythonic, but it's pretty low-level and
>>> mostly of use to people writing higher-level XML parsers (my parsexml
>>> module uses it).
>> Having to define classes that conform to a certain API and registering
>> instances of those classes as callbacks with the parser doesn't look
>> that pythonic to me. An iterator API seems much more pythonic.
> 
> Perhaps. Although the SAX API lets you leave a callback undefined if
> you don't have a need to handle those events; that's a bit trickier to
> do with an iterator.

Changing the iterator to only generate the events you need requires 
passing information to the iterator. And when you do that you can just 
as well pass information about which function to call at which event.

But IMHO the main difference isn't dispatching, but who's in control.

> Also the different callbacks have different
> signatures.

True, I've always wondered why SAX uses a startelement callback which 
gets passed a complete attribute dictionary. IMHO for <foo 
bar="baz"/>spam</foo> the following event sequence would be better:

starttagbegin foo
attributebegin bar
text baz
attributeend bar
starttagend foo
text spam
endtag foo

This would simplify signatures (always one string argument) and it would 
leave handling entity references inside attribute values to the 
application (or at least a higher level of the parser).

> But since /F solved this for ElementTree I have to mostly agree with you. :-)

Unfortunately there probably won't be that many parsers that support 
iterparse(). Most parsers existing outside the Python world use the 
callback model and turning a callback parser into a iterator parser 
requires support for incremental parsing (which has a certain latency) 
or stack switching tricks.

>> Then again, pythonic is whatever you say that it is. ;)
> 
> Not at all. I will argue but I will also take arguments from others. Seriously.

Bye,
    Walter D?rwald


From mal at egenix.com  Wed Dec 14 12:17:33 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 14 Dec 2005 12:17:33 +0100
Subject: [Python-Dev] "xml" package in standard library
In-Reply-To: <200512140200.18421.fdrake@acm.org>
References: <20051213063118.4eivt285x1f4sks0@login.werra.lunarpages.com>	<200512131554.00880.fdrake@acm.org>	<20051213234012.GA19680@rogue.amk.ca>
	<200512140200.18421.fdrake@acm.org>
Message-ID: <439FFF4D.8030607@egenix.com>

Fred L. Drake, Jr. wrote:
> On Tuesday 13 December 2005 18:40, A.M. Kuchling wrote:
>  > +1; it's what should have been done in the first place.
> 
> If only I'd understood that when I added the xml/PyXML hack to the stdlib 
> years ago.  :-(
> 
> Fixed now.  I'll deal with the documentation in a few days; I actually expect 
> to have some time.

I saw your checkin: wouldn't it be better to keep the xml directory
and xml/__init__.py instead of adding an xml.py module ?

The semantics of a package import are different than that
of a module import, so this may make a difference. I did the
same for the mx packages some years ago (when I moved everything
under the mx package) and then used code like this to make sure
that pickles continued to work as well as have them redirected
to the new package once they were stored again:

DateTime/__init__.py:

# Redirect all imports to the corresponding mx package
def _redirect(mx_subpackage):
    global __path__
    import os,mx
    __path__ = [os.path.join(mx.__path__[0],mx_subpackage)]
_redirect('DateTime')

# Now load all important symbols
from mx.DateTime import *
from mx.DateTime import __version__


Note that in doing so, pickles will get redirected to the new
package. I'm not sure whether that's what you would want for
the xml/PyXML package, though: it would either direct them
to the xmlcore package (bypassing PyXML on systems where it's
installed) or direct them to the PyXML versions (bypassing
the xmlcore package and causing them only to be unpicklable
on systems with PyXML installed).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 14 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Wed Dec 14 12:20:20 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 14 Dec 2005 12:20:20 +0100
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <dnmiaj$l8l$1@sea.gmane.org>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>	<439E1D36.1090009@v.loewis.de>	<17310.21156.907292.278567@montanaro.dyndns.org>	<ca471dc20512122052k4b985d98m9ce1662858f2d1bc@mail.gmail.com><439E9AD1.9080103@livinglogic.de>	<439EABFB.3020405@gmail.com><dnmc1q$km$1@sea.gmane.org>	<439EBBEA.9030705@egenix.com>
	<dnmiaj$l8l$1@sea.gmane.org>
Message-ID: <439FFFF4.4090005@egenix.com>

Fredrik Lundh wrote:
> M.-A. Lemburg wrote:
> 
>> Some questions:
>>
>> * Are you going to contribute cElementTree as well ?
> 
> yes, but there are some build issues we need to sort out first (both pyexpat
> and cET link to their own copies of expat)

Great !

> we also need to figure out how to import the bundled version; should it be
> cElementTree, xml.etree.cElementTree, or just xml.etree.ElementTree
> (which would then fallback on the Python version if cElementTree isn't
> built) ?

If the semantics are identical I'd prefer the latter approach
of using the faster variant if possible.

>> * What was the motivation to not include the whole ElementTree
>>  package ?
> 
> this is a perfect time to get rid of some little-used stuff.  if there's enough user
> demand, we can always add a few more modules before 2.5 goes out of the
> door...

Ok.

>> * I'm missing the usual "Licensed to PSF under a Contributor Agreement."
>>  in the copyright notices of the files:
>>
>>  http://www.python.org/psf/contrib.html
>>
>>  I assume that you'll add these, right ?
> 
> will fix.
> 
>> * How should users that want to use the latest and greatest
>>  (more recent) distribution directly from your site go about in
>>  their apps ? Using from...as contructs ?
> 
> from-import or import-as works fine

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 14 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mmclay at comcast.net  Wed Dec 14 15:11:08 2005
From: mmclay at comcast.net (Michael McLay)
Date: Wed, 14 Dec 2005 09:11:08 -0500
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnocne$kt4$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<200512132218.12749.mmclay@comcast.net>
	<dnocne$kt4$1@sea.gmane.org>
Message-ID: <200512140911.08319.mmclay@comcast.net>

On Wednesday 14 December 2005 01:05, Fredrik Lundh wrote:
> Michael McLay wrote:
> > Avoiding imaginaary name collisions and putting cElementTree into the xml
> > package
>
> there's nothing imaginary here -- cElementTree is an existing and quite
> popular module, and will remain available as a separate distribution.
>
> it would be nice if people could install that kit also under 2.5 without
> risking to mess up their Python installation.
>
> (another solution would of course to rule out use of cElementTree by
> modules shipped with Python...)

Renaming the cElementTree in the standard distribution to etree would 
elmiinate collisions with the existing cElementTree. A few other names to 
consider would be xmltree or xmlTree.  Or, if the consensus is to keep it in 
the xml package, the name could still be shorted to calling it xml.tree.  

Average name lengths in the standard distribution are getting longer. In some 
cases the longer names are very helpful when browsing the module index. For 
instance, the function of SimpleXMLRPCServer is immediately apparent. To me, 
the purpose of a package named ElementTree isn't as apparent. While there is 
value in having meaningful names, there is also an advantage in having names 
short for commonly used tools. Imagine how painful it would be to type 
Operatingsystem instead of os. The name xmltree would be shorter and as 
descriptive as xml.cElementTree. The name etree would be shorter, but less 
descriptive.

From mmclay at comcast.net  Wed Dec 14 15:12:35 2005
From: mmclay at comcast.net (Michael McLay)
Date: Wed, 14 Dec 2005 09:12:35 -0500
Subject: [Python-Dev] Website cruft
Message-ID: <200512140912.35909.mmclay@comcast.net>

The download page http://www.python.org/download/ has a reference to "CVS 
Access" http://www.python.org/download/cvs.html that references a page with 
pointers to the old SourceForge CVS. Would someone please update these pages 
to reflect the move to Subversion.


From steve at holdenweb.com  Wed Dec 14 15:51:10 2005
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 14 Dec 2005 14:51:10 +0000
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <200512140911.08319.mmclay@comcast.net>
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<200512132218.12749.mmclay@comcast.net>	<dnocne$kt4$1@sea.gmane.org>
	<200512140911.08319.mmclay@comcast.net>
Message-ID: <dnpbh4$l1m$1@sea.gmane.org>

Michael McLay wrote:
> On Wednesday 14 December 2005 01:05, Fredrik Lundh wrote:
> 
>>Michael McLay wrote:
>>
>>>Avoiding imaginaary name collisions and putting cElementTree into the xml
>>>package
>>
>>there's nothing imaginary here -- cElementTree is an existing and quite
>>popular module, and will remain available as a separate distribution.
>>
>>it would be nice if people could install that kit also under 2.5 without
>>risking to mess up their Python installation.
>>
>>(another solution would of course to rule out use of cElementTree by
>>modules shipped with Python...)
> 
> 
> Renaming the cElementTree in the standard distribution to etree would 
> elmiinate collisions with the existing cElementTree. A few other names to 
> consider would be xmltree or xmlTree.  Or, if the consensus is to keep it in 
> the xml package, the name could still be shorted to calling it xml.tree.  
> 
> Average name lengths in the standard distribution are getting longer. In some 
> cases the longer names are very helpful when browsing the module index. For 
> instance, the function of SimpleXMLRPCServer is immediately apparent. To me, 
> the purpose of a package named ElementTree isn't as apparent. While there is 
> value in having meaningful names, there is also an advantage in having names 
> short for commonly used tools. Imagine how painful it would be to type 
> Operatingsystem instead of os. The name xmltree would be shorter and as 
> descriptive as xml.cElementTree. The name etree would be shorter, but less 
> descriptive.

Which reminds me, what about the idea of reducing certain packages 
(Carbon, dostutils and email come to mind) to just a single entry in the 
global module index and adding a first-level TOC at the beginning of 
that section of content?

Even if we then have to call it the global package and module index!

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From steve at holdenweb.com  Wed Dec 14 16:12:58 2005
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 14 Dec 2005 15:12:58 +0000
Subject: [Python-Dev] Website cruft
In-Reply-To: <200512140912.35909.mmclay@comcast.net>
References: <200512140912.35909.mmclay@comcast.net>
Message-ID: <dnpcpv$pri$1@sea.gmane.org>

Michael McLay wrote:
> The download page http://www.python.org/download/ has a reference to "CVS 
> Access" http://www.python.org/download/cvs.html that references a page with 
> pointers to the old SourceForge CVS. Would someone please update these pages 
> to reflect the move to Subversion.
> 
I've made a quick first-cut change to ensure that people don't start 
using the CVS repository by accident. Someone else might want to replace 
the FAQ reference, though really it's asking for trouble (as we are 
currently finding out) to duplicate this type of information. Arguably 
the FAQ should reference the main web.

However, this change highlights the fact that the checkins mailing-list 
description at

   http://mail.python.org/mailman/listinfo/python-checkins

is so out of date it warns that the CVS repository on cvs.python.org 
should no longer be used as we have now migrated to CVS on Sourceforge :-(

Could someone fix that?

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From wl at flexis.de  Wed Dec 14 16:12:20 2005
From: wl at flexis.de (Wolfgang)
Date: Wed, 14 Dec 2005 16:12:20 +0100
Subject: [Python-Dev] PEP 8 updates/clarifications, function/method style
Message-ID: <dnpcok$pqi$1@sea.gmane.org>

Hello,

PEP 8 for function and method names:
-----
Function Names

       Function names should be lowercase, possibly with words separated by
       underscores to improve readability.  mixedCase is allowed only in
       contexts where that's already the prevailing style (e.g. threading.py),
       to retain backwards compatibility.

Method Names and Instance Variables

       The story is largely the same as with functions: in general, use
       lowercase with words separated by underscores as necessary to improve
       readability.
-----

We need a clear style for function and method names
now std lib uses "foo_bar" sometimes "foobar"
and sometimes "fooBar".

The use of lowercase with underscores is the default so
the python std lib should use it in all modules.
-> Python 3000 std lib


Or should we switch to camelCase with lowercase first letter ?
As most other Languages prefer this (Java, C#, C++, ...)


bye by Wolfgang


From fdrake at acm.org  Wed Dec 14 16:30:17 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 14 Dec 2005 10:30:17 -0500
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnpbh4$l1m$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<200512140911.08319.mmclay@comcast.net>
	<dnpbh4$l1m$1@sea.gmane.org>
Message-ID: <200512141030.17585.fdrake@acm.org>

On Wednesday 14 December 2005 09:51, Steve Holden wrote:
 > Which reminds me, what about the idea of reducing certain packages
 > (Carbon, dostutils and email come to mind) to just a single entry in the
 > global module index and adding a first-level TOC at the beginning of
 > that section of content?

This echos a recent Doc-SIG post from Skip Montanaro that I've been meaning to 
reply to.  I've now done that:

  http://mail.python.org/pipermail/doc-sig/2005-December/003453.html


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From fdrake at acm.org  Wed Dec 14 16:37:12 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 14 Dec 2005 10:37:12 -0500
Subject: [Python-Dev] Website cruft
In-Reply-To: <dnpcpv$pri$1@sea.gmane.org>
References: <200512140912.35909.mmclay@comcast.net>
	<dnpcpv$pri$1@sea.gmane.org>
Message-ID: <200512141037.12200.fdrake@acm.org>

On Wednesday 14 December 2005 10:12, Steve Holden wrote:
 > However, this change highlights the fact that the checkins mailing-list
 > description at
 >
 >    http://mail.python.org/mailman/listinfo/python-checkins
 >
 > is so out of date it warns that the CVS repository on cvs.python.org
 > should no longer be used as we have now migrated to CVS on Sourceforge :-(

Try it now.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From hoffman at ebi.ac.uk  Wed Dec 14 17:06:57 2005
From: hoffman at ebi.ac.uk (Michael Hoffman)
Date: Wed, 14 Dec 2005 16:06:57 +0000
Subject: [Python-Dev] PEP 8 updates/clarifications, function/method style
In-Reply-To: <dnpcok$pqi$1@sea.gmane.org>
References: <dnpcok$pqi$1@sea.gmane.org>
Message-ID: <Pine.LNX.4.62.0512141605320.32164@qnzvnan.rov.np.hx>

[Wolfgang]

> Or should we switch to camelCase with lowercase first letter ?
> As most other Languages prefer this (Java, C#, C++, ...)

They also use curly braces instead of indentation to indicate block
structure. Maybe we should switch to that too. <wink>
-- 
Michael Hoffman <hoffman at ebi.ac.uk>
European Bioinformatics Institute


From jason.orendorff at gmail.com  Wed Dec 14 17:39:55 2005
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Wed, 14 Dec 2005 11:39:55 -0500
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <439FD21B.5010306@livinglogic.de>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>
	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>
	<439E1D36.1090009@v.loewis.de>
	<17310.21156.907292.278567@montanaro.dyndns.org>
	<ca471dc20512122052k4b985d98m9ce1662858f2d1bc@mail.gmail.com>
	<439E9AD1.9080103@livinglogic.de>
	<ca471dc20512131109y6b02d371u6657be586089987a@mail.gmail.com>
	<439FD21B.5010306@livinglogic.de>
Message-ID: <bb8868b90512140839l5d3d36b1s60950af363cd0914@mail.gmail.com>

Guido van Rossum wrote:
> On 12/13/05, Walter D?rwald <walter at livinglogic.de> wrote:
> > Having to define classes that conform to a certain API and registering
> > instances of those classes as callbacks with the parser doesn't look
> > that pythonic to me. An iterator API seems much more pythonic.
>
> Perhaps. Although the SAX API lets you leave a callback undefined if
> you don't have a need to handle those events; that's a bit trickier to
> do with an iterator.

Well, suppose you want to dump the text of a document.

    for e in iterparse(filename):
        if e.isText():
            out.write(e.data)

Not tricky.

> > Also the different callbacks have different signatures.

True.  With SAX I always have to look up the signatures.  The iterator
yields Node-like objects in document order.  I don't have to remember
signatures.

But the biggest advantage of an iterator-based API would be: when you
hit an element, you can easily pass control to a function that knows
how to parse that particular element.  parsePlay() can call
parseAct(), which can call parseScene().  To do anything like that
with SAX, you have to write a bunch of dispatch code.

-j

From wl at flexis.de  Wed Dec 14 17:34:18 2005
From: wl at flexis.de (Wolfgang)
Date: Wed, 14 Dec 2005 17:34:18 +0100
Subject: [Python-Dev] PEP 8 updates/clarifications, function/method style
In-Reply-To: <Pine.LNX.4.62.0512141605320.32164@qnzvnan.rov.np.hx>
References: <dnpcok$pqi$1@sea.gmane.org>
	<Pine.LNX.4.62.0512141605320.32164@qnzvnan.rov.np.hx>
Message-ID: <dnphia$c59$1@sea.gmane.org>

Hi,

Michael Hoffman wrote:
> [Wolfgang]
> 
>> Or should we switch to camelCase with lowercase first letter ?
>> As most other Languages prefer this (Java, C#, C++, ...)
> 
> They also use curly braces instead of indentation to indicate block
> structure. Maybe we should switch to that too. <wink>

Or BEGIN, END style. :-)

bye by Wolfgang


From jeremy at alum.mit.edu  Wed Dec 14 17:43:22 2005
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed, 14 Dec 2005 11:43:22 -0500
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <439FFFF4.4090005@egenix.com>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>
	<439E1D36.1090009@v.loewis.de>
	<17310.21156.907292.278567@montanaro.dyndns.org>
	<ca471dc20512122052k4b985d98m9ce1662858f2d1bc@mail.gmail.com>
	<439E9AD1.9080103@livinglogic.de> <439EABFB.3020405@gmail.com>
	<dnmc1q$km$1@sea.gmane.org> <439EBBEA.9030705@egenix.com>
	<dnmiaj$l8l$1@sea.gmane.org> <439FFFF4.4090005@egenix.com>
Message-ID: <e8bf7a530512140843j521f2be6k9fa4a3130439383@mail.gmail.com>

On 12/14/05, M.-A. Lemburg <mal at egenix.com> wrote:
> > we also need to figure out how to import the bundled version; should it be
> > cElementTree, xml.etree.cElementTree, or just xml.etree.ElementTree
> > (which would then fallback on the Python version if cElementTree isn't
> > built) ?
>
> If the semantics are identical I'd prefer the latter approach
> of using the faster variant if possible.

That is my preference, too.

Jeremy

From fredrik at pythonware.com  Wed Dec 14 17:51:09 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 14 Dec 2005 17:51:09 +0100
Subject: [Python-Dev] Sharing expat instances
References: <17304.33755.693941.811233@montanaro.dyndns.org><43989A99.9000402@v.loewis.de>
	<dnhl9k$bt6$1@sea.gmane.org><439C7341.4050300@v.loewis.de>
	<dni8qe$5eq$1@sea.gmane.org><439D1B6D.9080208@v.loewis.de>
	<dnmsft$t1g$1@sea.gmane.org> <dnnh9n$c5j$1@sea.gmane.org>
	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
Message-ID: <dnpihv$gfn$1@sea.gmane.org>

Phillip J. Eby wrote.

> >my current idea is to
> >
> >     1.  include it under a different name (_elementtree.so)
> >
> >     2.  add a cElementTree.py under xml.etree, which simply does
> >
> >             from _elementtree import *
> >
> >does anyone have a better idea ?
>
> I was under the impression that simply installing cElementTree.so in the
> relevant package directory would work; this is what the distutils do for
> extensions with a package name.

it would work, of course, but the core puts all the binaries in a separate
directory (lib-dynload on unix, DLLs on windows, etc).

do we really want to put executables in other locations ?

</F>




From steve at holdenweb.com  Wed Dec 14 18:04:28 2005
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 14 Dec 2005 17:04:28 +0000
Subject: [Python-Dev] Website cruft
In-Reply-To: <200512141037.12200.fdrake@acm.org>
References: <200512140912.35909.mmclay@comcast.net>
	<dnpcpv$pri$1@sea.gmane.org> <200512141037.12200.fdrake@acm.org>
Message-ID: <43A0509C.90201@holdenweb.com>

Fred L. Drake, Jr. wrote:
> On Wednesday 14 December 2005 10:12, Steve Holden wrote:
>  > However, this change highlights the fact that the checkins mailing-list
>  > description at
>  >
>  >    http://mail.python.org/mailman/listinfo/python-checkins
>  >
>  > is so out of date it warns that the CVS repository on cvs.python.org
>  > should no longer be used as we have now migrated to CVS on Sourceforge :-(
> 
> Try it now.
> 
Much better!

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/

From mcherm at mcherm.com  Wed Dec 14 18:08:34 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed, 14 Dec 2005 09:08:34 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications, function/method	style
Message-ID: <20051214090834.23uxarpx9dwgwkwg@login.werra.lunarpages.com>

Wolfgang writes:
> We need a clear style for function and method names
> now std lib uses "foo_bar" sometimes "foobar"
> and sometimes "fooBar".

Personally, I prefer "fooBar". But I try not to use it in python
code... I try to always use "foo_bar" because that's what PEP 8 says.

I believe recall the conversation that preceded putting that into
PEP 8, and the impetus for that particular decision was input from
non-native English speakers (particularly those not coming from a
European alphabet) that it was far easier to read with underscore
separation rather than capWords.

-- Michael Chermside



From fredrik at pythonware.com  Wed Dec 14 18:13:26 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 14 Dec 2005 18:13:26 +0100
Subject: [Python-Dev] "xml" package in standard library
References: <20051213063118.4eivt285x1f4sks0@login.werra.lunarpages.com><200512131554.00880.fdrake@acm.org><20051213234012.GA19680@rogue.amk.ca>
	<200512140200.18421.fdrake@acm.org>
Message-ID: <dnpjro$lq6$1@sea.gmane.org>

Fred L. Drake, Jr. wrote:

> On Tuesday 13 December 2005 18:40, A.M. Kuchling wrote:
>  > +1; it's what should have been done in the first place.
>
> If only I'd understood that when I added the xml/PyXML hack to the stdlib
> years ago.  :-(
>
> Fixed now.  I'll deal with the documentation in a few days; I actually expect
> to have some time.

looks like you broken installation (the libinstall target hasn't been updated);
from comp.lang.python:

    gregory at home:~$ python
    Python 2.5a0 (#1, Dec 14 2005, 14:11:55)
    [GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import xml.etree.ElementTree as ET
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/local/lib/python2.5/xml.py", line 20, in <module>
        import xmlcore
    ImportError: No module named xmlcore
    >>>

(works under trunk, but not after installation)

you also broke my subversion:

$ svn up
svn: subversion/libsvn_ra_svn/marshal.c:434: vwrite_tuple: Assertion `opt || cstr' failed.
Aborted

(but that's probably not your fault).

has anyone seen this error?  any ideas on how to fix it?

</F>




From fdrake at acm.org  Wed Dec 14 18:41:26 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 14 Dec 2005 12:41:26 -0500
Subject: [Python-Dev] "xml" package in standard library
In-Reply-To: <dnpjro$lq6$1@sea.gmane.org>
References: <20051213063118.4eivt285x1f4sks0@login.werra.lunarpages.com>
	<200512140200.18421.fdrake@acm.org> <dnpjro$lq6$1@sea.gmane.org>
Message-ID: <200512141241.26354.fdrake@acm.org>

On Wednesday 14 December 2005 12:13, Fredrik Lundh wrote:
 > looks like you broken installation (the libinstall target hasn't been
 > updated); from comp.lang.python:

Ouch!  Thanks for fixing this.

 > you also broke my subversion:
 >
 > $ svn up
 > svn: subversion/libsvn_ra_svn/marshal.c:434: vwrite_tuple: Assertion `opt
 > || cstr' failed. Aborted

I've never seen anything like that from Subversion before.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From ianb at colorstudy.com  Wed Dec 14 18:39:33 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 14 Dec 2005 11:39:33 -0600
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <439FFFF4.4090005@egenix.com>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>	<d11dcfba0512121653j2d27e62ek72d06b45c934d374@mail.gmail.com>	<439E1D36.1090009@v.loewis.de>	<17310.21156.907292.278567@montanaro.dyndns.org>	<ca471dc20512122052k4b985d98m9ce1662858f2d1bc@mail.gmail.com><439E9AD1.9080103@livinglogic.de>	<439EABFB.3020405@gmail.com><dnmc1q$km$1@sea.gmane.org>	<439EBBEA.9030705@egenix.com>	<dnmiaj$l8l$1@sea.gmane.org>
	<439FFFF4.4090005@egenix.com>
Message-ID: <43A058D5.6010405@colorstudy.com>

M.-A. Lemburg wrote:
>>we also need to figure out how to import the bundled version; should it be
>>cElementTree, xml.etree.cElementTree, or just xml.etree.ElementTree
>>(which would then fallback on the Python version if cElementTree isn't
>>built) ?
> 
> 
> If the semantics are identical I'd prefer the latter approach
> of using the faster variant if possible.

I have myself in the past used or overridden non-public methods of 
ElementTree, which I'm sure wouldn't work with cElementTree.  While I'd 
also prefer automatic fallback, it would be nice if there was 
additionally an explicit path to each version.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From fredrik at pythonware.com  Wed Dec 14 18:35:25 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 14 Dec 2005 18:35:25 +0100
Subject: [Python-Dev] "xml" package in standard library
References: <20051213063118.4eivt285x1f4sks0@login.werra.lunarpages.com><200512131554.00880.fdrake@acm.org><20051213234012.GA19680@rogue.amk.ca><200512140200.18421.fdrake@acm.org>
	<dnpjro$lq6$1@sea.gmane.org>
Message-ID: <dnpl4u$r22$1@sea.gmane.org>


> you also broke my subversion:
>
> $ svn up
> svn: subversion/libsvn_ra_svn/marshal.c:434: vwrite_tuple: Assertion `opt || cstr' failed.
> Aborted
>
> (but that's probably not your fault).
>
> has anyone seen this error?  any ideas on how to fix it?

google indicated that people had seen this when directories were removed
from the repository, so I removed Lib/xml by hand, and that fixed it.

</F>




From fdrake at acm.org  Wed Dec 14 18:46:22 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 14 Dec 2005 12:46:22 -0500
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <43A058D5.6010405@colorstudy.com>
References: <200512130001.jBD01bXZ007657@chilled.skew.org>
	<439FFFF4.4090005@egenix.com> <43A058D5.6010405@colorstudy.com>
Message-ID: <200512141246.23162.fdrake@acm.org>

On Wednesday 14 December 2005 12:39, Ian Bicking wrote:
 > I have myself in the past used or overridden non-public methods of
 > ElementTree, which I'm sure wouldn't work with cElementTree.  While I'd
 > also prefer automatic fallback, it would be nice if there was
 > additionally an explicit path to each version.

I think the whole PyXML v. the standard library dabacle has taught us that 
there should *always* be an explicit path to each version of a module or 
package.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From Scott.Daniels at Acm.Org  Wed Dec 14 18:56:50 2005
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Wed, 14 Dec 2005 09:56:50 -0800
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnpihv$gfn$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org><43989A99.9000402@v.loewis.de>	<dnhl9k$bt6$1@sea.gmane.org><439C7341.4050300@v.loewis.de>	<dni8qe$5eq$1@sea.gmane.org><439D1B6D.9080208@v.loewis.de>	<dnmsft$t1g$1@sea.gmane.org>
	<dnnh9n$c5j$1@sea.gmane.org>	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
	<dnpihv$gfn$1@sea.gmane.org>
Message-ID: <dnpmcl$vn2$1@sea.gmane.org>

Fredrik Lundh wrote:
> Phillip J. Eby wrote.
> 
>>> my current idea is to
>>>
>>>     1.  include it under a different name (_elementtree.so)
>>>     2.  add a cElementTree.py under xml.etree, which simply does
>>>             from _elementtree import *
>>>
>>> does anyone have a better idea ?
>> I was under the impression that simply installing cElementTree.so in the
>> relevant package directory would work; this is what the distutils do for
>> extensions with a package name.
> 
> it would work, of course, but the core puts all the binaries in a separate
> directory (lib-dynload on unix, DLLs on windows, etc).

One good reason for this is that the .pyd's or .so's cannot necessarily
be used from zip files, so it is nice to keep those binaries out of the
package directory (packages can then easily go in a Python25.zip).
My (admittedly weak) understanding of how packages work is that all
parts of a package should lie off the same node of the PYTHONPATH.

--Scott David Daniels
Scott.Daniels at Acm.Org


From fredrik at pythonware.com  Wed Dec 14 18:47:07 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 14 Dec 2005 18:47:07 +0100
Subject: [Python-Dev] ElementTree in stdlib
References: <200512130001.jBD01bXZ007657@chilled.skew.org><439E1D36.1090009@v.loewis.de><17310.21156.907292.278567@montanaro.dyndns.org><ca471dc20512122052k4b985d98m9ce1662858f2d1bc@mail.gmail.com><439E9AD1.9080103@livinglogic.de>
	<439EABFB.3020405@gmail.com><dnmc1q$km$1@sea.gmane.org>
	<439EBBEA.9030705@egenix.com><dnmiaj$l8l$1@sea.gmane.org>
	<439FFFF4.4090005@egenix.com>
	<e8bf7a530512140843j521f2be6k9fa4a3130439383@mail.gmail.com>
Message-ID: <dnplqs$tm5$1@sea.gmane.org>

Jeremy Hylton wrote:

> On 12/14/05, M.-A. Lemburg <mal at egenix.com> wrote:
> > > we also need to figure out how to import the bundled version; should it be
> > > cElementTree, xml.etree.cElementTree, or just xml.etree.ElementTree
> > > (which would then fallback on the Python version if cElementTree isn't
> > > built) ?
> >
> > If the semantics are identical I'd prefer the latter approach
> > of using the faster variant if possible.
>
> That is my preference, too.

it's cStringIO vs. StringIO and cPickle vs. pickle situation again; the
modules are 99% compatible, but there's always someone that relies
on that last % (which is a result of ET being written in Python).

at this point, I think it's more important to guarantee that changing
"elementtree" to "xml.etree" will always work under Python 2.5 [1],
than to have a new set of potential subtle incompatibility issues.  but
I have changed my mind before...

</F>

1) except for users that need a newer version, of course.




From barry at python.org  Wed Dec 14 19:15:49 2005
From: barry at python.org (Barry Warsaw)
Date: Wed, 14 Dec 2005 13:15:49 -0500
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnpmcl$vn2$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de>	<dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de>	<dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de>	<dnmsft$t1g$1@sea.gmane.org>
	<dnnh9n$c5j$1@sea.gmane.org>
	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
	<dnpihv$gfn$1@sea.gmane.org>  <dnpmcl$vn2$1@sea.gmane.org>
Message-ID: <1134584149.10260.8.camel@geddy.wooz.org>

On Wed, 2005-12-14 at 09:56 -0800, Scott David Daniels wrote:

> One good reason for this is that the .pyd's or .so's cannot necessarily
> be used from zip files

When you say "cannot necessarily", are the situations where they can be
imported from zip files?  I thought the answer to that was always "no".

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051214/f34aa03d/attachment.pgp

From guido at python.org  Wed Dec 14 19:17:45 2005
From: guido at python.org (Guido van Rossum)
Date: Wed, 14 Dec 2005 10:17:45 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications, function/method style
In-Reply-To: <dnpcok$pqi$1@sea.gmane.org>
References: <dnpcok$pqi$1@sea.gmane.org>
Message-ID: <ca471dc20512141017g4d0fedadyc51a81e3768febec@mail.gmail.com>

On 12/14/05, Wolfgang <wl at flexis.de> wrote:
> Hello,
>
> PEP 8 for function and method names:
> -----
> Function Names
>
>        Function names should be lowercase, possibly with words separated by
>        underscores to improve readability.  mixedCase is allowed only in
>        contexts where that's already the prevailing style (e.g. threading.py),
>        to retain backwards compatibility.
>
> Method Names and Instance Variables
>
>        The story is largely the same as with functions: in general, use
>        lowercase with words separated by underscores as necessary to improve
>        readability.
> -----
>
> We need a clear style for function and method names
> now std lib uses "foo_bar" sometimes "foobar"
> and sometimes "fooBar".

Actually this (function, method and ivar names) is such a contentious
issue that I think the style guide should explicitly allow all
two/three styles and recommend to be consistent within a class, module
or package.

I want to be adamant about module, package and class names though:
module/package names should be short all-lowercase without
underscores; class names should be CapWords. I'd like to fix this for
StringIO.py, UserDict.py etc.

> The use of lowercase with underscores is the default so
> the python std lib should use it in all modules.

Too late. I don't think the diversity is all that distracting. I'd be
much more concerned about spelling MethodNames (as unfortunately the
Google style guide requires).

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From Scott.Daniels at Acm.Org  Wed Dec 14 19:38:19 2005
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Wed, 14 Dec 2005 10:38:19 -0800
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <1134584149.10260.8.camel@geddy.wooz.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de>	<dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de>	<dni8qe$5eq$1@sea.gmane.org>	<439D1B6D.9080208@v.loewis.de>	<dnmsft$t1g$1@sea.gmane.org>	<dnnh9n$c5j$1@sea.gmane.org>	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>	<dnpihv$gfn$1@sea.gmane.org>
	<dnpmcl$vn2$1@sea.gmane.org>
	<1134584149.10260.8.camel@geddy.wooz.org>
Message-ID: <dnpoqd$7ri$1@sea.gmane.org>

Barry Warsaw wrote:
> On Wed, 2005-12-14 at 09:56 -0800, Scott David Daniels wrote:
>> One good reason for this is that the .pyd's or .so's cannot necessarily
>> be used from zip files
> When you say "cannot necessarily", are the situations where they can be
> imported from zip files?  I thought the answer to that was always "no".

I thought so too, but was not sure enough to state it that way.

--Scott David Daniels
Scott.Daniels at Acm.Org


From martin at v.loewis.de  Wed Dec 14 20:12:33 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 14 Dec 2005 20:12:33 +0100
Subject: [Python-Dev] "xml" package in standard library
In-Reply-To: <200512131554.00880.fdrake@acm.org>
References: <20051213063118.4eivt285x1f4sks0@login.werra.lunarpages.com>	<200512131518.38810.fdrake@acm.org>
	<dnnau7$ldg$1@sea.gmane.org> <200512131554.00880.fdrake@acm.org>
Message-ID: <43A06EA1.90908@v.loewis.de>

Fred L. Drake, Jr. wrote:
> - Pickles containing classes from the xml package will break if we're not
>   really careful.  But I think they're pretty fragile now.
> 
> I'll be glad to make these or similar changes if there's concensus on this.

I don't agree with the change. You just broke source compatibility
between the core package and PyXML.

Regards,
Martin

From fredrik at pythonware.com  Wed Dec 14 20:21:27 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 14 Dec 2005 20:21:27 +0100
Subject: [Python-Dev] Sharing expat instances
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de>	<dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de>	<dni8qe$5eq$1@sea.gmane.org>	<439D1B6D.9080208@v.loewis.de>	<dnmsft$t1g$1@sea.gmane.org>	<dnnh9n$c5j$1@sea.gmane.org>	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>	<dnpihv$gfn$1@sea.gmane.org><dnpmcl$vn2$1@sea.gmane.org><1134584149.10260.8.camel@geddy.wooz.org>
	<dnpoqd$7ri$1@sea.gmane.org>
Message-ID: <dnprbq$i5v$1@sea.gmane.org>

Scott David Daniels wrote:

> > > One good reason for this is that the .pyd's or .so's cannot necessarily
> > > be used from zip files
> >
> > When you say "cannot necessarily", are the situations where they can be
> > imported from zip files?  I thought the answer to that was always "no".
>
> I thought so too, but was not sure enough to state it that way.

you could of course add them to the zip file, and automagically extract
them before you start importing things.

</F>




From blais at furius.ca  Wed Dec 14 20:07:00 2005
From: blais at furius.ca (Martin Blais)
Date: Wed, 14 Dec 2005 14:07:00 -0500
Subject: [Python-Dev] Linked lists
Message-ID: <8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>

Hello again.

As I'm digging deeper into LISP and Scheme these days, I was
wondering, is there a good compelling reason why in Python we don't
have a native singly-linked and doubly-linked list types?

That is, reasons other than
- "you can get by without it" (sometimes I *want* lists), or
- "you can use tuples to emulate lists" (only in limited contexts, and
it's not pretty), or
- "you can code your own in Python" (not as efficient as native types)

Maybe I'm slapping an old horse here, but searching the archives I
could not find some relevant controversy, that would--I hoped--include
a final word from the BDFL.

If I had "real" lists I would use them more often, where I now use
Python lists (vectors, really).  (Although In LISP I have the reverse
problem, I tend to use lists sometimes where I should use arrays--I
think I need to grow up).

Anyway, why not provide good lists or cons-cell classes in C?

From barry at python.org  Wed Dec 14 20:34:53 2005
From: barry at python.org (Barry Warsaw)
Date: Wed, 14 Dec 2005 14:34:53 -0500
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnprbq$i5v$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de>	<dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de>	<dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de>	<dnmsft$t1g$1@sea.gmane.org>
	<dnnh9n$c5j$1@sea.gmane.org>
	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
	<dnpihv$gfn$1@sea.gmane.org><dnpmcl$vn2$1@sea.gmane.org>
	<1134584149.10260.8.camel@geddy.wooz.org> <dnpoqd$7ri$1@sea.gmane.org>
	<dnprbq$i5v$1@sea.gmane.org>
Message-ID: <1134588893.10291.33.camel@geddy.wooz.org>

On Wed, 2005-12-14 at 20:21 +0100, Fredrik Lundh wrote:

> you could of course add them to the zip file, and automagically extract
> them before you start importing things.

Sure (although we don't).  I wonder if this is useful functionality for
the core.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051214/4614fdd3/attachment-0001.pgp

From mcherm at mcherm.com  Wed Dec 14 20:35:49 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed, 14 Dec 2005 11:35:49 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications, function/method	style
Message-ID: <20051214113549.swbgg3zat8g0wo4k@login.werra.lunarpages.com>

Guido writes:
> Actually this (function, method and ivar names) is such a contentious
> issue that I think the style guide should explicitly allow all
> two/three styles and recommend to be consistent within a class, module
> or package.

Hurray! Now I can go back to using capWords for functions, methods, and
variables!

Guido writes:
> Too late. I don't think the diversity is all that distracting.

I disagree. One of the things that Java got very much right was to
specify, from the very beginning, what the preferred conventions are
for naming conventions. (Packages in lowercase, Classes in CapWords,
methods and variables in lowerCapWords, constants optionally in
ALL_CAPS. Abbrevs avoided, acronyms have all letters capitalized, eg:
SimpleHTTPServer.)

The conventions are nearly universally followed, and as a result in
java I always know how to spell things. I never have to remember
whether it's myDict.hasKey() and myDict.popitem() or myDict.has_key()
and myDict.popItem(). Haskell goes too far -- they REQUIRE a certain
convention as part of the language... this prevents breaking the rules
on purpose (eg: wrapping a library from another language, or using
an object with attributes to represent an XML node with child nodes).

Of course, a really good IDE might make this irrelevent by popping up
the correct option as I type... but the IDEs I use for Python don't
even know the types of my variables, so they don't do this for me.

Anyhow... I don't think we can change it now, but I _do_ believe that
the lack of such conventions is a bit distracting. (Not enough to
make me prefer Java, of course!)

-- Michael Chermside


From pje at telecommunity.com  Wed Dec 14 20:44:18 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 14 Dec 2005 14:44:18 -0500
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnprbq$i5v$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de> <dnmsft$t1g$1@sea.gmane.org>
	<dnnh9n$c5j$1@sea.gmane.org>
	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
	<dnpihv$gfn$1@sea.gmane.org> <dnpmcl$vn2$1@sea.gmane.org>
	<1134584149.10260.8.camel@geddy.wooz.org>
	<dnpoqd$7ri$1@sea.gmane.org>
Message-ID: <5.1.1.6.0.20051214143931.03c30268@mail.telecommunity.com>

At 08:21 PM 12/14/2005 +0100, Fredrik Lundh wrote:
>Scott David Daniels wrote:
>
> > > > One good reason for this is that the .pyd's or .so's cannot necessarily
> > > > be used from zip files
> > >
> > > When you say "cannot necessarily", are the situations where they can be
> > > imported from zip files?  I thought the answer to that was always "no".
> >
> > I thought so too, but was not sure enough to state it that way.
>
>you could of course add them to the zip file, and automagically extract
>them before you start importing things.

The runtime system for Python Eggs does this; extraction is to a 
PYTHON_EGG_CACHE directory.  See e.g.:

    http://peak.telecommunity.com/DevCenter/PkgResources#resource-extraction

The egg builder writes a .py file alongside the .pyd or .so in the .egg 
file that contains something like:

    def __bootstrap__():
       global __bootstrap__, __loader__, __file__
       import sys, pkg_resources, imp
       __file__ = pkg_resources.resource_filename(__name__,'foobar.so')
       del __bootstrap__, __loader__
       imp.load_dynamic(__name__,__file__)

    __bootstrap__()

So, when you import from the zipfile, the .py file gets loaded (since 
zipimport doesn't support .pyd/.so/etc. imports directly) and then it 
reloads the module from the extracted file.  The other magic there is just 
to keep any of the special names from staying behind in the reloaded module.


From aahz at pythoncraft.com  Wed Dec 14 20:44:21 2005
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 14 Dec 2005 11:44:21 -0800
Subject: [Python-Dev] Linked lists
In-Reply-To: <8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>
References: <8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>
Message-ID: <20051214194421.GA1464@panix.com>

On Wed, Dec 14, 2005, Martin Blais wrote:
> 
> As I'm digging deeper into LISP and Scheme these days, I was
> wondering, is there a good compelling reason why in Python we don't
> have a native singly-linked and doubly-linked list types?

How about taking this dicussion to comp.lang.python and providing more
examples of use cases.  I think you should probably be able to throw
together a PEP for a library without too much trouble, but it'll take a
lot of convincing to make the push for a builtin type.  As with sets and
Decimal, plan on starting with a pure-Python module; finding one already
in the wild that people use would be even better.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Don't listen to schmucks on USENET when making legal decisions.  Hire
yourself a competent schmuck."  --USENET schmuck (aka Robert Kern)

From pje at telecommunity.com  Wed Dec 14 20:47:13 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 14 Dec 2005 14:47:13 -0500
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnpmcl$vn2$1@sea.gmane.org>
References: <dnpihv$gfn$1@sea.gmane.org>
	<17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de> <dnmsft$t1g$1@sea.gmane.org>
	<dnnh9n$c5j$1@sea.gmane.org>
	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
	<dnpihv$gfn$1@sea.gmane.org>
Message-ID: <5.1.1.6.0.20051214144502.03c87ea8@mail.telecommunity.com>

At 09:56 AM 12/14/2005 -0800, Scott David Daniels wrote:
>My (admittedly weak) understanding of how packages work is that all
>parts of a package should lie off the same node of the PYTHONPATH.

This isn't a requirement; packages have a __path__ attribute which can 
include more than one directory.  The 'pkgutil' module (added in Python 
2.3) even has a convenient way to merge parts of a package that are in 
separate directories.


From mcherm at mcherm.com  Wed Dec 14 20:46:08 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed, 14 Dec 2005 11:46:08 -0800
Subject: [Python-Dev] ElementTree in stdlib
Message-ID: <20051214114608.hzg6mvjwpjk0wcs0@login.werra.lunarpages.com>

/F writes:
> it's cStringIO vs. StringIO and cPickle vs. pickle situation again; the
> modules are 99% compatible, but there's always someone that relies
> on that last % (which is a result of ET being written in Python).

Yes!

> at this point, I think it's more important to guarantee that changing
> "elementtree" to "xml.etree" will always work under Python 2.5 [1],
> than to have a new set of potential subtle incompatibility issues.  but
> I have changed my mind before...

Consider changing it again. I fear that if ElementTree is part of the
core without cElementTree, then a meme will spread which says (and
PLEASE don't quote this!) "ElementTree has a great API, but it's
just too slow for real work."

We already know that Python is particularly susceptable to "too slow"
memes, even invalid ones. I think the best all-around solution is to
include cElementTree and use it wherever possible unless the user
specially imports the pure-python version. Perhaps importing
"xml.etree" gets you cElementTree unless that isn't compiled on your
platform, but you can import "xml.pure_python.etree" or something
like that to get the pure Python version if you really want it.

-- Michael Chermside


From pje at telecommunity.com  Wed Dec 14 20:52:04 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 14 Dec 2005 14:52:04 -0500
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnpihv$gfn$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de> <dnmsft$t1g$1@sea.gmane.org>
	<dnnh9n$c5j$1@sea.gmane.org>
	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051214144853.03b0fff8@mail.telecommunity.com>

At 05:51 PM 12/14/2005 +0100, Fredrik Lundh wrote:
>Phillip J. Eby wrote.
>
> > >my current idea is to
> > >
> > >     1.  include it under a different name (_elementtree.so)
> > >
> > >     2.  add a cElementTree.py under xml.etree, which simply does
> > >
> > >             from _elementtree import *
> > >
> > >does anyone have a better idea ?
> >
> > I was under the impression that simply installing cElementTree.so in the
> > relevant package directory would work; this is what the distutils do for
> > extensions with a package name.
>
>it would work, of course, but the core puts all the binaries in a separate
>directory (lib-dynload on unix, DLLs on windows, etc).
>
>do we really want to put executables in other locations ?

I don't know.  I can see that the split makes sense for prefix/exec-prefix 
distinctions, but then again, the disutils will install an entire 
distribution in exec-prefix if it contains "impure" parts, so that's 
certainly an option here.

On the other hand, it's not clear to me *why* the lib-dynload/DLLs 
directories exist, since it seems to me that that's what exec-prefix is 
for.  Perhaps somebody can explain why lib-dynload/ and DLLs/ 
exist?  Perhaps some platforms have to add these directories to some 
godforsaken environment variables like LD_LIBRARY_PATH or something?


From barry at python.org  Wed Dec 14 20:59:36 2005
From: barry at python.org (Barry Warsaw)
Date: Wed, 14 Dec 2005 14:59:36 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439EBDF1.80209@zope.com>
References: <4399F967.3080300@colorstudy.com>  <439C51C8.2010909@zope.com>
	<1134404788.950.24.camel@geddy.wooz.org>  <439EBDF1.80209@zope.com>
Message-ID: <1134590376.10291.38.camel@geddy.wooz.org>

On Tue, 2005-12-13 at 07:26 -0500, Jim Fulton wrote:

> I'd add somewhere: "If in doubt, chose non-public. You can always change your
> mind later."

Added.

> 
> >       We don't use the term "private" here, since no attribute is really
> >       private in Python (without a generally unnecessary amount of work).
> >       However, another category of attribute are those which, while not being
> >       public, are intended for use by subclasses (often called "protected" in
> >       other languages).  Some classes are designed to be inherited from,
> >       either to extend or modify aspects of the class's behavior.  When
> >       designing such a class, take care to make explicit decisions about which
> >       attributes are public, which are non-public but useful for subclasses, and
> >       which are truly only to be used by your base class.
> 
> A useful term might be "subclass API".  Decide which non-public attributes
> are part of the subclass API.

Excellent suggestion, thanks.  I like the term "subclass API".  Added.

> >       - For simple public data attributes, it is fine to expose just the
> >         attribute name, without complicated accessor/mutator methods.  Keep in
> >         mind that Python provides an easy path to future enhancement, should
> >         you find that a simple data attribute needs to grow functional
> >         behavior.  In that case, use properties to hide functional
> >         implementation behind simple data attribute access syntax.
> > 
> >         Note 1: Properties only work on new-style classes.
> > 
> >         Note 2: Try to keep the functional behavior side-effect free, although
> >         side-effects such as caching are generally fine.
> 
> Personally, I'd actively discourage use of trivial accessors.  Simple
> attribute access is not only "fine", IMO, but it is much better than
> trivial accessors.  This is an important point, IMO, because, in my
> experience, the vast majority of accessors *are* trivial.

I've changed "fine" to "best".

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051214/9f782435/attachment.pgp

From barry at python.org  Wed Dec 14 21:05:50 2005
From: barry at python.org (Barry Warsaw)
Date: Wed, 14 Dec 2005 15:05:50 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <5.1.1.6.0.20051213115321.01f79e58@mail.telecommunity.com>
References: <439EDF41.6080101@zope.com> <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com> <439EB711.5030005@zope.com>
	<Pine.LNX.4.62.0512131255090.26997@qnzvnan.rov.np.hx>
	<439EDF41.6080101@zope.com>
	<5.1.1.6.0.20051213115321.01f79e58@mail.telecommunity.com>
Message-ID: <1134590751.10291.41.camel@geddy.wooz.org>

On Tue, 2005-12-13 at 12:00 -0500, Phillip J. Eby wrote:

> In any case, the algorithms involved are near-trivial; the most complex 
> piece is the processing of complex version specifications like 
> "CherryPy>=2.1.0,!=2.1.1-rc2,<2.2a" into a series of version intervals.
> 
> The only outstanding feature request for the dependency resolution 
> algorithm is supporting optional or replaceable dependencies such as "we 
> need either scipy *or* Numeric".

You might look at Gentoo's portage system and its package dependencies
for a model here.  I think they have a notion of "virtual" packages,
such as "mta" or "X" which various specific packages such as sendmail,
postfix, X11 or xorg can fulfill.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051214/1a13766c/attachment.pgp

From barry at python.org  Wed Dec 14 21:07:57 2005
From: barry at python.org (Barry Warsaw)
Date: Wed, 14 Dec 2005 15:07:57 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications, function/method	style
In-Reply-To: <20051214090834.23uxarpx9dwgwkwg@login.werra.lunarpages.com>
References: <20051214090834.23uxarpx9dwgwkwg@login.werra.lunarpages.com>
Message-ID: <1134590877.10260.43.camel@geddy.wooz.org>

On Wed, 2005-12-14 at 09:08 -0800, Michael Chermside wrote:
> Wolfgang writes:
> > We need a clear style for function and method names
> > now std lib uses "foo_bar" sometimes "foobar"
> > and sometimes "fooBar".
> 
> Personally, I prefer "fooBar". But I try not to use it in python
> code... I try to always use "foo_bar" because that's what PEP 8 says.
> 
> I believe recall the conversation that preceded putting that into
> PEP 8, and the impetus for that particular decision was input from
> non-native English speakers (particularly those not coming from a
> European alphabet) that it was far easier to read with underscore
> separation rather than capWords.

Correct, which is one reason I feel strongly that we should keep the
current recommendation of lower_case_words.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051214/b3f7dce2/attachment-0001.pgp

From barry at python.org  Wed Dec 14 21:13:24 2005
From: barry at python.org (Barry Warsaw)
Date: Wed, 14 Dec 2005 15:13:24 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications, function/method	style
In-Reply-To: <ca471dc20512141017g4d0fedadyc51a81e3768febec@mail.gmail.com>
References: <dnpcok$pqi$1@sea.gmane.org>
	<ca471dc20512141017g4d0fedadyc51a81e3768febec@mail.gmail.com>
Message-ID: <1134591204.10261.49.camel@geddy.wooz.org>

On Wed, 2005-12-14 at 10:17 -0800, Guido van Rossum wrote:

> Actually this (function, method and ivar names) is such a contentious
> issue that I think the style guide should explicitly allow all
> two/three styles and recommend to be consistent within a class, module
> or package.

My own feeling is that the PEP should keep its current recommendation of
lower_case_words.  It allows for the other styles under the "be
internally consistent" guideline.

My current rewrite (to be checked in soon), has this also to say:

      mixedCase is allowed only in contexts where that's already the
      prevailing style (e.g. threading.py), to retain backwards compatibility.

It could be argued that this guideline should be relaxed to give
lower_case_words and mixedCase equal footing.  I'd disagree, but will
make that change to the PEP if there's strong consensus in favor of
taking that position.

> I want to be adamant about module, package and class names though:
> module/package names should be short all-lowercase without
> underscores; class names should be CapWords. I'd like to fix this for
> StringIO.py, UserDict.py etc.

Absolutely.

> > The use of lowercase with underscores is the default so
> > the python std lib should use it in all modules.
> 
> Too late. I don't think the diversity is all that distracting. I'd be
> much more concerned about spelling MethodNames (as unfortunately the
> Google style guide requires).

That's right out. :)

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051214/f05c2db3/attachment.pgp

From martin at v.loewis.de  Wed Dec 14 21:19:29 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 14 Dec 2005 21:19:29 +0100
Subject: [Python-Dev] Linked lists
In-Reply-To: <8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>
References: <8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>
Message-ID: <43A07E51.3030900@v.loewis.de>

Martin Blais wrote:
> As I'm digging deeper into LISP and Scheme these days, I was
> wondering, is there a good compelling reason why in Python we don't
> have a native singly-linked and doubly-linked list types?

As you seem to be asking for the historical reason: because nobody
ever wanted it so badly to do anything about it.

I would personally consider it a waste of energy to implement such
a thing with so little use, given the alternatives.

Regards,
Martin

From guido at python.org  Wed Dec 14 21:37:31 2005
From: guido at python.org (Guido van Rossum)
Date: Wed, 14 Dec 2005 12:37:31 -0800
Subject: [Python-Dev] Linked lists
In-Reply-To: <8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>
References: <8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>
Message-ID: <ca471dc20512141237i4f09098cudb951fbe70623085@mail.gmail.com>

On 12/14/05, Martin Blais <blais at furius.ca> wrote:
> Hello again.
>
> As I'm digging deeper into LISP and Scheme these days, I was
> wondering, is there a good compelling reason why in Python we don't
> have a native singly-linked and doubly-linked list types?
>
> That is, reasons other than
> - "you can get by without it" (sometimes I *want* lists), or
> - "you can use tuples to emulate lists" (only in limited contexts, and
> it's not pretty), or
> - "you can code your own in Python" (not as efficient as native types)
>
> Maybe I'm slapping an old horse here, but searching the archives I
> could not find some relevant controversy, that would--I hoped--include
> a final word from the BDFL.
>
> If I had "real" lists I would use them more often, where I now use
> Python lists (vectors, really).  (Although In LISP I have the reverse
> problem, I tend to use lists sometimes where I should use arrays--I
> think I need to grow up).
>
> Anyway, why not provide good lists or cons-cell classes in C?

So here's a word from the BDFL. :)

I'm curious about the use cases you have in mind.

Python's philosophy about (built-in) data types, inherited from ABC,
is to offer a few powerful clearly distinct choices rather than lots
of alternatives with overlapping usages. This reduces the time it
takes to choose a data type and reduces the risk of picking the wrong
type. (You seem to be indicating that this is indeed what's happening
to you in Lisp. :-)

In this philosophy, there is a real cost associated with offering more
data types which isn't necessarily offset by the advantage.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From bcannon at gmail.com  Wed Dec 14 22:16:08 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Wed, 14 Dec 2005 13:16:08 -0800
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <20051214114608.hzg6mvjwpjk0wcs0@login.werra.lunarpages.com>
References: <20051214114608.hzg6mvjwpjk0wcs0@login.werra.lunarpages.com>
Message-ID: <bbaeab100512141316i575ce4b6r17f6326991e9f046@mail.gmail.com>

On 12/14/05, Michael Chermside <mcherm at mcherm.com> wrote:
> /F writes:
> > it's cStringIO vs. StringIO and cPickle vs. pickle situation again; the
> > modules are 99% compatible, but there's always someone that relies
> > on that last % (which is a result of ET being written in Python).
>
> Yes!
>
> > at this point, I think it's more important to guarantee that changing
> > "elementtree" to "xml.etree" will always work under Python 2.5 [1],
> > than to have a new set of potential subtle incompatibility issues.  but
> > I have changed my mind before...
>
> Consider changing it again. I fear that if ElementTree is part of the
> core without cElementTree, then a meme will spread which says (and
> PLEASE don't quote this!) "ElementTree has a great API, but it's
> just too slow for real work."
>
> We already know that Python is particularly susceptable to "too slow"
> memes, even invalid ones. I think the best all-around solution is to
> include cElementTree and use it wherever possible unless the user
> specially imports the pure-python version. Perhaps importing
> "xml.etree" gets you cElementTree unless that isn't compiled on your
> platform, but you can import "xml.pure_python.etree" or something
> like that to get the pure Python version if you really want it.
>

I don't think this will necessarily happen.  You are assuming people
are going to know there is a faster way than ET written in Python.  I
think most people consider stuff in the stdlib good and fast enough
for most uses and when they want faster they roll their own.

And since I have always voted on the side of "have a C version only if
someone wants to maintain a C version but don't have both C and
Python", I say /F should include which ever he wants, but I personally
vote for only one version.  So if /F is going to continue to maintain
cElementTree and since it is already written I say use that and just
get the speed boost and eliminate the isssue of people relying on that
1% semantic difference between the Python and C version.

-Brett

From barry at python.org  Wed Dec 14 22:18:28 2005
From: barry at python.org (Barry Warsaw)
Date: Wed, 14 Dec 2005 16:18:28 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <439E1137.4010403@colorstudy.com>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
Message-ID: <1134595108.10260.55.camel@geddy.wooz.org>

I've pushed out a revised PEP 8

http://www.python.org/peps/pep-0008.html

Please review and comment.  Thanks everyone for providing an excellent
discussion.  Hopefully I have captured our current collective
recommendations.  I've also tried to simplify the text, while making it
(somewhat) more prescriptive.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051214/a1d888e9/attachment.pgp

From bcannon at gmail.com  Wed Dec 14 22:25:35 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Wed, 14 Dec 2005 13:25:35 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications, function/method style
In-Reply-To: <20051214113549.swbgg3zat8g0wo4k@login.werra.lunarpages.com>
References: <20051214113549.swbgg3zat8g0wo4k@login.werra.lunarpages.com>
Message-ID: <bbaeab100512141325h7440e0dane9e8cd34cc670a59@mail.gmail.com>

On 12/14/05, Michael Chermside <mcherm at mcherm.com> wrote:
> Guido writes:
> > Actually this (function, method and ivar names) is such a contentious
> > issue that I think the style guide should explicitly allow all
> > two/three styles and recommend to be consistent within a class, module
> > or package.
>
> Hurray! Now I can go back to using capWords for functions, methods, and
> variables!
>
> Guido writes:
> > Too late. I don't think the diversity is all that distracting.
>
> I disagree. One of the things that Java got very much right was to
> specify, from the very beginning, what the preferred conventions are
> for naming conventions. (Packages in lowercase, Classes in CapWords,
> methods and variables in lowerCapWords, constants optionally in
> ALL_CAPS. Abbrevs avoided, acronyms have all letters capitalized, eg:
> SimpleHTTPServer.)
>
> The conventions are nearly universally followed, and as a result in
> java I always know how to spell things. I never have to remember
> whether it's myDict.hasKey() and myDict.popitem() or myDict.has_key()
> and myDict.popItem(). Haskell goes too far -- they REQUIRE a certain
> convention as part of the language... this prevents breaking the rules
> on purpose (eg: wrapping a library from another language, or using
> an object with attributes to represent an XML node with child nodes).

I agree completely with this.  I might remember the name of a method,
but I don't always remember the capping and the possible use of
underscores.  Consistency would be really nice.  I am not saying we
should rename everything (at least not until Python 3  =), but at
least we can make sure new stuff that is not preexisting can use a
consistent naming scheme.

And as for it being contentious, I say Guido can pronounce on this. 
We are all grown-ups and can learn to name things in a certain way to
give our memories an easier time.  =)

-Brett

From Scott.Daniels at Acm.Org  Wed Dec 14 22:28:04 2005
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Wed, 14 Dec 2005 13:28:04 -0800
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <5.1.1.6.0.20051214144853.03b0fff8@mail.telecommunity.com>
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de>
	<dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de>
	<dni8qe$5eq$1@sea.gmane.org>	<439D1B6D.9080208@v.loewis.de>
	<dnmsft$t1g$1@sea.gmane.org>	<dnnh9n$c5j$1@sea.gmane.org>	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
	<dnpihv$gfn$1@sea.gmane.org>
	<5.1.1.6.0.20051214144853.03b0fff8@mail.telecommunity.com>
Message-ID: <dnq2on$drt$1@sea.gmane.org>

Phillip J. Eby wrote:
> At 05:51 PM 12/14/2005 +0100, Fredrik Lundh wrote:
>> Phillip J. Eby wrote.
>>
>>>> my current idea is to
>>>>
>>>>     1.  include it under a different name (_elementtree.so)
>>>>
>>>>     2.  add a cElementTree.py under xml.etree, which simply does
>>>>
>>>>             from _elementtree import *
>>>>
>>>> does anyone have a better idea ?
>>> I was under the impression that simply installing cElementTree.so in the
>>> relevant package directory would work; this is what the distutils do for
>>> extensions with a package name.
>> it would work, of course, but the core puts all the binaries in a separate
>> directory (lib-dynload on unix, DLLs on windows, etc).
>>
>> do we really want to put executables in other locations ?
> 
> I don't know.  I can see that the split makes sense for prefix/exec-prefix 
> distinctions, but then again, the disutils will install an entire 
> distribution in exec-prefix if it contains "impure" parts, so that's 
> certainly an option here.
> 
> On the other hand, it's not clear to me *why* the lib-dynload/DLLs 
> directories exist, since it seems to me that that's what exec-prefix is 
> for.  Perhaps somebody can explain why lib-dynload/ and DLLs/ 
> exist?  Perhaps some platforms have to add these directories to some 
> godforsaken environment variables like LD_LIBRARY_PATH or something?

What I believe I understand about /.pyd / .so / .dll / shared libraries
is that they are meant to allow several processes to map the same disk
backing store to the same same virtual address for more than a single
process.  If the .egg strategy is followed, I expect that either the
file shared is in a user(or even process)-specific location or there
is a shared folder that is writable by many processes from which
executable code can be run.  The one solution reduces sharing, the
other violates security principles.

--Scott David Daniels
Scott.Daniels at Acm.Org


From pje at telecommunity.com  Wed Dec 14 22:36:45 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 14 Dec 2005 16:36:45 -0500
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <bbaeab100512141316i575ce4b6r17f6326991e9f046@mail.gmail.co
 m>
References: <20051214114608.hzg6mvjwpjk0wcs0@login.werra.lunarpages.com>
	<20051214114608.hzg6mvjwpjk0wcs0@login.werra.lunarpages.com>
Message-ID: <5.1.1.6.0.20051214163333.01de4da0@mail.telecommunity.com>

At 01:16 PM 12/14/2005 -0800, Brett Cannon wrote:
>On 12/14/05, Michael Chermside <mcherm at mcherm.com> wrote:
> > We already know that Python is particularly susceptable to "too slow"
> > memes, even invalid ones. I think the best all-around solution is to
> > include cElementTree and use it wherever possible unless the user
> > specially imports the pure-python version. Perhaps importing
> > "xml.etree" gets you cElementTree unless that isn't compiled on your
> > platform, but you can import "xml.pure_python.etree" or something
> > like that to get the pure Python version if you really want it.
>
>I don't think this will necessarily happen.  You are assuming people
>are going to know there is a faster way than ET written in Python.

Actually, he's said that the C version should be the default, with the 
Python version only used if you have subclassing needs that can't be met by 
the C version.


>And since I have always voted on the side of "have a C version only if
>someone wants to maintain a C version but don't have both C and
>Python", I say /F should include which ever he wants, but I personally
>vote for only one version.  So if /F is going to continue to maintain
>cElementTree and since it is already written I say use that and just
>get the speed boost and eliminate the isssue of people relying on that
>1% semantic difference between the Python and C version.

Having a Python version available for Jython, PyPy, etc., is a good idea; 
Michael's proposal lets us have your cake (C version be the default) and 
eat it too (have the pure Python available for other platforms and for 
explicit use by subclassers.


From Scott.Daniels at Acm.Org  Wed Dec 14 22:31:08 2005
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Wed, 14 Dec 2005 13:31:08 -0800
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <20051214114608.hzg6mvjwpjk0wcs0@login.werra.lunarpages.com>
References: <20051214114608.hzg6mvjwpjk0wcs0@login.werra.lunarpages.com>
Message-ID: <dnq2uf$drt$2@sea.gmane.org>

Michael Chermside wrote:
> ... a meme will spread which says (and PLEASE don't quote this!)
> "ElementTree has a great API, but it's just too slow for real work."
+1 DNQOTW :-)   (Do Not Quote Of The Week)

--Scott David Daniels
Scott.Daniels at Acm.Org


From bcannon at gmail.com  Wed Dec 14 22:48:35 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Wed, 14 Dec 2005 13:48:35 -0800
Subject: [Python-Dev] ElementTree in stdlib
In-Reply-To: <5.1.1.6.0.20051214163333.01de4da0@mail.telecommunity.com>
References: <20051214114608.hzg6mvjwpjk0wcs0@login.werra.lunarpages.com>
	<5.1.1.6.0.20051214163333.01de4da0@mail.telecommunity.com>
Message-ID: <bbaeab100512141348h62bf32dep6ff8b4b4a999c783@mail.gmail.com>

On 12/14/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 01:16 PM 12/14/2005 -0800, Brett Cannon wrote:
> >On 12/14/05, Michael Chermside <mcherm at mcherm.com> wrote:
> > > We already know that Python is particularly susceptable to "too slow"
> > > memes, even invalid ones. I think the best all-around solution is to
> > > include cElementTree and use it wherever possible unless the user
> > > specially imports the pure-python version. Perhaps importing
> > > "xml.etree" gets you cElementTree unless that isn't compiled on your
> > > platform, but you can import "xml.pure_python.etree" or something
> > > like that to get the pure Python version if you really want it.
> >
> >I don't think this will necessarily happen.  You are assuming people
> >are going to know there is a faster way than ET written in Python.
>
> Actually, he's said that the C version should be the default, with the
> Python version only used if you have subclassing needs that can't be met by
> the C version.
>

Ah, misread it.

>
> >And since I have always voted on the side of "have a C version only if
> >someone wants to maintain a C version but don't have both C and
> >Python", I say /F should include which ever he wants, but I personally
> >vote for only one version.  So if /F is going to continue to maintain
> >cElementTree and since it is already written I say use that and just
> >get the speed boost and eliminate the isssue of people relying on that
> >1% semantic difference between the Python and C version.
>
> Having a Python version available for Jython, PyPy, etc., is a good idea;
> Michael's proposal lets us have your cake (C version be the default) and
> eat it too (have the pure Python available for other platforms and for
> explicit use by subclassers.
>

Good point.  My preference then would be to not directly expose it but
have it there for the other distributions to use with an added note to
make sure to not use anyt edge semantics that might crop up from the
different versions since they might be using the Python version.

-Brett

From t-meyer at ihug.co.nz  Wed Dec 14 22:50:53 2005
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Thu, 15 Dec 2005 10:50:53 +1300
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134595108.10260.55.camel@geddy.wooz.org>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
Message-ID: <3EEA75BC-77E7-452F-81E2-857FAE892EE4@ihug.co.nz>

> I've pushed out a revised PEP 8
>
> http://www.python.org/peps/pep-0008.html
>
> Please review and comment.

Why does PEP 8 continually refer to one particular editor (Emacs)?   
(There are even parts in the form "x is better because it works  
better in Emacs", when surely it's actually the case that x is better  
because it's better in the majority of editors.)

ISTM that the PEP would be much simpler if it was completely editor- 
agnostic, but perhaps there's some historical reasoning I'm not aware  
of?

=Tony.Meyer

From guido at python.org  Wed Dec 14 23:10:48 2005
From: guido at python.org (Guido van Rossum)
Date: Wed, 14 Dec 2005 14:10:48 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134595108.10260.55.camel@geddy.wooz.org>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
Message-ID: <ca471dc20512141410qb84ba99ve275fb95ab8ba8dc@mail.gmail.com>

On 12/14/05, Barry Warsaw <barry at python.org> wrote:
> I've pushed out a revised PEP 8
>
> http://www.python.org/peps/pep-0008.html
>
> Please review and comment.  Thanks everyone for providing an excellent
> discussion.  Hopefully I have captured our current collective
> recommendations.  I've also tried to simplify the text, while making it
> (somewhat) more prescriptive.

Thanks, Barry! I've made another pass and added a couple more tweaks,
hopefully non-controversial.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Dec 14 23:14:39 2005
From: guido at python.org (Guido van Rossum)
Date: Wed, 14 Dec 2005 14:14:39 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <3EEA75BC-77E7-452F-81E2-857FAE892EE4@ihug.co.nz>
References: <4399F967.3080300@colorstudy.com> <439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<3EEA75BC-77E7-452F-81E2-857FAE892EE4@ihug.co.nz>
Message-ID: <ca471dc20512141414j19fac098gd77125d75278534a@mail.gmail.com>

On 12/14/05, Tony Meyer <t-meyer at ihug.co.nz> wrote:
> > I've pushed out a revised PEP 8
> >
> > http://www.python.org/peps/pep-0008.html
> >
> > Please review and comment.
>
> Why does PEP 8 continually refer to one particular editor (Emacs)?
> (There are even parts in the form "x is better because it works
> better in Emacs", when surely it's actually the case that x is better
> because it's better in the majority of editors.)
>
> ISTM that the PEP would be much simpler if it was completely editor-
> agnostic, but perhaps there's some historical reasoning I'm not aware
> of?

Historically many Python developers use Emacs. Barry & I still do.

I think the best way to avoid editor wars is to pick one editor and
stick with it. :-)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Wed Dec 14 23:56:48 2005
From: barry at python.org (Barry Warsaw)
Date: Wed, 14 Dec 2005 17:56:48 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <ca471dc20512141410qb84ba99ve275fb95ab8ba8dc@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<ca471dc20512141410qb84ba99ve275fb95ab8ba8dc@mail.gmail.com>
Message-ID: <1134601008.11465.11.camel@geddy.wooz.org>

On Wed, 2005-12-14 at 14:10 -0800, Guido van Rossum wrote:

> Thanks, Barry! I've made another pass and added a couple more tweaks,
> hopefully non-controversial.

Cool thanks Guido.  I fixed a couple of small typos.

One question: you made the suggestion that a blank line separate the
last line in a tqs docstring from its closing tqs.  I'm wondering why
that is.  Note that python-mode now (or shall we say with the fix I just
made ;) properly fills paragraphs in those strings, even if there is no
blank line before the closing tqs line.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051214/9918d145/attachment.pgp

From barry at python.org  Wed Dec 14 23:58:29 2005
From: barry at python.org (Barry Warsaw)
Date: Wed, 14 Dec 2005 17:58:29 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <ca471dc20512141414j19fac098gd77125d75278534a@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<3EEA75BC-77E7-452F-81E2-857FAE892EE4@ihug.co.nz>
	<ca471dc20512141414j19fac098gd77125d75278534a@mail.gmail.com>
Message-ID: <1134601109.11466.13.camel@geddy.wooz.org>

On Wed, 2005-12-14 at 14:14 -0800, Guido van Rossum wrote:

> Historically many Python developers use Emacs. Barry & I still do.
> 
> I think the best way to avoid editor wars is to pick one editor and
> stick with it. :-)

Dinosaurs rule! :)

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051214/0336c9b2/attachment.pgp

From lambacck at computer.org  Thu Dec 15 00:18:57 2005
From: lambacck at computer.org (Chris Lambacher)
Date: Wed, 14 Dec 2005 18:18:57 -0500
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <1134584149.10260.8.camel@geddy.wooz.org>
References: <dnhl9k$bt6$1@sea.gmane.org> <439C7341.4050300@v.loewis.de>
	<dni8qe$5eq$1@sea.gmane.org> <439D1B6D.9080208@v.loewis.de>
	<dnmsft$t1g$1@sea.gmane.org> <dnnh9n$c5j$1@sea.gmane.org>
	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
	<dnpihv$gfn$1@sea.gmane.org> <dnpmcl$vn2$1@sea.gmane.org>
	<1134584149.10260.8.camel@geddy.wooz.org>
Message-ID: <20051214231857.GA29865@kateandchris.net>

Py2exe manages to load .pyd files and dlls from zip.  Apparently they have
written an alternate dll loader that does not need the file to be on the file
system.  This is used for single file apps.

I don't know if it is possible to write a portable Unix equivalent for .so
files.

-Chris

P.S.  I think it is standard practice to do an introduction on first post to
pydev.  I have been lurking on the Dev list for about a year.  I recently
completed my masters thesis which used Python extensively.  I am helping to
promote the adoption of Python for various tasks at my place of work. 


On Wed, Dec 14, 2005 at 01:15:49PM -0500, Barry Warsaw wrote:
> On Wed, 2005-12-14 at 09:56 -0800, Scott David Daniels wrote:
> 
> > One good reason for this is that the .pyd's or .so's cannot necessarily
> > be used from zip files
> 
> When you say "cannot necessarily", are the situations where they can be
> imported from zip files?  I thought the answer to that was always "no".
> 
> -Barry
> 



> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/lambacck%40computer.org


From djc at object-craft.com.au  Thu Dec 15 01:13:23 2005
From: djc at object-craft.com.au (Dave Cole)
Date: Thu, 15 Dec 2005 11:13:23 +1100
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134595108.10260.55.camel@geddy.wooz.org>
References: <4399F967.3080300@colorstudy.com>	<1134168683.19370.29.camel@geddy.wooz.org>	<439A110E.8090605@colorstudy.com>	<1134411382.5676.35.camel@geddy.wooz.org>	<439DDCD2.10803@colorstudy.com>	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
Message-ID: <43A0B523.5040406@object-craft.com.au>

Barry Warsaw wrote:
> I've pushed out a revised PEP 8
> 
> http://www.python.org/peps/pep-0008.html
> 
> Please review and comment.  Thanks everyone for providing an excellent
> discussion.  Hopefully I have captured our current collective
> recommendations.  I've also tried to simplify the text, while making it
> (somewhat) more prescriptive.

The only thing I strongly disagree with is the promotion of javaNaming
to equal footing with python_naming.  We can look forward to future
programs that look like this:

class Something:

  def __init__(self, i_care):
     self.mem_a = pkga.uses_python_naming()
     self.mem_b = pkgb.usesJavaNaming()
     if i_care:
        self.this_looks(self.mem_b.reallyCrap())

It gets worse if you subclass from modules using different conventions
in your own code.  Ugly code.

- Dave

-- 
http://www.object-craft.com.au

From pinard at iro.umontreal.ca  Thu Dec 15 02:00:58 2005
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Wed, 14 Dec 2005 20:00:58 -0500
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <d11dcfba0512130813y24f4908bu139821e8fc3cdb9b@mail.gmail.com>
References: <439CA89B.4030600@colorstudy.com> <439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<439CE06D.70600@gmail.com>
	<ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com>
	<d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com>
	<ca471dc20512121111i2596a04dlbe2e151780829ec3@mail.gmail.com>
	<d11dcfba0512121333o4e17b735mcb5f91df4387d380@mail.gmail.com>
	<20051213145313.GA23676@alcyon.progiciels-bpi.ca>
	<d11dcfba0512130813y24f4908bu139821e8fc3cdb9b@mail.gmail.com>
Message-ID: <20051215010058.GA30219@alcyon.progiciels-bpi.ca>

[Steven Bethard]

>import ellogon.utils as utils
>import ellogon.features.relations as features_relations
>import ellogon.chunking as chunking
>import ml.classifiers as _ml_classifiers
>import ml.data as _ml_data

>The only two-letter one was ElementTree, and the vast majority were
>unabbreviated, though  as you can see,  some of them drop one of the
>items in the import chain.  Do you find imports like the above
>problematic?

No, all of the above seems very clear.  What creates us problems is the 
flurry of two-letter abbreviations for imported modules.

>FWIW, I don't like importing objects from modules directly for the
>same reason that when I write Java now, I always use an explicit
>"this" for instance variables.  When I see a name which isn't local to
>a function, I want to have some idea where it came from...

Reasonable, and clearer.  Yet, in some cases, the reuse of a few
prefixes all over the code may develop being more polluting than 
helping.  Good taste may be better than too inflexible principles.  One
may search the source with an editor and quickly find the proper import.  
_Given_, of course, it is not of the form "from HELL import *". :-)

But this gets a bit far from the original topic.  Let's drop the matter 
here as not being much more of general interest, or else, let's switch 
to private email.

                                       Keep happy, all.

-- 
Fran?ois Pinard   http://pinard.progiciels-bpi.ca

From guido at python.org  Thu Dec 15 02:04:20 2005
From: guido at python.org (Guido van Rossum)
Date: Wed, 14 Dec 2005 17:04:20 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134601008.11465.11.camel@geddy.wooz.org>
References: <4399F967.3080300@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<ca471dc20512141410qb84ba99ve275fb95ab8ba8dc@mail.gmail.com>
	<1134601008.11465.11.camel@geddy.wooz.org>
Message-ID: <ca471dc20512141704x4d562fa5wad14cc2d19376243@mail.gmail.com>

On 12/14/05, Barry Warsaw <barry at python.org> wrote:
> On Wed, 2005-12-14 at 14:10 -0800, Guido van Rossum wrote:
>
> > Thanks, Barry! I've made another pass and added a couple more tweaks,
> > hopefully non-controversial.
>
> Cool thanks Guido.  I fixed a couple of small typos.
>
> One question: you made the suggestion that a blank line separate the
> last line in a tqs docstring from its closing tqs.  I'm wondering why
> that is.  Note that python-mode now (or shall we say with the fix I just
> made ;) properly fills paragraphs in those strings, even if there is no
> blank line before the closing tqs line.

Primarily because I didn't have that fix when I wrote it, and I think
many people don't have it. And did you fix it in IDLE too?

I can make up another reason too, but this is the main one. (I think
it also visually separates the doc string paragraphs from the code
better.)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Dec 15 02:31:54 2005
From: guido at python.org (Guido van Rossum)
Date: Wed, 14 Dec 2005 17:31:54 -0800
Subject: [Python-Dev] Add timeout to subprocess.py?
Message-ID: <ca471dc20512141731n5104b9b9sa707552aa91bc194@mail.gmail.com>

A frequent requirement seems to be to run a subprocess but give up if
it takes more than N seconds. I suppose you can hack this using the
poll() method and time.sleep(), but perhaps a more direct approach can
be added to the subprocess module? This should directly support
reading output / stderr until the time limit is up as well. Any
takers?

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From aleaxit at gmail.com  Thu Dec 15 02:31:37 2005
From: aleaxit at gmail.com (Alex Martelli)
Date: Wed, 14 Dec 2005 17:31:37 -0800
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <20051214231857.GA29865@kateandchris.net>
References: <dnhl9k$bt6$1@sea.gmane.org> <dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de> <dnmsft$t1g$1@sea.gmane.org>
	<dnnh9n$c5j$1@sea.gmane.org>
	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
	<dnpihv$gfn$1@sea.gmane.org> <dnpmcl$vn2$1@sea.gmane.org>
	<1134584149.10260.8.camel@geddy.wooz.org>
	<20051214231857.GA29865@kateandchris.net>
Message-ID: <e8a0972d0512141731r76d79760ncaddfe7d63a5fe32@mail.gmail.com>

On 12/14/05, Chris Lambacher <lambacck at computer.org> wrote:
> Py2exe manages to load .pyd files and dlls from zip.  Apparently they have
> written an alternate dll loader that does not need the file to be on the file
> system.  This is used for single file apps.
>
> I don't know if it is possible to write a portable Unix equivalent for .so
> files.

If you mean, portable to all dialects of Unix and Unix-like systems, I
am pretty confident in answering "no".  But it might be possible to
support many such systems, e.g. by exploiting the DMG format ("disk
image" within a single file), native to MacOSX, that can be used in
Linux (and I believe in some other Unix variants) by mounting the file
as a "loop device" (there's all sort of niggling issues, such as the
need to be root to do so, but perhaps there might be workarounds...).


Alex

From pinard at iro.umontreal.ca  Thu Dec 15 02:16:59 2005
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Wed, 14 Dec 2005 20:16:59 -0500
Subject: [Python-Dev] Deprecate __ private (was Re: PEP 8
	updates/clarifications)
In-Reply-To: <5.1.1.6.0.20051213114517.01f73c48@mail.telecommunity.com>
References: <439C51C8.2010909@zope.com> <439CA89B.4030600@colorstudy.com>
	<439CAEF6.4000304@zope.com>
	<ee2a432c0512111752w734a165amcbbc92f1d49e93c2@mail.gmail.com>
	<439CE06D.70600@gmail.com>
	<ca471dc20512111958y317fd18cx68c6289e43db361f@mail.gmail.com>
	<d11dcfba0512112053l3dc718acu94f610ca3d3ff4d1@mail.gmail.com>
	<ca471dc20512121111i2596a04dlbe2e151780829ec3@mail.gmail.com>
	<d11dcfba0512121333o4e17b735mcb5f91df4387d380@mail.gmail.com>
	<5.1.1.6.0.20051213114517.01f73c48@mail.telecommunity.com>
Message-ID: <20051215011659.GA30618@alcyon.progiciels-bpi.ca>

[Phillip J. Eby]
>At 09:53 AM 12/13/2005 -0500, Fran?ois Pinard wrote:
>>Everybody here agrees that this style makes the code much less legible.

>I hope you mean, "here at your company or organization", as I disagree.  :)

Yes, of course!  Sorry for the ambiguity.

-- 
Fran?ois Pinard   http://pinard.progiciels-bpi.ca

From ianb at colorstudy.com  Thu Dec 15 03:09:37 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 14 Dec 2005 20:09:37 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <43A0B523.5040406@object-craft.com.au>
References: <4399F967.3080300@colorstudy.com>	<1134168683.19370.29.camel@geddy.wooz.org>	<439A110E.8090605@colorstudy.com>	<1134411382.5676.35.camel@geddy.wooz.org>	<439DDCD2.10803@colorstudy.com>	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>	<439E1137.4010403@colorstudy.com>	<1134595108.10260.55.camel@geddy.wooz.org>
	<43A0B523.5040406@object-craft.com.au>
Message-ID: <43A0D061.5000506@colorstudy.com>

Dave Cole wrote:
> Barry Warsaw wrote:
> 
>>I've pushed out a revised PEP 8
>>
>>http://www.python.org/peps/pep-0008.html
>>
>>Please review and comment.  Thanks everyone for providing an excellent
>>discussion.  Hopefully I have captured our current collective
>>recommendations.  I've also tried to simplify the text, while making it
>>(somewhat) more prescriptive.
> 
> 
> The only thing I strongly disagree with is the promotion of javaNaming
> to equal footing with python_naming.  We can look forward to future
> programs that look like this:

I don't see this suggestion...?  Here's what seems to be the current 
text on the matter:

     Function Names

       Function names should be lowercase, with words separated by 
underscores
       as necessary to improve readability.

       mixedCase is allowed only in contexts where that's already the
       prevailing style (e.g. threading.py), to retain backwards 
compatibility.

     Method Names and Instance Variables

       Use the function naming rules: lowercase with words separated by
       underscores as necessary to improve readability.

I know Guido in a later thread seems to be going back on that, which 
would be disappointing to me.  I think it's reasonable to loosen the 
phrasing a bit -- it's nearly always better to stay consistent with a 
package than follow PEP 8 on this point.  The current text merely 
implies that it's acceptable to follow previous convention, not preferable.

Many people who come to Python *want* to follow whatever the prevalent 
and preferred style is, but it's not immediately clear what that is.  So 
I think PEP 8 should as often as possible at least indicate preference. 
  Which it pretty much does now, so we should leave well enough alone ;)


-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org

From bob at redivi.com  Thu Dec 15 02:50:04 2005
From: bob at redivi.com (Bob Ippolito)
Date: Wed, 14 Dec 2005 17:50:04 -0800
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <e8a0972d0512141731r76d79760ncaddfe7d63a5fe32@mail.gmail.com>
References: <dnhl9k$bt6$1@sea.gmane.org> <dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de> <dnmsft$t1g$1@sea.gmane.org>
	<dnnh9n$c5j$1@sea.gmane.org>
	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
	<dnpihv$gfn$1@sea.gmane.org> <dnpmcl$vn2$1@sea.gmane.org>
	<1134584149.10260.8.camel@geddy.wooz.org>
	<20051214231857.GA29865@kateandchris.net>
	<e8a0972d0512141731r76d79760ncaddfe7d63a5fe32@mail.gmail.com>
Message-ID: <84B90EB6-904E-4C15-91BD-78BD516B8331@redivi.com>


On Dec 14, 2005, at 5:31 PM, Alex Martelli wrote:

> On 12/14/05, Chris Lambacher <lambacck at computer.org> wrote:
>> Py2exe manages to load .pyd files and dlls from zip.  Apparently  
>> they have
>> written an alternate dll loader that does not need the file to be  
>> on the file
>> system.  This is used for single file apps.
>>
>> I don't know if it is possible to write a portable Unix equivalent  
>> for .so
>> files.
>
> If you mean, portable to all dialects of Unix and Unix-like systems, I
> am pretty confident in answering "no".  But it might be possible to
> support many such systems, e.g. by exploiting the DMG format ("disk
> image" within a single file), native to MacOSX, that can be used in
> Linux (and I believe in some other Unix variants) by mounting the file
> as a "loop device" (there's all sort of niggling issues, such as the
> need to be root to do so, but perhaps there might be workarounds...).

For Mac OS X 10.4+, you don't even have to bother.  The dynamic  
linker finally implements the function that lets you load a code  
bundle straight from RAM:
extern NSObjectFileImageReturnCode NSCreateObjectFileImageFromMemory(
     const void *address,
     size_t size,
     NSObjectFileImage *objectFileImage);

Of course, you could always just use temporary files or a cache  
somewhere.

-bob


From steve at holdenweb.com  Thu Dec 15 03:19:47 2005
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 15 Dec 2005 02:19:47 +0000
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnq2on$drt$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de>	<dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de>	<dni8qe$5eq$1@sea.gmane.org>	<439D1B6D.9080208@v.loewis.de>	<dnmsft$t1g$1@sea.gmane.org>	<dnnh9n$c5j$1@sea.gmane.org>	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>	<dnpihv$gfn$1@sea.gmane.org>	<5.1.1.6.0.20051214144853.03b0fff8@mail.telecommunity.com>
	<dnq2on$drt$1@sea.gmane.org>
Message-ID: <dnqjsa$ud5$1@sea.gmane.org>

Scott David Daniels wrote:
> Phillip J. Eby wrote:
> 
>>At 05:51 PM 12/14/2005 +0100, Fredrik Lundh wrote:
>>
>>>Phillip J. Eby wrote.
>>>
>>>
>>>>>my current idea is to
>>>>>
>>>>>    1.  include it under a different name (_elementtree.so)
>>>>>
>>>>>    2.  add a cElementTree.py under xml.etree, which simply does
>>>>>
>>>>>            from _elementtree import *
>>>>>
>>>>>does anyone have a better idea ?
>>>>
>>>>I was under the impression that simply installing cElementTree.so in the
>>>>relevant package directory would work; this is what the distutils do for
>>>>extensions with a package name.
>>>
>>>it would work, of course, but the core puts all the binaries in a separate
>>>directory (lib-dynload on unix, DLLs on windows, etc).
>>>
>>>do we really want to put executables in other locations ?
>>
>>I don't know.  I can see that the split makes sense for prefix/exec-prefix 
>>distinctions, but then again, the disutils will install an entire 
>>distribution in exec-prefix if it contains "impure" parts, so that's 
>>certainly an option here.
>>
>>On the other hand, it's not clear to me *why* the lib-dynload/DLLs 
>>directories exist, since it seems to me that that's what exec-prefix is 
>>for.  Perhaps somebody can explain why lib-dynload/ and DLLs/ 
>>exist?  Perhaps some platforms have to add these directories to some 
>>godforsaken environment variables like LD_LIBRARY_PATH or something?
> 
> 
> What I believe I understand about /.pyd / .so / .dll / shared libraries
> is that they are meant to allow several processes to map the same disk
> backing store to the same same virtual address for more than a single
> process.  If the .egg strategy is followed, I expect that either the
> file shared is in a user(or even process)-specific location or there
> is a shared folder that is writable by many processes from which
> executable code can be run.  The one solution reduces sharing, the
> other violates security principles.
> 
When extension modules are to be run from zip files it's difficult to 
avoid a reduction in sharing anyway - without some kind of additional 
mechanism there's no guarantee that two modules with the same filename 
are actually the same library anyway, and there's every likelihood that 
over time several zipped applications will be released containing 
different versions of the same extension module. In that case each app 
would want the version that came with it anyway, no?

There are slightly different requirements for packaged distribution than 
for the standard interpreter, which I assume can always rely on having 
an installed module available as a file (though this might not be 
desirable in the future).

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From pinard at iro.umontreal.ca  Thu Dec 15 03:21:39 2005
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Wed, 14 Dec 2005 21:21:39 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134595108.10260.55.camel@geddy.wooz.org>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
Message-ID: <20051215022139.GB30618@alcyon.progiciels-bpi.ca>

[Barry Warsaw]

>I've pushed out a revised PEP 8
>http://www.python.org/peps/pep-0008.html
>Please review and comment.

Hi, Barry, and people.  Allow me a few remarks, nothing essential.


* Within "Naming Conventions", "Prescriptive: Naming Conventions", 
"Module names", we read:

    Modules should have short, lowercase names, without underscores.

I would like that PEP 0008 add some cement around this idea that common 
English words, properly spelled, which are likely to be user variable 
names, be avoided whenever reasonable.


* Within "Code lay-out", "Blank Lines", we read:

    Separate top-level function and class definitions with two blank lines.

In a running English text (like in this message!), it might make sense, 
but after tried for a long while both a single line and two lines 
between top-level elements in Python programs, I did not see that two 
lines gives so much more legibility to be worth a prescription.  One 
white line is enough in my opinion, and I wonder if "with two blank 
lines" could not be replaced by "at least one blank line".


* Within "Programming Recommendations", writers are suggested to prefer 
str methods of the string module, as well as a few others more recent 
additions (recent in the history of Python).  I wonder if this would not 
be a good place to suggest more of such.  These come to mind:

    - Replacing "apply(func, args)" with "func(*args)".
    - Replacing "for line in file.readlines():" with "for line in file:".
    - Replacing "for key in dict.keys():" with "for key in dict:".
    - Replacing "if has_key(dict, key):" with "if key in dict:".

and there might be others.  Deprecated functions could be listed, too.

-- 
Fran?ois Pinard   http://pinard.progiciels-bpi.ca

From ianb at colorstudy.com  Thu Dec 15 03:22:42 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 14 Dec 2005 20:22:42 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134595108.10260.55.camel@geddy.wooz.org>
References: <4399F967.3080300@colorstudy.com>	<1134168683.19370.29.camel@geddy.wooz.org>	<439A110E.8090605@colorstudy.com>	<1134411382.5676.35.camel@geddy.wooz.org>	<439DDCD2.10803@colorstudy.com>	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
Message-ID: <43A0D372.1020001@colorstudy.com>

Barry Warsaw wrote:
> I've pushed out a revised PEP 8
> 
> http://www.python.org/peps/pep-0008.html

Just two little things...


     Use 4 spaces per indentation level.

     This is the default for Emacs's python-mode.  For really old code 
that you
     don't want to mess up, you can continue to use 8-space tabs.  Emacs
     python-mode auto-detects the prevailing indentation level used in a 
file
     and sets its indentation parameters accordingly.


I think the reference to Emacs here is unneeded.  "Use 4 spaces per 
indentation level" is sufficient instruction on its own.




     ...Latin-1 should only be
     used when a comment or docstring needs to mention an author name that
     requires Latin-1; otherwise, using \x escapes is the preferred way to
     include non-ASCII data in string literals.  An exception is made 
for those
     files that are part of the test suite for the code implementing PEP 
263.


I think the reference to PEP 263 tests is kind of obvious.



-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org

From jcarlson at uci.edu  Thu Dec 15 03:07:52 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 14 Dec 2005 18:07:52 -0800
Subject: [Python-Dev] Add timeout to subprocess.py?
In-Reply-To: <ca471dc20512141731n5104b9b9sa707552aa91bc194@mail.gmail.com>
References: <ca471dc20512141731n5104b9b9sa707552aa91bc194@mail.gmail.com>
Message-ID: <20051214180301.6B99.JCARLSON@uci.edu>


I've got a recipe in the Python cookbook which adds async subprocess
support[1], which can be wrapped to support handling IO to/from the
subprocess until a time limit occurs.

 - Josiah

[1] - http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440554


Guido van Rossum <guido at python.org> wrote:
> 
> A frequent requirement seems to be to run a subprocess but give up if
> it takes more than N seconds. I suppose you can hack this using the
> poll() method and time.sleep(), but perhaps a more direct approach can
> be added to the subprocess module? This should directly support
> reading output / stderr until the time limit is up as well. Any
> takers?
> 
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jcarlson%40uci.edu


From guido at python.org  Thu Dec 15 03:42:25 2005
From: guido at python.org (Guido van Rossum)
Date: Wed, 14 Dec 2005 18:42:25 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <20051215022139.GB30618@alcyon.progiciels-bpi.ca>
References: <4399F967.3080300@colorstudy.com> <439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<20051215022139.GB30618@alcyon.progiciels-bpi.ca>
Message-ID: <ca471dc20512141842t5451ff6cj9cd19525005a42a@mail.gmail.com>

On 12/14/05, Fran?ois Pinard <pinard at iro.umontreal.ca> wrote:
> I would like that PEP 0008 add some cement around this idea that common
> English words, properly spelled, which are likely to be user variable
> names, be avoided whenever reasonable.

I don't think that's a reasonable rule. There are too many English
words and the rule as formulated is too vague. Also, module/package
names can only conflict with *global* user variable names.

> * Within "Code lay-out", "Blank Lines", we read:
>
>     Separate top-level function and class definitions with two blank lines.
>
> In a running English text (like in this message!), it might make sense,
> but after tried for a long while both a single line and two lines
> between top-level elements in Python programs, I did not see that two
> lines gives so much more legibility to be worth a prescription.  One
> white line is enough in my opinion, and I wonder if "with two blank
> lines" could not be replaced by "at least one blank line".

Here I agree, especially bewteen top-level functions. You *might* want
to separate classes with two blank lines if the methods within them
are separated by single blank lines, but even there it's probably
overkill.
>
> * Within "Programming Recommendations", writers are suggested to prefer
> str methods of the string module, as well as a few others more recent
> additions (recent in the history of Python).  I wonder if this would not
> be a good place to suggest more of such.  These come to mind:
>
>     - Replacing "apply(func, args)" with "func(*args)".
>     - Replacing "for line in file.readlines():" with "for line in file:".
>     - Replacing "for key in dict.keys():" with "for key in dict:".
>     - Replacing "if has_key(dict, key):" with "if key in dict:".
>
> and there might be others.

+; however see below.

> Deprecated functions could be listed, too.

I think that's more proper for a separate PEP -- the style guide
shouldn't have to be updated each time something else is deprecated.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From t-meyer at ihug.co.nz  Thu Dec 15 03:50:37 2005
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Thu, 15 Dec 2005 15:50:37 +1300
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <ca471dc20512141414j19fac098gd77125d75278534a@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<3EEA75BC-77E7-452F-81E2-857FAE892EE4@ihug.co.nz>
	<ca471dc20512141414j19fac098gd77125d75278534a@mail.gmail.com>
Message-ID: <9D0BE0F7-8283-4E87-96AF-32654CAAE632@ihug.co.nz>

[Barry]
>>> I've pushed out a revised PEP 8
>>>
>>> http://www.python.org/peps/pep-0008.html
>>>
>>> Please review and comment.

[Tony Meyer]
>> Why does PEP 8 continually refer to one particular editor (Emacs)?

[Guido]
> I think the best way to avoid editor wars is to pick one editor and
> stick with it. :-)

I wasn't suggesting an editor war, or replacing or adding to the  
Emacs references.  My point (echoed in part by Ian Bicking) was that  
the Emacs references don't seem to add anything worthwhile to the PEP  
(it's the style guide, not the "Python in Emacs" guide), and if the  
aim is to simplify the text then this would be another way to do that.

=Tony.Meyer


From pinard at iro.umontreal.ca  Thu Dec 15 04:09:45 2005
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Wed, 14 Dec 2005 22:09:45 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <ca471dc20512141842t5451ff6cj9cd19525005a42a@mail.gmail.com>
References: <439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<20051215022139.GB30618@alcyon.progiciels-bpi.ca>
	<ca471dc20512141842t5451ff6cj9cd19525005a42a@mail.gmail.com>
Message-ID: <20051215030945.GA382@alcyon.progiciels-bpi.ca>

[Guido van Rossum]
>On 12/14/05, Fran?ois Pinard <pinard at iro.umontreal.ca> wrote:

>> I would like that PEP 0008 add some cement around this idea that common
>> English words, properly spelled, which are likely to be user variable
>> names, be avoided whenever reasonable.

>I don't think that's a reasonable rule.  There are too many English
>words and the rule as formulated is too vague. Also, module/package
>names can only conflict with *global* user variable names.

It would be hard making a precise formulation for it indeed.  Yet, the 
danger exists and would be more easily avoided if stated in PEP 0008.  
A good exemple is the "textwrap" module, for which the name is very 
acceptable to me, while "text" (which was suggested here on python-dev) 
would have been a bit nightmarish in my own code: I used to do a lot of 
text processing, and I selected "text" as the common name for strings 
under consideration, exactly because "string" was already taken! ? :-)

I do not understand your statement that module/package names can only 
conflict with *global* user variable names.  Local variables hide global 
variables with same names, and imported modules are often global 
variables.  So, using a local name in a function prevents accessing 
a module by the same name (defined either locally or globally).

>>     - Replacing "apply(func, args)" with "func(*args)".
>>     - Replacing "for line in file.readlines():" with "for line in file:".
>>     - Replacing "for key in dict.keys():" with "for key in dict:".
>>     - Replacing "if has_key(dict, key):" with "if key in dict:".

>> Deprecated functions could be listed, too.

>I think that's more proper for a separate PEP -- the style guide
>shouldn't have to be updated each time something else is deprecated.

I was more thinking of things like xreadlines, having replacements 
suggested in the style guile already (given you accept the second 
"Replacing" above).  Not every single deprecated function, of course, 
but only the prominent ones.  On the other hand, xreadlines is not even 
prominent.  :-)

-- 
Fran?ois Pinard   http://pinard.progiciels-bpi.ca

From pje at telecommunity.com  Thu Dec 15 05:25:44 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 14 Dec 2005 23:25:44 -0500
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnqjsa$ud5$1@sea.gmane.org>
References: <dnq2on$drt$1@sea.gmane.org>
	<17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de> <dnmsft$t1g$1@sea.gmane.org>
	<dnnh9n$c5j$1@sea.gmane.org>
	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
	<dnpihv$gfn$1@sea.gmane.org>
	<5.1.1.6.0.20051214144853.03b0fff8@mail.telecommunity.com>
	<dnq2on$drt$1@sea.gmane.org>
Message-ID: <5.1.1.6.0.20051214232157.01de5678@mail.telecommunity.com>

At 02:19 AM 12/15/2005 +0000, Steve Holden wrote:
>Scott David Daniels wrote:
> > Phillip J. Eby wrote:
> >
> >>At 05:51 PM 12/14/2005 +0100, Fredrik Lundh wrote:
> >>
> >>>Phillip J. Eby wrote.
> >>>
> >>>
> >>>>>my current idea is to
> >>>>>
> >>>>>    1.  include it under a different name (_elementtree.so)
> >>>>>
> >>>>>    2.  add a cElementTree.py under xml.etree, which simply does
> >>>>>
> >>>>>            from _elementtree import *
> >>>>>
> >>>>>does anyone have a better idea ?
> >>>>
> >>>>I was under the impression that simply installing cElementTree.so in the
> >>>>relevant package directory would work; this is what the distutils do for
> >>>>extensions with a package name.
> >>>
> >>>it would work, of course, but the core puts all the binaries in a separate
> >>>directory (lib-dynload on unix, DLLs on windows, etc).
> >>>
> >>>do we really want to put executables in other locations ?
> >>
> >>I don't know.  I can see that the split makes sense for prefix/exec-prefix
> >>distinctions, but then again, the disutils will install an entire
> >>distribution in exec-prefix if it contains "impure" parts, so that's
> >>certainly an option here.
> >>
> >>On the other hand, it's not clear to me *why* the lib-dynload/DLLs
> >>directories exist, since it seems to me that that's what exec-prefix is
> >>for.  Perhaps somebody can explain why lib-dynload/ and DLLs/
> >>exist?  Perhaps some platforms have to add these directories to some
> >>godforsaken environment variables like LD_LIBRARY_PATH or something?
> >
> >
> > What I believe I understand about /.pyd / .so / .dll / shared libraries
> > is that they are meant to allow several processes to map the same disk
> > backing store to the same same virtual address for more than a single
> > process.  If the .egg strategy is followed, I expect that either the
> > file shared is in a user(or even process)-specific location or there
> > is a shared folder that is writable by many processes from which
> > executable code can be run.  The one solution reduces sharing, the
> > other violates security principles.

Actually, the discussion wasn't about eggs, but about the placement of C 
extensions in standard, "normal" package directories.  The Python 
installation uses only top-level C extensions and puts them in separate 
directories.  The question outstanding is, why?  Since there already exists 
exec-prefix for architecture-specific files, and the distutils installs an 
entire package to either prefix or exec-prefix depending upon its "purity".

I suspect that the reason is legacy: originally packages didn't exist, and 
then even when they did, the distutils didn't.  Most of the stdlib is 
developed in-place without benefit of the distutils, so it makes sense 
historically that the modules would be where they are, especially since the 
distutils need some of those modules to run.

However, this alone wouldn't be a reason not to build and install 
ElementTree using the distutils.


From barry at python.org  Thu Dec 15 05:26:38 2005
From: barry at python.org (Barry Warsaw)
Date: Wed, 14 Dec 2005 23:26:38 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <43A0B523.5040406@object-craft.com.au>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<43A0B523.5040406@object-craft.com.au>
Message-ID: <1134620798.10290.3.camel@geddy.wooz.org>

On Thu, 2005-12-15 at 11:13 +1100, Dave Cole wrote:

> The only thing I strongly disagree with is the promotion of javaNaming
> to equal footing with python_naming.

Actually, they're not on equal footing atm.  I happen to agree with you
though.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051214/fa044310/attachment-0001.pgp

From guido at python.org  Thu Dec 15 05:31:03 2005
From: guido at python.org (Guido van Rossum)
Date: Wed, 14 Dec 2005 20:31:03 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <20051215030945.GA382@alcyon.progiciels-bpi.ca>
References: <439A110E.8090605@colorstudy.com> <439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<20051215022139.GB30618@alcyon.progiciels-bpi.ca>
	<ca471dc20512141842t5451ff6cj9cd19525005a42a@mail.gmail.com>
	<20051215030945.GA382@alcyon.progiciels-bpi.ca>
Message-ID: <ca471dc20512142031v49a0bd6ex7009ff56aac0c8a2@mail.gmail.com>

On 12/14/05, Fran?ois Pinard <pinard at iro.umontreal.ca> wrote:
> I do not understand your statement that module/package names can only
> conflict with *global* user variable names.  Local variables hide global
> variables with same names, and imported modules are often global
> variables.  So, using a local name in a function prevents accessing
> a module by the same name (defined either locally or globally).

In a module where you import 'foo' I don't understand why you would
name a local variable 'foo'. So I'm not sure how the conflict would
arise.

Perhaps more to the point, I would expect that non-words are just as
likely to accidentally conflict with variable or function names, your
personal habits notwithstanding.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Dec 15 05:41:25 2005
From: guido at python.org (Guido van Rossum)
Date: Wed, 14 Dec 2005 20:41:25 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134620798.10290.3.camel@geddy.wooz.org>
References: <4399F967.3080300@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<43A0B523.5040406@object-craft.com.au>
	<1134620798.10290.3.camel@geddy.wooz.org>
Message-ID: <ca471dc20512142041q7136185aq55ca83ace22ed60@mail.gmail.com>

On 12/14/05, Barry Warsaw <barry at python.org> wrote:
> On Thu, 2005-12-15 at 11:13 +1100, Dave Cole wrote:
>
> > The only thing I strongly disagree with is the promotion of javaNaming
> > to equal footing with python_naming.
>
> Actually, they're not on equal footing atm.  I happen to agree with you
> though.

It doesn't matter. Many large projects are adopting the camelCase
convention, either by choice or by accident. I did a brief review of
Zope 3 and Chandler, and while neither is consistent, camelCase
prevails (Chandler also has a lot of CapWords method names, wihch
suggests they didn't get this from Java -- maybe from C++?).

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ianb at colorstudy.com  Thu Dec 15 06:50:19 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 14 Dec 2005 23:50:19 -0600
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <ca471dc20512142041q7136185aq55ca83ace22ed60@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com>	<1134411382.5676.35.camel@geddy.wooz.org>	<439DDCD2.10803@colorstudy.com>	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>	<439E1137.4010403@colorstudy.com>	<1134595108.10260.55.camel@geddy.wooz.org>	<43A0B523.5040406@object-craft.com.au>	<1134620798.10290.3.camel@geddy.wooz.org>
	<ca471dc20512142041q7136185aq55ca83ace22ed60@mail.gmail.com>
Message-ID: <43A1041B.40801@colorstudy.com>

Guido van Rossum wrote:
> On 12/14/05, Barry Warsaw <barry at python.org> wrote:
> 
>>On Thu, 2005-12-15 at 11:13 +1100, Dave Cole wrote:
>>
>>
>>>The only thing I strongly disagree with is the promotion of javaNaming
>>>to equal footing with python_naming.
>>
>>Actually, they're not on equal footing atm.  I happen to agree with you
>>though.
> 
> 
> It doesn't matter. Many large projects are adopting the camelCase
> convention, either by choice or by accident. I did a brief review of
> Zope 3 and Chandler, and while neither is consistent, camelCase
> prevails (Chandler also has a lot of CapWords method names, wihch
> suggests they didn't get this from Java -- maybe from C++?).

Everything that touches wx seems to adopt CapWords method names -- in 
part (hopefully) or in whole.  Wx's API comes from Windows, and the 
Microsoft method conventions.

I'd say that there's pretty strong support of underscores for functions, 
and less so for methods.

But that doesn't really matter that much -- it's still valid to say that 
there's a preference, because there's a lot of code being created that 
doesn't have to be consistent with anything in particular, or has to be 
consistent with conflicting styles and must choose one.  A suggestion in 
PEP 8 is better than flipping a coin.

-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org

From pje at telecommunity.com  Thu Dec 15 07:01:13 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 15 Dec 2005 01:01:13 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <43A1041B.40801@colorstudy.com>
References: <ca471dc20512142041q7136185aq55ca83ace22ed60@mail.gmail.com>
	<4399F967.3080300@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<43A0B523.5040406@object-craft.com.au>
	<1134620798.10290.3.camel@geddy.wooz.org>
	<ca471dc20512142041q7136185aq55ca83ace22ed60@mail.gmail.com>
Message-ID: <5.1.1.6.0.20051215005630.01de86f8@mail.telecommunity.com>

At 11:50 PM 12/14/2005 -0600, Ian Bicking wrote:
>Guido van Rossum wrote:
> > On 12/14/05, Barry Warsaw <barry at python.org> wrote:
> >
> >>On Thu, 2005-12-15 at 11:13 +1100, Dave Cole wrote:
> >>
> >>
> >>>The only thing I strongly disagree with is the promotion of javaNaming
> >>>to equal footing with python_naming.
> >>
> >>Actually, they're not on equal footing atm.  I happen to agree with you
> >>though.
> >
> >
> > It doesn't matter. Many large projects are adopting the camelCase
> > convention, either by choice or by accident. I did a brief review of
> > Zope 3 and Chandler, and while neither is consistent, camelCase
> > prevails (Chandler also has a lot of CapWords method names, wihch
> > suggests they didn't get this from Java -- maybe from C++?).
>
>Everything that touches wx seems to adopt CapWords method names -- in
>part (hopefully) or in whole.  Wx's API comes from Windows, and the
>Microsoft method conventions.

Yes, at least the Chandler use of CapWords is due to wx influence (and 
perhaps a little of Visual Basic as well).  In theory we use PEP 8 as the 
basis for the project's style guidelines, but in practice, code that works 
with wxPython has to at least use CapWords for overriding methods defined 
by wx.  In addition, there's a heavy Java influence due to the use of 
PyLucene and other SWIG-wrapped Java compiled to C.

So, as a practical matter, a One True Naming Style isn't going to happen 
soon, certainly not in Chandler.  We are making some progress getting rid 
of Java-isms like classes named 'blah.foo.foo.foo.Foo', though.  :)


From fredrik at pythonware.com  Thu Dec 15 08:02:48 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 15 Dec 2005 08:02:48 +0100
Subject: [Python-Dev] Sharing expat instances
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de><dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de><dni8qe$5eq$1@sea.gmane.org><439D1B6D.9080208@v.loewis.de><dnmsft$t1g$1@sea.gmane.org>
	<dnnh9n$c5j$1@sea.gmane.org>
Message-ID: <dnr4eq$71c$1@sea.gmane.org>

I wrote:

> my current idea is to
>
>     1.  include it under a different name (_elementtree.so)
>
>     2.  add a cElementTree.py under xml.etree, which simply does
>
>             from _elementtree import *

I've implemented this, for now.

can anyone with a working windows setup look at building the
_elementtree module for windows ?

(see the setup.py file for build options; it wants the same defines
as pyexpat, plus one extra define.).

</F>




From janneke at gnu.org  Thu Dec 15 09:20:38 2005
From: janneke at gnu.org (Jan Nieuwenhuizen)
Date: Thu, 15 Dec 2005 08:20:38 +0000 (UTC)
Subject: [Python-Dev] patch tracker ping: cross compile and mingw support
Message-ID: <loom.20051215T085700-328@post.gmane.org>

Is the patch tracker still the preferred place to present a patch?

I had the impression that previous versions of these patches had almost gone in,
but for the cleanups that I sought to implement in

https://sourceforge.net/tracker/index.php\
?func=detail&aid=1339673&group_id=5470&atid=305470

Greetings,
Jan.

Alternatively, see
    http://lilypond.org/~janneke/software/python-context-cross.patch
    http://lilypond.org/~janneke/software/python-context-mingw.patch

-- 
Jan Nieuwenhuizen <janneke at gnu.org> | GNU LilyPond - The music typesetter
http://www.xs4all.nl/~jantien       | http://www.lilypond.org



From wlangner at googlemail.com  Thu Dec 15 11:26:36 2005
From: wlangner at googlemail.com (wlangner@googlemail.com)
Date: Thu, 15 Dec 2005 11:26:36 +0100
Subject: [Python-Dev] PEP 8 updates/clarifications, function/method style
Message-ID: <946c5eed0512150226x7618d0l3ceef8f9a948522c@mail.gmail.com>

Hi,

>>> Too late. I don't think the diversity is all that distracting.
>> I disagree. One of the things that Java got very much right was to
>> specify, from the very beginning, what the preferred conventions are
>> for naming conventions. (Packages in lowercase, Classes in CapWords,
>> methods and variables in lowerCapWords, constants optionally in
>> ALL_CAPS. Abbrevs avoided, acronyms have all letters capitalized, eg:
>> SimpleHTTPServer.)
>>
>> The conventions are nearly universally followed, and as a result in
>> java I always know how to spell things. I never have to remember
>> whether it's myDict.hasKey() and myDict.popitem() or myDict.has_key()
>> and myDict.popItem(). Haskell goes too far -- they REQUIRE a certain
>> convention as part of the language... this prevents breaking the rules
>> on purpose (eg: wrapping a library from another language, or using
>> an object with attributes to represent an XML node with child nodes).
>
> I agree completely with this.  I might remember the name of a method,
> but I don't always remember the capping and the possible use of
> underscores.  Consistency would be really nice.  I am not saying we
> should rename everything (at least not until Python 3  =), but at
> least we can make sure new stuff that is not preexisting can use a
> consistent naming scheme.
>
> And as for it being contentious, I say Guido can pronounce on this. We are all grown-ups and can learn to name things in a certain way to
> give our memories an easier time.  =)

Same for me. Most time I can remember the name but stuck with
capping of the word. And one of the advantages of Java std lib is that
naming is consistent.
I hope new stuff will follow only one naming style. And now we should
(or one person :-) should) decide which one. And that's the way to go
for new stuff in std lib. And it states as an example for external modules.
As an example we can check if ElementTree matches this. <g>

For Python 3 it's possible to switch to this consistent style. For a new
and better world.


bye by Wolfgang

From greg.ewing at canterbury.ac.nz  Thu Dec 15 12:05:54 2005
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 16 Dec 2005 00:05:54 +1300
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <ca471dc20512141842t5451ff6cj9cd19525005a42a@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<20051215022139.GB30618@alcyon.progiciels-bpi.ca>
	<ca471dc20512141842t5451ff6cj9cd19525005a42a@mail.gmail.com>
Message-ID: <43A14E12.5060108@canterbury.ac.nz>

Guido van Rossum wrote:

> You *might* want
> to separate classes with two blank lines if the methods within them
> are separated by single blank lines, but even there it's probably
> overkill.

Lately I've taken to putting a separator like this
between consecutive class definitions of any substantial
size:

#-------------------------------------------------------

I find it helps a lot when I'm skimming through looking
for the beginnings of classes.

Not suggesting this should go in PEP 8, though!

Greg

From nico at tekNico.net  Thu Dec 15 14:16:09 2005
From: nico at tekNico.net (Nicola Larosa)
Date: Thu, 15 Dec 2005 14:16:09 +0100
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <43A14E12.5060108@canterbury.ac.nz>
References: <4399F967.3080300@colorstudy.com>
	<439A110E.8090605@colorstudy.com>	<1134411382.5676.35.camel@geddy.wooz.org>	<439DDCD2.10803@colorstudy.com>	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>	<439E1137.4010403@colorstudy.com>	<1134595108.10260.55.camel@geddy.wooz.org>	<20051215022139.GB30618@alcyon.progiciels-bpi.ca>	<ca471dc20512141842t5451ff6cj9cd19525005a42a@mail.gmail.com>
	<43A14E12.5060108@canterbury.ac.nz>
Message-ID: <dnrqar$6j6$1@sea.gmane.org>

> Lately I've taken to putting a separator like this
> between consecutive class definitions of any substantial
> size:
> 
> #-------------------------------------------------------
> 
> I find it helps a lot when I'm skimming through looking
> for the beginnings of classes.

An editor/IDE with folding support (keyboard shortcuts included) is a more
comfortable solution. :-)

-- 
Nicola Larosa - nico at tekNico.net

The only thing the meek inherit is their ass being handed to them.
 -- Elaine "hfb" Ashton, October 2005


From barry at python.org  Thu Dec 15 16:11:53 2005
From: barry at python.org (Barry Warsaw)
Date: Thu, 15 Dec 2005 10:11:53 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <ca471dc20512141704x4d562fa5wad14cc2d19376243@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<ca471dc20512141410qb84ba99ve275fb95ab8ba8dc@mail.gmail.com>
	<1134601008.11465.11.camel@geddy.wooz.org>
	<ca471dc20512141704x4d562fa5wad14cc2d19376243@mail.gmail.com>
Message-ID: <1134659513.10291.16.camel@geddy.wooz.org>

On Wed, 2005-12-14 at 17:04 -0800, Guido van Rossum wrote:

> > One question: you made the suggestion that a blank line separate the
> > last line in a tqs docstring from its closing tqs.  I'm wondering why
> > that is.  Note that python-mode now (or shall we say with the fix I just
> > made ;) properly fills paragraphs in those strings, even if there is no
> > blank line before the closing tqs line.
> 
> Primarily because I didn't have that fix when I wrote it, and I think
> many people don't have it. 

Actually, the fix just made paragraph filling not throw an error.  I'm
pretty sure the bug of M-q filling past the closing tqs has been fixed
for a long time.

> And did you fix it in IDLE too?

Nope, I assigned that to Tim <wink>.

> I can make up another reason too, but this is the main one. (I think
> it also visually separates the doc string paragraphs from the code
> better.)

I'm just trying to suss out whether the reason is linked to the old
python-mode bug or whether there's a stylistic motivation here.  If it's
the former, I think we should remove the recommendation.  If it's the
latter, I might disagree but we can leave it in.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051215/a842457f/attachment.pgp

From reinhold-birkenfeld-nospam at wolke7.net  Thu Dec 15 16:18:52 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Thu, 15 Dec 2005 16:18:52 +0100
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <43A1041B.40801@colorstudy.com>
References: <4399F967.3080300@colorstudy.com>	<1134411382.5676.35.camel@geddy.wooz.org>	<439DDCD2.10803@colorstudy.com>	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>	<439E1137.4010403@colorstudy.com>	<1134595108.10260.55.camel@geddy.wooz.org>	<43A0B523.5040406@object-craft.com.au>	<1134620798.10290.3.camel@geddy.wooz.org>	<ca471dc20512142041q7136185aq55ca83ace22ed60@mail.gmail.com>
	<43A1041B.40801@colorstudy.com>
Message-ID: <dns1gs$bn$1@sea.gmane.org>

Ian Bicking wrote:
> Guido van Rossum wrote:
>> On 12/14/05, Barry Warsaw <barry at python.org> wrote:
>> 
>>>On Thu, 2005-12-15 at 11:13 +1100, Dave Cole wrote:
>>>
>>>
>>>>The only thing I strongly disagree with is the promotion of javaNaming
>>>>to equal footing with python_naming.
>>>
>>>Actually, they're not on equal footing atm.  I happen to agree with you
>>>though.
>> 
>> 
>> It doesn't matter. Many large projects are adopting the camelCase
>> convention, either by choice or by accident. I did a brief review of
>> Zope 3 and Chandler, and while neither is consistent, camelCase
>> prevails (Chandler also has a lot of CapWords method names, wihch
>> suggests they didn't get this from Java -- maybe from C++?).
> 
> Everything that touches wx seems to adopt CapWords method names -- in 
> part (hopefully) or in whole.  Wx's API comes from Windows, and the 
> Microsoft method conventions.

Bad, that. While the Windows API names once made sense when they referred to
standalone functions, not methods, they now look ugly in wx or .NET.

Reinhold

-- 
Mail address is perfectly valid!


From barry at python.org  Thu Dec 15 16:25:33 2005
From: barry at python.org (Barry Warsaw)
Date: Thu, 15 Dec 2005 10:25:33 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <43A0D061.5000506@colorstudy.com>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<43A0B523.5040406@object-craft.com.au>
	<43A0D061.5000506@colorstudy.com>
Message-ID: <1134660333.10290.23.camel@geddy.wooz.org>

On Wed, 2005-12-14 at 20:09 -0600, Ian Bicking wrote:

> I think it's reasonable to loosen the 
> phrasing a bit -- it's nearly always better to stay consistent with a 
> package than follow PEP 8 on this point.

I agree, but actually I think there's a wider point to be made.  The
worst of all possible worlds is a module that is internally inconsistent
about any of the guidelines.  But I think the text in the introduction
makes this point well enough.  There should be one clear recommendation
for new code, and since we've had this for four years now, I think the
underline_words recommendation should stand.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051215/cb1ab47b/attachment.pgp

From barry at python.org  Thu Dec 15 16:27:18 2005
From: barry at python.org (Barry Warsaw)
Date: Thu, 15 Dec 2005 10:27:18 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <43A0D372.1020001@colorstudy.com>
References: <4399F967.3080300@colorstudy.com>
	<1134168683.19370.29.camel@geddy.wooz.org>
	<439A110E.8090605@colorstudy.com>	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<43A0D372.1020001@colorstudy.com>
Message-ID: <1134660438.10290.26.camel@geddy.wooz.org>

On Wed, 2005-12-14 at 20:22 -0600, Ian Bicking wrote:

> I think the reference to Emacs here is unneeded.  "Use 4 spaces per 
> indentation level" is sufficient instruction on its own.

Agreed.  I've removed all references to Emacs in the PEP.

> I think the reference to PEP 263 tests is kind of obvious.

Agreed.
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051215/a4bd09ca/attachment.pgp

From barry at python.org  Thu Dec 15 16:31:02 2005
From: barry at python.org (Barry Warsaw)
Date: Thu, 15 Dec 2005 10:31:02 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <ca471dc20512141842t5451ff6cj9cd19525005a42a@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com> <439A110E.8090605@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<20051215022139.GB30618@alcyon.progiciels-bpi.ca>
	<ca471dc20512141842t5451ff6cj9cd19525005a42a@mail.gmail.com>
Message-ID: <1134660662.10290.28.camel@geddy.wooz.org>

On Wed, 2005-12-14 at 18:42 -0800, Guido van Rossum wrote:

> > Deprecated functions could be listed, too.
> 
> I think that's more proper for a separate PEP -- the style guide
> shouldn't have to be updated each time something else is deprecated.

+1 to that, although I won't write that PEP.  If someone does, we can
link to it from PEP 8.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051215/696f31cc/attachment.pgp

From barry at python.org  Thu Dec 15 16:40:21 2005
From: barry at python.org (Barry Warsaw)
Date: Thu, 15 Dec 2005 10:40:21 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <ca471dc20512142041q7136185aq55ca83ace22ed60@mail.gmail.com>
References: <4399F967.3080300@colorstudy.com>
	<1134411382.5676.35.camel@geddy.wooz.org>
	<439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<43A0B523.5040406@object-craft.com.au>
	<1134620798.10290.3.camel@geddy.wooz.org>
	<ca471dc20512142041q7136185aq55ca83ace22ed60@mail.gmail.com>
Message-ID: <1134661221.10291.37.camel@geddy.wooz.org>

On Wed, 2005-12-14 at 20:41 -0800, Guido van Rossum wrote:
> On 12/14/05, Barry Warsaw <barry at python.org> wrote:
> > On Thu, 2005-12-15 at 11:13 +1100, Dave Cole wrote:
> >
> > > The only thing I strongly disagree with is the promotion of javaNaming
> > > to equal footing with python_naming.
> >
> > Actually, they're not on equal footing atm.  I happen to agree with you
> > though.
> 
> It doesn't matter. Many large projects are adopting the camelCase
> convention, either by choice or by accident. I did a brief review of
> Zope 3 and Chandler, and while neither is consistent, camelCase
> prevails (Chandler also has a lot of CapWords method names, wihch
> suggests they didn't get this from Java -- maybe from C++?).

That's fine.  As always, projects (especially big framework-y ones like
Zope and Chandler) are free to adopt whatever they want.  Their internal
consistency is more important anyway than adherence to PEP 8.

PEP 8 though is primarily about establishing guidelines for the standard
library.  The underline_words recommendation has been in place for 4+
years now, and modules that have been written in that time frame have
been written against to those rules.  I see no reason to change now just
to be more Java-like.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051215/58cf01be/attachment.pgp

From pinard at iro.umontreal.ca  Thu Dec 15 16:39:45 2005
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Thu, 15 Dec 2005 10:39:45 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <ca471dc20512142031v49a0bd6ex7009ff56aac0c8a2@mail.gmail.com>
References: <439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<20051215022139.GB30618@alcyon.progiciels-bpi.ca>
	<ca471dc20512141842t5451ff6cj9cd19525005a42a@mail.gmail.com>
	<20051215030945.GA382@alcyon.progiciels-bpi.ca>
	<ca471dc20512142031v49a0bd6ex7009ff56aac0c8a2@mail.gmail.com>
Message-ID: <20051215153945.GA6674@phenix.sram.qc.ca>

[Guido van Rossum]

>In a module where you import 'foo' I don't understand why you would
>name a local variable 'foo'. So I'm not sure how the conflict would
>arise.

It goes the other way around.  First suppose, following an example in 
a previous message, that I have a lot of variables named 'text', as this 
is kind of the default name everywhere for the text being handled in 
a function, in a program handling text.

Now, let's presume that Python adds a new standard module named 'text', 
containing many things that interest me, enough that I would want to 
import the whole module globally in that program.  The module would be 
shadowed at many places, and whenever I'll have to use it, I'll have to 
make some stunts, like changing the local 'text' here and there for 
unshadowing the global module, or importing it the global module under 
a different name.  In the first case, I have inconsistent naming.  In 
the second case, I impose an indirection in the understanding to whoever 
will maintain my program after me.  In any case, it looks messed up.

Now, I'm sure I'm not alone, as a programmer, having plenty of habits
or own conventions.  What I do in a program, I'm likely doing in many 
others.  Seeing a module name added in the standard Python library with 
a name that I already much use usually means that I have no choice than 
to give up on my own choices, because Python will not change his (:-), 
and I ought to do tons of changes on my side so programs stay clean in 
the long run, and do not finish looking like the remains of battle 
field.

'text' is a particularly good example for me, because when I started 
using Python 1.5.2 after having done a lot of C and Perl, I already had 
the strong habit of using 'string' everywhere for the string under 
consideration, and had to change this particular habit so the 'string' 
module would be available.  This is why I started to use 'text' 
everywhere, and would not like changing conventions again.

Let's consider how people name variables.  Some people write programs 
like they do algebra and stick with one letter variables, merely one of 
those habits directly or indirectly inherited from FORTRAN (FORmula 
TRANslator).  These people like using `k' for counters, and `s' for 
string or socket, depending on context.  They either want to spare 
either the memory required to store the full identifier, or more 
probably, their own keystrokes within their editor, unaware of the 
modern un-abbreviating facilities. :-)  A second set of programmers (I'm 
in this set) prefer identifiers written as natural language words fully 
written -- my counters are written `counter', and cursors are written 
`cursor' --, or agglomeration of a few words tied with underlines.  The 
remainder of programmers might do clever things like using neologisms 
('textwrap' is an example of those), or much, much less clever things 
like making their identifiers by random removal of letters from words --
vowels often being the first to go --.  By random, I mean that it 
constantly and gratuitously strains the memory of people later having to 
read their code.

No standard Python module name uses single-letter algebraic-style names, 
thanks God, so the FORTRANic programmers are already protected.  I would 
like if module naming convention was extending its protective wing over 
the second set of programmers (the set I'm in, of course!).  Programmers 
not being in the first two sets do not require so much protection: for 
the clever ones, because the probability of clashes between neologisms 
is much smaller than the probability of clashes with usual natural 
language words; for the randomising programmers, well, because they do 
not deserve being protected :-).

-- 
Fran?ois Pinard   http://pinard.progiciels-bpi.ca

From aahz at pythoncraft.com  Thu Dec 15 17:35:19 2005
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 15 Dec 2005 08:35:19 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134661221.10291.37.camel@geddy.wooz.org>
References: <439DDCD2.10803@colorstudy.com>
	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>
	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>
	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>
	<439E1137.4010403@colorstudy.com>
	<1134595108.10260.55.camel@geddy.wooz.org>
	<43A0B523.5040406@object-craft.com.au>
	<1134620798.10290.3.camel@geddy.wooz.org>
	<ca471dc20512142041q7136185aq55ca83ace22ed60@mail.gmail.com>
	<1134661221.10291.37.camel@geddy.wooz.org>
Message-ID: <20051215163519.GA14001@panix.com>

On Thu, Dec 15, 2005, Barry Warsaw wrote:
>
> PEP 8 though is primarily about establishing guidelines for the standard
> library.  The underline_words recommendation has been in place for 4+
> years now, and modules that have been written in that time frame have
> been written against to those rules.  I see no reason to change now just
> to be more Java-like.

+1
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Don't listen to schmucks on USENET when making legal decisions.  Hire
yourself a competent schmuck."  --USENET schmuck (aka Robert Kern)

From wlangner at googlemail.com  Thu Dec 15 17:58:03 2005
From: wlangner at googlemail.com (Wolfgang)
Date: Thu, 15 Dec 2005 17:58:03 +0100
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <1134661221.10291.37.camel@geddy.wooz.org>
References: <4399F967.3080300@colorstudy.com>	<1134411382.5676.35.camel@geddy.wooz.org>	<439DDCD2.10803@colorstudy.com>	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>	<439E1137.4010403@colorstudy.com>	<1134595108.10260.55.camel@geddy.wooz.org>	<43A0B523.5040406@object-craft.com.au>	<1134620798.10290.3.camel@geddy.wooz.org>	<ca471dc20512142041q7136185aq55ca83ace22ed60@mail.gmail.com>
	<1134661221.10291.37.camel@geddy.wooz.org>
Message-ID: <43A1A09B.8090904@googlemail.com>

Hi,

Barry Warsaw wrote:
> That's fine.  As always, projects (especially big framework-y ones like
> Zope and Chandler) are free to adopt whatever they want.  Their internal
> consistency is more important anyway than adherence to PEP 8.
> 
> PEP 8 though is primarily about establishing guidelines for the standard
> library.  The underline_words recommendation has been in place for 4+
> years now, and modules that have been written in that time frame have
> been written against to those rules.  I see no reason to change now just
> to be more Java-like.

There is no need to change this, but to follow this.
I checked new added modules for last python versions and most modules
used lowercase with "_" (foo_bar).
But "datetiem" doesn't. There "_" is lost and all is lowercase
like "fromtimestamp" or "utcfromtimestamp" or "toordinal".
I thought this should be "from_timestamp" or "to_ordinal".
But I'm not a native speaker.


bye by Wolfgang

From steve at holdenweb.com  Thu Dec 15 17:47:17 2005
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 15 Dec 2005 16:47:17 +0000
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <20051215153945.GA6674@phenix.sram.qc.ca>
References: <439DDCD2.10803@colorstudy.com>	<5.1.1.6.0.20051212154003.020c3e48@mail.telecommunity.com>	<5.1.1.6.0.20051212181259.01f55980@mail.telecommunity.com>	<e8a0972d0512121557n73692bfcge19e2bffb2cce76@mail.gmail.com>	<439E1137.4010403@colorstudy.com>	<1134595108.10260.55.camel@geddy.wooz.org>	<20051215022139.GB30618@alcyon.progiciels-bpi.ca>	<ca471dc20512141842t5451ff6cj9cd19525005a42a@mail.gmail.com>	<20051215030945.GA382@alcyon.progiciels-bpi.ca>	<ca471dc20512142031v49a0bd6ex7009ff56aac0c8a2@mail.gmail.com>
	<20051215153945.GA6674@phenix.sram.qc.ca>
Message-ID: <dns6mt$hmg$2@sea.gmane.org>

Fran?ois Pinard wrote:
> [Guido van Rossum]
> 
> 
>>In a module where you import 'foo' I don't understand why you would
>>name a local variable 'foo'. So I'm not sure how the conflict would
>>arise.
> 
> 
> It goes the other way around.  First suppose, following an example in 
> a previous message, that I have a lot of variables named 'text', as this 
> is kind of the default name everywhere for the text being handled in 
> a function, in a program handling text.
> 
[lots of hypothetical talk about the addition of the 'text' module]

Seems to me your solution is obvious:

   import text as somethingelse

regartds
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From johannes.nicolai at hpi.uni-potsdam.de  Thu Dec 15 19:52:14 2005
From: johannes.nicolai at hpi.uni-potsdam.de (Johannes Nicolai)
Date: Thu, 15 Dec 2005 19:52:14 +0100
Subject: [Python-Dev] Needed tester for patch in urllib.py module
Message-ID: <200512151952.14933.johannes.nicolai@hpi.uni-potsdam.de>

I have written a patch, that will enable urllib.py to use http and https 
proxies with password authentification, if the password is stored in the 
corresponding environment variable (e. g. 
http_proxy=password:username at www.foo.com)

If a http or https proxy requires authentification, but there was no entry in 
the environment variable, the password is asked from the user and stored for 
succeeding attempts.

While writing the patch, I recognized, that the old code will give a 
traceback, if an environment variable for a protocol is set, where proxy 
support is not (yet) supported in urllib.py (e. g. ftp protocol)
My patch will raise a meaningful exception in this case.

Finally, I noticed, that the old implementation of the https_proxy code is 
incorrect, if some page accessed via the https_proxy requires authorization. 
In this case, the user has to enter the password and then is directly 
directed to the page where the proxy is skipped.
I have fixed this problem too.

The patch was intensively tested with squid 2.5 and squid 3.0 (supports https 
protocol as a proxy) and applies to the trunk in the subversion repository.

You can find the patch and its history on
https://sourceforge.net/tracker/?func=detail&aid=1349118&group_id=5470&atid=305470

Mr. van Rossum asked me, whether I could find a developer on this list in 
order to test the patch.
Any help would be appreciated.

Thanks in advance

Johannes Nicolai

From jcarlson at uci.edu  Thu Dec 15 20:35:39 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu, 15 Dec 2005 11:35:39 -0800
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <dns1gs$bn$1@sea.gmane.org>
References: <43A1041B.40801@colorstudy.com> <dns1gs$bn$1@sea.gmane.org>
Message-ID: <20051215112438.6BB1.JCARLSON@uci.edu>


Reinhold Birkenfeld <reinhold-birkenfeld-nospam at wolke7.net> wrote:
> Ian Bicking wrote:
> > Guido van Rossum wrote:
> >> It doesn't matter. Many large projects are adopting the camelCase
> >> convention, either by choice or by accident. I did a brief review of
> >> Zope 3 and Chandler, and while neither is consistent, camelCase
> >> prevails (Chandler also has a lot of CapWords method names, wihch
> >> suggests they didn't get this from Java -- maybe from C++?).
> > 
> > Everything that touches wx seems to adopt CapWords method names -- in 
> > part (hopefully) or in whole.  Wx's API comes from Windows, and the 
> > Microsoft method conventions.
> 
> Bad, that. While the Windows API names once made sense when they referred to
> standalone functions, not methods, they now look ugly in wx or .NET.

In regards to naming conventions, I find that CapWords are easier for me
to read and remember as a native speaker of english.  I've heard
statements from non-native english speakers that CapWords are hard to
read and/or understand, but in the realm of wxPython, changing literally
thousands of CapWords references to lower_underscore during wrapping,
along with the hundreds of thousands of references in just a few of the
larger wxPython projects is a bit out of line.  Or even convincing the
hundreds of other packages which use CapWords or lower_underscore to
change how their naming conventions work is going to be a tough fight.

Personally, I'm of the opinion that as long as a module or package uses
consistant naming semantics, whether it is CapWords, camelCase, or
lower_underscore (all of which beat lowerwithoutspaces), then users of
that module or package should be able to cope.  Yeah, consistancy
cross-packages would be nice, heck consistancy in the standard library
would be nice, but changing the naming in the standard library for a
purity argument, I think, is a red herring.

 - Josiah


From wolfgang.langner at googlemail.com  Thu Dec 15 11:22:13 2005
From: wolfgang.langner at googlemail.com (wolfgang.langner@googlemail.com)
Date: Thu, 15 Dec 2005 11:22:13 +0100
Subject: [Python-Dev] PEP 8 updates/clarifications, function/method style
In-Reply-To: <bbaeab100512141325h7440e0dane9e8cd34cc670a59@mail.gmail.com>
References: <20051214113549.swbgg3zat8g0wo4k@login.werra.lunarpages.com>
	<bbaeab100512141325h7440e0dane9e8cd34cc670a59@mail.gmail.com>
Message-ID: <43A143D5.1050307@googlemail.com>

Hi,

>>> Too late. I don't think the diversity is all that distracting.
>> I disagree. One of the things that Java got very much right was to
>> specify, from the very beginning, what the preferred conventions are
>> for naming conventions. (Packages in lowercase, Classes in CapWords,
>> methods and variables in lowerCapWords, constants optionally in
>> ALL_CAPS. Abbrevs avoided, acronyms have all letters capitalized, eg:
>> SimpleHTTPServer.)
>>
>> The conventions are nearly universally followed, and as a result in
>> java I always know how to spell things. I never have to remember
>> whether it's myDict.hasKey() and myDict.popitem() or myDict.has_key()
>> and myDict.popItem(). Haskell goes too far -- they REQUIRE a certain
>> convention as part of the language... this prevents breaking the rules
>> on purpose (eg: wrapping a library from another language, or using
>> an object with attributes to represent an XML node with child nodes).
> 
> I agree completely with this.  I might remember the name of a method,
> but I don't always remember the capping and the possible use of
> underscores.  Consistency would be really nice.  I am not saying we
> should rename everything (at least not until Python 3  =), but at
> least we can make sure new stuff that is not preexisting can use a
> consistent naming scheme.
> 
> And as for it being contentious, I say Guido can pronounce on this. 
> We are all grown-ups and can learn to name things in a certain way to
> give our memories an easier time.  =)

Same for me. Most time I can remember the name but stuck with
capping of the word. And one of the advantages of Java std lib is that naming 
is consistent.
I hope new stuff will follow only one naming style. And now we should
(or one person :-) should) decide which one. And that's the way to go
for new stuff in std lib. And it states as an example for external modules.
As an example we can check if ElementTree matches this. <g>

For Python 3 it's possible to switch to this consistent style. For a new
and better world.


bye by Wolfgang

From martin at v.loewis.de  Thu Dec 15 21:30:16 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 15 Dec 2005 21:30:16 +0100
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <5.1.1.6.0.20051214144853.03b0fff8@mail.telecommunity.com>
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de>
	<dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de>
	<dni8qe$5eq$1@sea.gmane.org>	<439D1B6D.9080208@v.loewis.de>
	<dnmsft$t1g$1@sea.gmane.org>	<dnnh9n$c5j$1@sea.gmane.org>	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
	<5.1.1.6.0.20051214144853.03b0fff8@mail.telecommunity.com>
Message-ID: <43A1D258.4050804@v.loewis.de>

Phillip J. Eby wrote:
> I don't know.  I can see that the split makes sense for prefix/exec-prefix 
> distinctions, but then again, the disutils will install an entire 
> distribution in exec-prefix if it contains "impure" parts, so that's 
> certainly an option here.
> 
> On the other hand, it's not clear to me *why* the lib-dynload/DLLs 
> directories exist, since it seems to me that that's what exec-prefix is 
> for.

Can you please explain? exec_prefix will point to, say,
/usr/i686; it shouldn't be that .so files are directly installed in
that location. Instead, Python searches them in
EXEC_PREFIX "/lib/python" VERSION "/lib-dynload".

> Perhaps somebody can explain why lib-dynload/ and DLLs/
> exist?

To have a directory on sys.path where native modules are found.

> Perhaps some platforms have to add these directories to some 
> godforsaken environment variables like LD_LIBRARY_PATH or something?

Not to my knowledge, no. lib-dynload was introduced in revision 8976,
where it was renamed from "/sharedmodules". This, in turn, was
introduced into getpath.c in revision 7775 (and 7776). It was added to
Modules/Setup.in in revision 6313, and to Makefile.in in 6321.
Unfortunately, the checkin message of 6321 only says

  More changes to install targets.

The notion of a separate makefile variable for shared libraries
goes back to Modules/Makefile.pre.in at 4333, which first introduced
dynamic loading (in 1994).

Regards,
Martin

From trentm at ActiveState.com  Thu Dec 15 20:48:04 2005
From: trentm at ActiveState.com (Trent Mick)
Date: Thu, 15 Dec 2005 11:48:04 -0800
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <dnr4eq$71c$1@sea.gmane.org>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<dnnh9n$c5j$1@sea.gmane.org> <dnr4eq$71c$1@sea.gmane.org>
Message-ID: <20051215194804.GB28044@ActiveState.com>

[Fredrik Lundh wrote]
> can anyone with a working windows setup look at building the
> _elementtree module for windows ?
> 
> (see the setup.py file for build options; it wants the same defines
> as pyexpat, plus one extra define.).


    C:\trentm\src\python\python\PCbuild>python
    Python 2.5a0 (#60, Dec 15 2005, 11:17:15) [MSC v.1310 32 bit (Intel)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from _elementtree import Element, ElementTree
    >>> root = Element("root")
    >>> root.append(Element("one"))
    >>> tree = ElementTree(root)
    >>> import sys
    >>> tree.write(sys.stdout)
    <root><one /></root>>>>


    C:\trentm\src\python\python\PCbuild>svn st
    A      _elementtree.vcproj
    M      pcbuild.sln

Shall I checkin the build changes?


Trent

-- 
Trent Mick
TrentM at ActiveState.com

From martin at v.loewis.de  Thu Dec 15 21:35:36 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 15 Dec 2005 21:35:36 +0100
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <20051215194804.GB28044@ActiveState.com>
References: <17304.33755.693941.811233@montanaro.dyndns.org>	<dnnh9n$c5j$1@sea.gmane.org>
	<dnr4eq$71c$1@sea.gmane.org>
	<20051215194804.GB28044@ActiveState.com>
Message-ID: <43A1D398.2020505@v.loewis.de>

Trent Mick wrote:
>     C:\trentm\src\python\python\PCbuild>svn st
>     A      _elementtree.vcproj
>     M      pcbuild.sln
> 
> Shall I checkin the build changes?

Go ahead. A change to Tools/msi/msi.py is also in order.

Regards,
Martin

From pje at telecommunity.com  Thu Dec 15 21:42:22 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 15 Dec 2005 15:42:22 -0500
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <43A1D258.4050804@v.loewis.de>
References: <5.1.1.6.0.20051214144853.03b0fff8@mail.telecommunity.com>
	<17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de> <dnmsft$t1g$1@sea.gmane.org>
	<dnnh9n$c5j$1@sea.gmane.org>
	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
	<5.1.1.6.0.20051214144853.03b0fff8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051215154009.030b4260@mail.telecommunity.com>

At 09:30 PM 12/15/2005 +0100, Martin v. L?wis wrote:
>Phillip J. Eby wrote:
> > I don't know.  I can see that the split makes sense for prefix/exec-prefix
> > distinctions, but then again, the disutils will install an entire
> > distribution in exec-prefix if it contains "impure" parts, so that's
> > certainly an option here.
> >
> > On the other hand, it's not clear to me *why* the lib-dynload/DLLs
> > directories exist, since it seems to me that that's what exec-prefix is
> > for.
>
>Can you please explain? exec_prefix will point to, say,
>/usr/i686; it shouldn't be that .so files are directly installed in
>that location. Instead, Python searches them in
>EXEC_PREFIX "/lib/python" VERSION "/lib-dynload".

Right; the question is why not just EXEC_PREFIX "/lib/python" VERSION 
instead.  What benefit does the separate directory offer?

Note that the distutils, when installing a package containing C extensions, 
will install to site-packages under sys.exec_prefix; it does not separate 
the C extensions into special alternate library directories.


From martin at v.loewis.de  Thu Dec 15 22:14:17 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 15 Dec 2005 22:14:17 +0100
Subject: [Python-Dev] Location of .so files (Was: Sharing expat
	instances)
In-Reply-To: <5.1.1.6.0.20051215154009.030b4260@mail.telecommunity.com>
References: <5.1.1.6.0.20051214144853.03b0fff8@mail.telecommunity.com>	<17304.33755.693941.811233@montanaro.dyndns.org>	<43989A99.9000402@v.loewis.de>
	<dnhl9k$bt6$1@sea.gmane.org>	<439C7341.4050300@v.loewis.de>
	<dni8qe$5eq$1@sea.gmane.org>	<439D1B6D.9080208@v.loewis.de>
	<dnmsft$t1g$1@sea.gmane.org>	<dnnh9n$c5j$1@sea.gmane.org>	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>	<5.1.1.6.0.20051214144853.03b0fff8@mail.telecommunity.com>
	<5.1.1.6.0.20051215154009.030b4260@mail.telecommunity.com>
Message-ID: <43A1DCA9.8040903@v.loewis.de>

Phillip J. Eby wrote:
> Right; the question is why not just EXEC_PREFIX "/lib/python" VERSION
>  instead.  What benefit does the separate directory offer?

I can only guess: it's primarily a matter of tidiness.

> Note that the distutils, when installing a package containing C
> extensions, will install to site-packages under sys.exec_prefix; it
> does not separate the C extensions into special alternate library
> directories.

Yes, that's inconsistent. One way to make it consistent would be to
honor lib-dynload; the other to get rid of lib-dynload. I cannot see
how one of these option is better than the other (nor can I see
what the relevance for the expat discussion is, so I changed the
subject).

Regards,
Martin


From fredrik at pythonware.com  Thu Dec 15 22:09:39 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 15 Dec 2005 22:09:39 +0100
Subject: [Python-Dev] Sharing expat instances
References: <17304.33755.693941.811233@montanaro.dyndns.org><dnnh9n$c5j$1@sea.gmane.org>
	<dnr4eq$71c$1@sea.gmane.org>
	<20051215194804.GB28044@ActiveState.com>
Message-ID: <dnsm2k$c1j$1@sea.gmane.org>

Trent Mick wrote:

> [Fredrik Lundh wrote]
> > can anyone with a working windows setup look at building the
> > _elementtree module for windows ?
> >
> > (see the setup.py file for build options; it wants the same defines
> > as pyexpat, plus one extra define.).
>
>     C:\trentm\src\python\python\PCbuild>python
>     Python 2.5a0 (#60, Dec 15 2005, 11:17:15) [MSC v.1310 32 bit (Intel)] on win32
>     Type "help", "copyright", "credits" or "license" for more information.
>     >>> from _elementtree import Element, ElementTree
>     >>> root = Element("root")
>     >>> root.append(Element("one"))
>     >>> tree = ElementTree(root)
>     >>> import sys
>     >>> tree.write(sys.stdout)
>     <root><one /></root>>>>

most excellent!

>     C:\trentm\src\python\python\PCbuild>svn st
>     A      _elementtree.vcproj
>     M      pcbuild.sln
>
> Shall I checkin the build changes?

I sure won't stop you.

thanks! /F




From pje at telecommunity.com  Thu Dec 15 22:26:02 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 15 Dec 2005 16:26:02 -0500
Subject: [Python-Dev] Location of .so files (Was: Sharing expat
 instances)
In-Reply-To: <43A1DCA9.8040903@v.loewis.de>
References: <5.1.1.6.0.20051215154009.030b4260@mail.telecommunity.com>
	<5.1.1.6.0.20051214144853.03b0fff8@mail.telecommunity.com>
	<17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de> <dnmsft$t1g$1@sea.gmane.org>
	<dnnh9n$c5j$1@sea.gmane.org>
	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
	<5.1.1.6.0.20051214144853.03b0fff8@mail.telecommunity.com>
	<5.1.1.6.0.20051215154009.030b4260@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051215162310.030d8ef8@mail.telecommunity.com>

At 10:14 PM 12/15/2005 +0100, Martin v. L?wis wrote:
>Phillip J. Eby wrote:
> > Right; the question is why not just EXEC_PREFIX "/lib/python" VERSION
> >  instead.  What benefit does the separate directory offer?
>
>I can only guess: it's primarily a matter of tidiness.
>
> > Note that the distutils, when installing a package containing C
> > extensions, will install to site-packages under sys.exec_prefix; it
> > does not separate the C extensions into special alternate library
> > directories.
>
>Yes, that's inconsistent. One way to make it consistent would be to
>honor lib-dynload; the other to get rid of lib-dynload. I cannot see
>how one of these option is better than the other (nor can I see
>what the relevance for the expat discussion is, so I changed the
>subject).

The relevance to "sharing expat instances" was Fredrik's original question 
about placement of the shared library for cElementTree.  The question 
wasn't whether to get rid of lib-dynload in general, but whether there was 
any reason to do something other than just making ElementTree a 
distutils-installed package with the C extensions inside, the way normal 
distutils-installed packages are.

In any case the discussion is probably moot by now if ElementTree is now 
already built as part of the core.


From martin at v.loewis.de  Thu Dec 15 22:36:30 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 15 Dec 2005 22:36:30 +0100
Subject: [Python-Dev] Location of .so files (Was: Sharing expat
	instances)
In-Reply-To: <5.1.1.6.0.20051215162310.030d8ef8@mail.telecommunity.com>
References: <5.1.1.6.0.20051215154009.030b4260@mail.telecommunity.com>
	<5.1.1.6.0.20051214144853.03b0fff8@mail.telecommunity.com>
	<17304.33755.693941.811233@montanaro.dyndns.org>
	<43989A99.9000402@v.loewis.de> <dnhl9k$bt6$1@sea.gmane.org>
	<439C7341.4050300@v.loewis.de> <dni8qe$5eq$1@sea.gmane.org>
	<439D1B6D.9080208@v.loewis.de> <dnmsft$t1g$1@sea.gmane.org>
	<dnnh9n$c5j$1@sea.gmane.org>
	<5.1.1.6.0.20051213194456.03b44d58@mail.telecommunity.com>
	<5.1.1.6.0.20051214144853.03b0fff8@mail.telecommunity.com>
	<5.1.1.6.0.20051215154009.030b4260@mail.telecommunity.com>
	<5.1.1.6.0.20051215162310.030d8ef8@mail.telecommunity.com>
Message-ID: <43A1E1DE.1050503@v.loewis.de>

Phillip J. Eby wrote:
> The relevance to "sharing expat instances" was Fredrik's original
> question about placement of the shared library for cElementTree.  The
> question wasn't whether to get rid of lib-dynload in general, but
> whether there was any reason to do something other than just making
> ElementTree a distutils-installed package with the C extensions inside,
> the way normal distutils-installed packages are.

I understood the question was whether specifically to place cElementTree
into xml.etree; I think such scenarios had been avoided because people
want to run Python from the build tree, both on Unix and Windows. This
breaks if the extension modules are expected in packages, rather than
being toplevel.

Whether or not they then end up in lib-dynload is a different issue;
as I said, I can't see anything wrong with it.

Regards,
Martin

From barry at python.org  Thu Dec 15 23:01:38 2005
From: barry at python.org (Barry Warsaw)
Date: Thu, 15 Dec 2005 17:01:38 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications, function/method	style
In-Reply-To: <43A143D5.1050307@googlemail.com>
References: <20051214113549.swbgg3zat8g0wo4k@login.werra.lunarpages.com>
	<bbaeab100512141325h7440e0dane9e8cd34cc670a59@mail.gmail.com>
	<43A143D5.1050307@googlemail.com>
Message-ID: <1134684098.10290.90.camel@geddy.wooz.org>

On Thu, 2005-12-15 at 11:22 +0100, wolfgang.langner at googlemail.com
wrote:

> I hope new stuff will follow only one naming style. And now we should
> (or one person :-) should) decide which one.

I guess my point boils down to, we already did decide 4 years ago.
Let's stick with what we've got.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051215/7209aeca/attachment.pgp

From barry at python.org  Thu Dec 15 23:07:38 2005
From: barry at python.org (Barry Warsaw)
Date: Thu, 15 Dec 2005 17:07:38 -0500
Subject: [Python-Dev] PEP 8 updates/clarifications
In-Reply-To: <20051215112438.6BB1.JCARLSON@uci.edu>
References: <43A1041B.40801@colorstudy.com> <dns1gs$bn$1@sea.gmane.org>
	<20051215112438.6BB1.JCARLSON@uci.edu>
Message-ID: <1134684458.10290.97.camel@geddy.wooz.org>

On Thu, 2005-12-15 at 11:35 -0800, Josiah Carlson wrote:

> In regards to naming conventions, I find that CapWords are easier for me
> to read and remember as a native speaker of english.  I've heard
> statements from non-native english speakers that CapWords are hard to
> read and/or understand, but in the realm of wxPython, changing literally
> thousands of CapWords references to lower_underscore during wrapping,
> along with the hundreds of thousands of references in just a few of the
> larger wxPython projects is a bit out of line.  Or even convincing the
> hundreds of other packages which use CapWords or lower_underscore to
> change how their naming conventions work is going to be a tough fight.

None of which PEP 8 recommends (in fact, it strongly recommends against
massive renames).  What PEP 8 is saying is that /new/ code, especially
new code intended for the stdlib, should follow the PEP's guidelines.

Which is why I think we need one strong recommendation.  Over time, as
modules are naturally updated or supplanted, we'll get more consistency
in the stdlib across packages.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051215/860bceeb/attachment.pgp

From trentm at ActiveState.com  Thu Dec 15 23:03:08 2005
From: trentm at ActiveState.com (Trent Mick)
Date: Thu, 15 Dec 2005 14:03:08 -0800
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <43A1D398.2020505@v.loewis.de>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<dnnh9n$c5j$1@sea.gmane.org> <dnr4eq$71c$1@sea.gmane.org>
	<20051215194804.GB28044@ActiveState.com>
	<43A1D398.2020505@v.loewis.de>
Message-ID: <20051215220308.GA23627@ActiveState.com>

[Martin v. Loewis wrote]
> Trent Mick wrote:
> >     C:\trentm\src\python\python\PCbuild>svn st
> >     A      _elementtree.vcproj
> >     M      pcbuild.sln
> > 
> > Shall I checkin the build changes?
> 
> Go ahead. A change to Tools/msi/msi.py is also in order.

Okay, done. I naively added "_elementtree" to the extensions list in
msi.py. I'm guessing that that was all that was necessary.

Trent

-- 
Trent Mick
TrentM at ActiveState.com

From martin at v.loewis.de  Thu Dec 15 23:21:47 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 15 Dec 2005 23:21:47 +0100
Subject: [Python-Dev] Sharing expat instances
In-Reply-To: <20051215220308.GA23627@ActiveState.com>
References: <17304.33755.693941.811233@montanaro.dyndns.org>
	<dnnh9n$c5j$1@sea.gmane.org> <dnr4eq$71c$1@sea.gmane.org>
	<20051215194804.GB28044@ActiveState.com>
	<43A1D398.2020505@v.loewis.de>
	<20051215220308.GA23627@ActiveState.com>
Message-ID: <43A1EC7B.7010006@v.loewis.de>

Trent Mick wrote:
> Okay, done. I naively added "_elementtree" to the extensions list in
> msi.py. I'm guessing that that was all that was necessary.

Right. Thanks!

Martin

From barry at python.org  Fri Dec 16 06:16:49 2005
From: barry at python.org (Barry Warsaw)
Date: Fri, 16 Dec 2005 00:16:49 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
Message-ID: <1134710209.15629.6.camel@geddy.wooz.org>

SF patch # 1382163 is a fairly simple patch to expose the Subversion
revision number to Python, both in the Py_GetBuildInfo() text, and in a
new Py_GetBuildNumber() C API function, and via a new sys.build_number
attribute.  This number is calculated from the output of "svn info" at
the top of the tree.  If not building Python from a Subversion checkout,
the old way of calculating buildno is kept.

(Of course, this change only gets the right number into the Unix build.
Someone else can hopefully add the necessary magic for Windows.  Patch
tested only on Linux though.)

Is there any interest in this patch?  I'm happy to commit it if there
are no objections.

-Barry

Python 2.5a0 (#41708, Dec 15 2005, 23:59:14) 
[GCC 3.3.6 (Gentoo 3.3.6, ssp-3.3.6-1.0, pie-8.7.8)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.build_number
41708
>>> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051216/15ebad1b/attachment-0001.pgp

From bcannon at gmail.com  Fri Dec 16 07:13:32 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Thu, 15 Dec 2005 22:13:32 -0800
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <1134710209.15629.6.camel@geddy.wooz.org>
References: <1134710209.15629.6.camel@geddy.wooz.org>
Message-ID: <bbaeab100512152213i61baeb1al45185f8445d93192@mail.gmail.com>

On 12/15/05, Barry Warsaw <barry at python.org> wrote:
> SF patch # 1382163 is a fairly simple patch to expose the Subversion
> revision number to Python, both in the Py_GetBuildInfo() text, and in a
> new Py_GetBuildNumber() C API function, and via a new sys.build_number
> attribute.  This number is calculated from the output of "svn info" at
> the top of the tree.  If not building Python from a Subversion checkout,
> the old way of calculating buildno is kept.
>
> (Of course, this change only gets the right number into the Unix build.
> Someone else can hopefully add the necessary magic for Windows.  Patch
> tested only on Linux though.)
>
> Is there any interest in this patch?  I'm happy to commit it if there
> are no objections.
>
> -Barry
>
> Python 2.5a0 (#41708, Dec 15 2005, 23:59:14)
> [GCC 3.3.6 (Gentoo 3.3.6, ssp-3.3.6-1.0, pie-8.7.8)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys
> >>> sys.build_number
> 41708

+0

It makes it easy to request the revision number from people who submit
patches and bugs.  But I also don't find it vital since running ``svn
info .``.

-Brett

From pje at telecommunity.com  Fri Dec 16 07:38:53 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 16 Dec 2005 01:38:53 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <1134710209.15629.6.camel@geddy.wooz.org>
Message-ID: <5.1.1.6.0.20051216013250.01f78bc0@mail.telecommunity.com>

At 12:16 AM 12/16/2005 -0500, Barry Warsaw wrote:
>SF patch # 1382163 is a fairly simple patch to expose the Subversion
>revision number to Python, both in the Py_GetBuildInfo() text, and in a
>new Py_GetBuildNumber() C API function, and via a new sys.build_number
>attribute.  This number is calculated from the output of "svn info" at
>the top of the tree.

FYI, this is not the true revision number; it's only the revision number in 
which the directory was last modified, not the latest revision number 
within the tree.

You probably want to do a bit more processing of "svn info -R", or else 
parse the .svn/entries files yourself.  There's some Python code that does 
this at:

http://svn.python.org/projects/sandbox/trunk/setuptools/setuptools/command/egg_info.py

Specifically, see the 'get_svn_revision' method for an example of 
processing .svn/entries files to find out the highest committed revision 
number in a source tree.


From wlangner at googlemail.com  Fri Dec 16 11:13:41 2005
From: wlangner at googlemail.com (Wolfgang)
Date: Fri, 16 Dec 2005 11:13:41 +0100
Subject: [Python-Dev] PEP 8 updates/clarifications, function/method	style
In-Reply-To: <1134684098.10290.90.camel@geddy.wooz.org>
References: <20051214113549.swbgg3zat8g0wo4k@login.werra.lunarpages.com>	
	<bbaeab100512141325h7440e0dane9e8cd34cc670a59@mail.gmail.com>	
	<43A143D5.1050307@googlemail.com>
	<1134684098.10290.90.camel@geddy.wooz.org>
Message-ID: <43A29355.2000301@googlemail.com>

Hi,

>> I hope new stuff will follow only one naming style. And now we should
>> (or one person :-) should) decide which one.
> 
> I guess my point boils down to, we already did decide 4 years ago.
> Let's stick with what we've got.

Ok, then let's stick with lower_case and check this if new libraries were 
added to std lib.

Possible add a note to not forbid the use of cameCase in external libraries ?


bye by Wolfgang

From arigo at tunes.org  Fri Dec 16 11:20:24 2005
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 16 Dec 2005 11:20:24 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <1134710209.15629.6.camel@geddy.wooz.org>
References: <1134710209.15629.6.camel@geddy.wooz.org>
Message-ID: <20051216102024.GA18656@code1.codespeak.net>

Hi Barry,

On Fri, Dec 16, 2005 at 12:16:49AM -0500, Barry Warsaw wrote:
> SF patch # 1382163 is a fairly simple patch to expose the Subversion
> revision number to Python, both in the Py_GetBuildInfo() text, and in a
> new Py_GetBuildNumber() C API function, and via a new sys.build_number
> attribute.

I have a minor concern about people starting to use sys.build_number to
check for features in their programs, instead of using sys.version_info
or hasattr() or whatever is relevant -- e.g. because it seems to them
that comparing a single number is easier than a tuple.  The problem is
that this build number would most likely have no meaning in non-CPython
implementations.

What about having instead:

    sys.build_info = ("CPython", <svn rev>, "trunk")

This would make it clear that it's the CPython svn rev number, and it
could possibly be used to distinguish between branches, too, which the
revision number alone cannot do.  ("trunk" is the last part of the path
returned by "svn info".)

Of course, what I'm trying to sneak in here is that it may be a good
occasion to introduce an official way to determine which Python
implementation the program is running on top of -- something more
immediate than the sys.platform=="java" occasionally used in the test
suite to look for Jython.  (I know programs should not depend on this in
general; I'm more thinking about places like the test suite.)


A bientot,

Armin

From fredrik at pythonware.com  Fri Dec 16 11:30:27 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri, 16 Dec 2005 11:30:27 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
References: <1134710209.15629.6.camel@geddy.wooz.org>
	<20051216102024.GA18656@code1.codespeak.net>
Message-ID: <dnu505$7n7$1@sea.gmane.org>

Armin Rigo wrote:

> What about having instead:
>
>     sys.build_info = ("CPython", <svn rev>, "trunk")
>
> This would make it clear that it's the CPython svn rev number, and it
> could possibly be used to distinguish between branches, too, which the
> revision number alone cannot do.  ("trunk" is the last part of the path
> returned by "svn info".)

+1

</F>




From skip at pobox.com  Fri Dec 16 12:02:19 2005
From: skip at pobox.com (skip@pobox.com)
Date: Fri, 16 Dec 2005 05:02:19 -0600
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <20051216102024.GA18656@code1.codespeak.net>
References: <1134710209.15629.6.camel@geddy.wooz.org>
	<20051216102024.GA18656@code1.codespeak.net>
Message-ID: <17314.40635.783451.278318@montanaro.dyndns.org>


    Armin> What about having instead:

    Armin>     sys.build_info = ("CPython", <svn rev>, "trunk")

    Armin> This would make it clear that it's the CPython svn rev number,
    Armin> and it could possibly be used to distinguish between branches,
    Armin> too, which the revision number alone cannot do.  ("trunk" is the
    Armin> last part of the path returned by "svn info".)

What was your directory path when you got "trunk" from "svn info"?  In my
sandbox I get this:

    % pwd
    /Users/skip/src/python-svn/trunk
    % svn info
    Path: .
    URL: svn+ssh://pythondev at svn.python.org/python/trunk
    Repository UUID: 6015fed2-1504-0410-9fe1-9d1591cc4771
    Revision: 41708
    Node Kind: directory
    Schedule: normal
    Last Changed Author: trent.mick
    Last Changed Rev: 41708
    Last Changed Date: 2005-12-15 16:16:49 -0600 (Thu, 15 Dec 2005)
    Properties Last Updated: 2005-12-15 20:45:15 -0600 (Thu, 15 Dec 2005)

Did you mean the last part of the URL?

Skip

From barry at python.org  Fri Dec 16 14:29:54 2005
From: barry at python.org (Barry Warsaw)
Date: Fri, 16 Dec 2005 08:29:54 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <bbaeab100512152213i61baeb1al45185f8445d93192@mail.gmail.com>
References: <1134710209.15629.6.camel@geddy.wooz.org>
	<bbaeab100512152213i61baeb1al45185f8445d93192@mail.gmail.com>
Message-ID: <1134739794.15629.11.camel@geddy.wooz.org>

On Thu, 2005-12-15 at 22:13 -0800, Brett Cannon wrote:

> +0
> 
> It makes it easy to request the revision number from people who submit
> patches and bugs.  But I also don't find it vital since running ``svn
> info .``.

That's really more the point, that you can talk about a specific svn
revision easily from a built Python, even if that's installed and moved
to a location apart from the svn working directory.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051216/94475472/attachment.pgp

From barry at python.org  Fri Dec 16 14:35:41 2005
From: barry at python.org (Barry Warsaw)
Date: Fri, 16 Dec 2005 08:35:41 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <5.1.1.6.0.20051216013250.01f78bc0@mail.telecommunity.com>
References: <5.1.1.6.0.20051216013250.01f78bc0@mail.telecommunity.com>
Message-ID: <1134740141.15629.17.camel@geddy.wooz.org>

On Fri, 2005-12-16 at 01:38 -0500, Phillip J. Eby wrote:

> FYI, this is not the true revision number; it's only the revision number in 
> which the directory was last modified, not the latest revision number 
> within the tree.

Yep, I know.  At work, we've gone through many iterations of this,
including essentially what you do in setuptools.

I opted against that for several reasons.  First, I wanted to keep the
patch as simple as possible.  Second, I didn't want to depend on Python
already being built (i.e. write a Python script to do this).  Third, I
think most Python developers will just svn up at the top of the source
tree, then rebuild, rather than svn up some buried sub-tree, cd back to
the top and rebuild from there.  At least, that's how I generally work
with the Python tree.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051216/21a6e11c/attachment.pgp

From pje at telecommunity.com  Fri Dec 16 16:51:33 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 16 Dec 2005 10:51:33 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <1134740141.15629.17.camel@geddy.wooz.org>
References: <5.1.1.6.0.20051216013250.01f78bc0@mail.telecommunity.com>
	<5.1.1.6.0.20051216013250.01f78bc0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051216103912.01b0a5c8@mail.telecommunity.com>

At 08:35 AM 12/16/2005 -0500, Barry Warsaw wrote:
>On Fri, 2005-12-16 at 01:38 -0500, Phillip J. Eby wrote:
>
> > FYI, this is not the true revision number; it's only the revision 
> number in
> > which the directory was last modified, not the latest revision number
> > within the tree.
>
>Yep, I know.  At work, we've gone through many iterations of this,
>including essentially what you do in setuptools.
>
>I opted against that for several reasons.  First, I wanted to keep the
>patch as simple as possible.  Second, I didn't want to depend on Python
>already being built (i.e. write a Python script to do this).  Third, I
>think most Python developers will just svn up at the top of the source
>tree, then rebuild, rather than svn up some buried sub-tree, cd back to
>the top and rebuild from there.  At least, that's how I generally work
>with the Python tree.

Actually, the issue I was concerned about was that when you make a change 
to some file and commit it, the build number won't change unless you also 
svn up, and maybe not even then.  I never figured out how to get a good 
answer without reading the full "svn info -R" or the .svn/entries files.

Note that you can just use:

     svn info -R|grep '^Last Changed Rev'|sort -nr|head -1|cut -f 4 -d" "

To get the highest-numbered revision.  However, both this approach and 
yours will not deal with Subversion messages in non-English locales.  I 
discovered this with setuptools when I was using "svn info" when somebody 
reported that the text before the numbers is different in non-English locales.

After a bit of experimentation, here's a pipeline that gets the info 
directly from the entries files, without reliance on the language of svn 
info's output:

     find . -name entries | grep '\.svn/entries$' | xargs grep -h 
committed-rev \
          | cut -f2 -d'"' | sort -nr |head -1


From pje at telecommunity.com  Fri Dec 16 16:59:23 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 16 Dec 2005 10:59:23 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com
 >
Message-ID: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>

At 07:42 AM 12/16/2005 -0800, Michael Chermside wrote:
>Phillip writes:
> > FYI, this is not the true revision number; it's only the revision number in
> > which the directory was last modified, not the latest revision number
> > within the tree.
>
>Barry responds:
> > I opted against that for several reasons.  First, I wanted to keep the
> > patch as simple as possible.  Second, I didn't want to depend on Python
> > already being built [...] Third, I
> > think most Python developers will just svn up at the top of the source
> > tree, then rebuild, rather than svn up some buried sub-tree, cd back to
> > the top and rebuild from there.
>
>I agree with Barry. If you have done your "svn up" from anywhere other
>than the root, then you are building Python with a mix of revisions. I
>think that sys.build_number should produce undefined behavior when
>executing in a Python built from mixed revisions -- anyone doing this
>gets what they deserve.

The "Revision" from "svn info" isn't reliable; it doesn't actually relate 
to what version of code is in the subtree.  It can change when nothing has 
changed.

For example, in my setuptools checkout, I just did an "svn up", and the 
"Revision" is now 41708.  But *nothing* in the tree has changed since 
41701; this is simply the current highest SVN revision, repository 
wide.  When people change things in branches, the sandbox, PEPs, etc. - 
that number will change, even though nothing changed in the trunk.

I suggest that having an SVN revision number that changes when no code has 
actually changed is confusing and unreliable.  SVN does track the actual 
*changed* revision, it just takes a little more work to get it.


From arigo at tunes.org  Fri Dec 16 17:09:17 2005
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 16 Dec 2005 17:09:17 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <5.1.1.6.0.20051216103912.01b0a5c8@mail.telecommunity.com>
References: <5.1.1.6.0.20051216013250.01f78bc0@mail.telecommunity.com>
	<5.1.1.6.0.20051216013250.01f78bc0@mail.telecommunity.com>
	<5.1.1.6.0.20051216103912.01b0a5c8@mail.telecommunity.com>
Message-ID: <20051216160917.GA26696@code1.codespeak.net>

Hi Phillip,

On Fri, Dec 16, 2005 at 10:51:33AM -0500, Phillip J. Eby wrote:
>      svn info -R|grep '^Last Changed Rev'|sort -nr|head -1|cut -f 4 -d" "
> 
> To get the highest-numbered revision.  However, both this approach and 
> yours will not deal with Subversion messages in non-English locales.

The 'py' lib works around this problem by running "LC_ALL=C svn info".


A bientot,

Armin

From arigo at tunes.org  Fri Dec 16 17:11:06 2005
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 16 Dec 2005 17:11:06 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <17314.40635.783451.278318@montanaro.dyndns.org>
References: <1134710209.15629.6.camel@geddy.wooz.org>
	<20051216102024.GA18656@code1.codespeak.net>
	<17314.40635.783451.278318@montanaro.dyndns.org>
Message-ID: <20051216161106.GB26696@code1.codespeak.net>

Hi Skip,

On Fri, Dec 16, 2005 at 05:02:19AM -0600, skip at pobox.com wrote:
>     Armin> ("trunk" is the last part of the path returned by "svn info".)

> Did you mean the last part of the URL?

Yes, sorry.


Armin

From arigo at tunes.org  Fri Dec 16 17:17:28 2005
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 16 Dec 2005 17:17:28 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
References: <20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
Message-ID: <20051216161728.GC26696@code1.codespeak.net>

Hi Phillip,

On Fri, Dec 16, 2005 at 10:59:23AM -0500, Phillip J. Eby wrote:
> The "Revision" from "svn info" isn't reliable; it doesn't actually relate 
> to what version of code is in the subtree.  It can change when nothing has 
> changed.

Indeed, the patch should not use the "Revision" line but the "Last
Changed Rev" one.

> SVN does track the actual 
> *changed* revision, it just takes a little more work to get it.

Not if you're happy with "Last Changed Rev":

    LC_ALL=C svn info | grep -i "last changed rev" | cut -f 4 -d " "


A bientot,

Armin

From pje at telecommunity.com  Fri Dec 16 17:33:00 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 16 Dec 2005 11:33:00 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <20051216161728.GC26696@code1.codespeak.net>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>

At 05:17 PM 12/16/2005 +0100, Armin Rigo wrote:
>Hi Phillip,
>
>On Fri, Dec 16, 2005 at 10:59:23AM -0500, Phillip J. Eby wrote:
> > The "Revision" from "svn info" isn't reliable; it doesn't actually relate
> > to what version of code is in the subtree.  It can change when nothing has
> > changed.
>
>Indeed, the patch should not use the "Revision" line but the "Last
>Changed Rev" one.
>
> > SVN does track the actual
> > *changed* revision, it just takes a little more work to get it.
>
>Not if you're happy with "Last Changed Rev":
>
>     LC_ALL=C svn info | grep -i "last changed rev" | cut -f 4 -d " "

You left off the all-important "-R" from "svn info", and the "sort -nr | 
head -1" at the end.  The "Last Changed Rev" of the root is not necessarily 
the highest "Last Changed Rev", no matter how or where you update or check 
out.  Try it and see.


From jjl at pobox.com  Fri Dec 16 18:09:55 2005
From: jjl at pobox.com (John J Lee)
Date: Fri, 16 Dec 2005 17:09:55 +0000 (GMT Standard Time)
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
Message-ID: <Pine.WNT.4.64.0512161702540.4016@shaolin>

On Fri, 16 Dec 2005, Phillip J. Eby wrote:
[...to-and-fro re magic required to get a good SVN revision...]

Shouldn't the command 'svnversion' be used instead? -

http://svnbook.red-bean.com/en/1.1/re57.html


It's true that the output of this command does change with 'svn up', even 
if the update makes no changes to files under version control in your 
working copy.  It *seems* to be sane & reproducible once you've done a 
single svn up, though (and if there are no locally modified files, mixed 
checkouts etc., the version it reports will be a single revision number 
with no non-numeric characters).



John

From martin at v.loewis.de  Fri Dec 16 20:30:40 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 16 Dec 2005 20:30:40 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
Message-ID: <43A315E0.1010006@v.loewis.de>

Phillip J. Eby wrote:
> The "Revision" from "svn info" isn't reliable; it doesn't actually relate 
> to what version of code is in the subtree.  It can change when nothing has 
> changed.

That is not true. It does relate - it is the revision that was current
when "svn up" was last done. This *does* allow you to tell what changed
last, and it *does* allow to restore the precise sources (unless the
user only made a partial "svn up"). If some person tells you this
revision R, and you do 'svn up -rR', then you have precisely the same
sources as that person.

Regards,
Martin

From pje at telecommunity.com  Fri Dec 16 21:01:04 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 16 Dec 2005 15:01:04 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <43A315E0.1010006@v.loewis.de>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051216145513.03184690@mail.telecommunity.com>

At 08:30 PM 12/16/2005 +0100, Martin v. L?wis wrote:
>Phillip J. Eby wrote:
> > The "Revision" from "svn info" isn't reliable; it doesn't actually relate
> > to what version of code is in the subtree.  It can change when nothing has
> > changed.
>
>That is not true. It does relate - it is the revision that was current
>when "svn up" was last done.

But you can also have more than one revision number that represents the 
*exact same code*, with no changes at all.


>  This *does* allow you to tell what changed
>last,

It can also give you a false indicator of change, when nothing has in fact 
changed.  Don't believe me?  Check in a change to a sandbox project or a 
branch, and go see if the Python trunk still has the same "revision" 
afterwards.

I'm rather baffled as to why everyone seems so insistent on not using "Last 
Changed Rev" and "-R", given that the information is demonstrably a 1:1 
mapping with actual changes to the project, while "Revision" without -R is 
demonstrably *not* a 1:1 mapping with actual changes to the project.


From rivest at mit.edu  Fri Dec 16 16:25:02 2005
From: rivest at mit.edu (Ronald  L. Rivest)
Date: Fri, 16 Dec 2005 10:25:02 -0500
Subject: [Python-Dev]  hashlib - faster md5/sha, adds sha256/512 support
Message-ID: <6.2.5.6.2.20051216102116.02c32d60@mit.edu>

Hi --

I'm curious as to the status of upgrading cryptographic
hash function support in Python, now that md5 and sha1 are
both clearly broken (in terms of collision-resistance).

The consensus of researchers in this area (at least as
expressed at the NIST Hash Function Workshop 10/31/05),
is that SHA-256 is a good choice for the time being, but
that research should continue, and other alternatives may
arise from this research.  The larger SHA's also seem OK,
but I think will have less demand...

I'd like to see sha-256 supported in Python.  Has this
already happened (and I didn't notice) and/or will it
be happening soon?

Thanks!

         Cheers,
         Ron Rivest

P.S. Please cc your reply to me at rivest at mit.edu as well
      Thanks!


Ronald L. Rivest
Room 32-G692, Stata Center, MIT, Cambridge MA 02139
Tel 617-253-5880, Email <rivest at mit.edu>



From jeremy at alum.mit.edu  Fri Dec 16 21:13:00 2005
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Fri, 16 Dec 2005 15:13:00 -0500
Subject: [Python-Dev] hashlib - faster md5/sha, adds sha256/512 support
In-Reply-To: <6.2.5.6.2.20051216102116.02c32d60@mit.edu>
References: <6.2.5.6.2.20051216102116.02c32d60@mit.edu>
Message-ID: <e8bf7a530512161213x3a179affo51c81a472f0a5e5f@mail.gmail.com>

Python 2.5 will include sha-256 and sha-512.  It will be released
sometime next year.

Jeremy

On 12/16/05, Ronald  L. Rivest <rivest at mit.edu> wrote:
> Hi --
>
> I'm curious as to the status of upgrading cryptographic
> hash function support in Python, now that md5 and sha1 are
> both clearly broken (in terms of collision-resistance).
>
> The consensus of researchers in this area (at least as
> expressed at the NIST Hash Function Workshop 10/31/05),
> is that SHA-256 is a good choice for the time being, but
> that research should continue, and other alternatives may
> arise from this research.  The larger SHA's also seem OK,
> but I think will have less demand...
>
> I'd like to see sha-256 supported in Python.  Has this
> already happened (and I didn't notice) and/or will it
> be happening soon?
>
> Thanks!
>
>          Cheers,
>          Ron Rivest
>
> P.S. Please cc your reply to me at rivest at mit.edu as well
>       Thanks!
>
>
> Ronald L. Rivest
> Room 32-G692, Stata Center, MIT, Cambridge MA 02139
> Tel 617-253-5880, Email <rivest at mit.edu>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>

From tim.peters at gmail.com  Fri Dec 16 21:26:25 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Fri, 16 Dec 2005 15:26:25 -0500
Subject: [Python-Dev] hashlib - faster md5/sha, adds sha256/512 support
In-Reply-To: <6.2.5.6.2.20051216102116.02c32d60@mit.edu>
References: <6.2.5.6.2.20051216102116.02c32d60@mit.edu>
Message-ID: <1f7befae0512161226k524646f6j3e8e9a78378e2f5a@mail.gmail.com>

[Ronald  L. Rivest]
> I'm curious as to the status of upgrading cryptographic
> hash function support in Python, now that md5 and sha1 are
> both clearly broken (in terms of collision-resistance).
>
> The consensus of researchers in this area (at least as
> expressed at the NIST Hash Function Workshop 10/31/05),
> is that SHA-256 is a good choice for the time being, but
> that research should continue, and other alternatives may
> arise from this research.  The larger SHA's also seem OK,
> but I think will have less demand...
>
> I'd like to see sha-256 supported in Python.  Has this
> already happened (and I didn't notice) and/or will it
> be happening soon?

I'm gratified that you think highly enough of Python to ask ;-)

A new core `hashlib` module will be included in Python 2.5, but will
not be backported to older Python versions.  It includes new
implementations for SHA-224, -256, -384 and -512.  The code and tests
are already written, and can be gotten from Python's SVN trunk.

python-dev'ers:  I failed to find anything in the trunk's NEWS file
about this (neither about `hashlib`, nor about any of the specific new
hash functions).  It's not like it isn't newsworthy ;-)

From martin at v.loewis.de  Fri Dec 16 22:03:09 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 16 Dec 2005 22:03:09 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <5.1.1.6.0.20051216145513.03184690@mail.telecommunity.com>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216145513.03184690@mail.telecommunity.com>
Message-ID: <43A32B8D.4060800@v.loewis.de>

Phillip J. Eby wrote:
> But you can also have more than one revision number that represents the
> *exact same code*, with no changes at all.

That's correct. I don't see this as a problem - in particular not in
the context of the proposed patch.

The idea is that you can reliably tell what code base a certain
executable image originates from. With that patch, you can

> It can also give you a false indicator of change, when nothing has in
> fact changed.  Don't believe me?

I believe that the version number changes. It is a false indicator only
if you are unaware of that fact. To me, different version numbers don't
indicate different code bases, because I know how subversion works.

> I'm rather baffled as to why everyone seems so insistent on not using
> "Last Changed Rev" and "-R"

That's easy to tell: because it is expensive.

Regards,
Martin

From arigo at tunes.org  Fri Dec 16 22:16:03 2005
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 16 Dec 2005 22:16:03 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
Message-ID: <20051216211603.GA951@code1.codespeak.net>

Hi Phillip,

On Fri, Dec 16, 2005 at 11:33:00AM -0500, Phillip J. Eby wrote:
> >Not if you're happy with "Last Changed Rev":
> >
> >     LC_ALL=C svn info | grep -i "last changed rev" | cut -f 4 -d " "
> 
> You left off the all-important "-R" from "svn info", and the "sort -nr | 
> head -1" at the end.  The "Last Changed Rev" of the root is not necessarily 
> the highest "Last Changed Rev", no matter how or where you update or check 
> out.  Try it and see.

I was proposing this line as a slight extension of the one currently in
the SF patch.  In accordance with Martin I am still unconvinced that
'svn info -R' or more fancy tools are really useful here.

If you meant that the following situation is possible:

    trunk$  svn up
    At revision xxx.
    trunk$  svn info
    Last Changed Rev: 10000
    trunk$  cd Python
    trunk/python$  svn info
    Last Changed Rev: 10001

then I object.  As far as I can tell this is not possible.


A bientot,

Armin

From pje at telecommunity.com  Fri Dec 16 22:25:58 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 16 Dec 2005 16:25:58 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <43A32B8D.4060800@v.loewis.de>
References: <5.1.1.6.0.20051216145513.03184690@mail.telecommunity.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216145513.03184690@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051216161627.030ca518@mail.telecommunity.com>

At 10:03 PM 12/16/2005 +0100, Martin v. L?wis wrote:
>Phillip J. Eby wrote:
> > But you can also have more than one revision number that represents the
> > *exact same code*, with no changes at all.
>
>That's correct. I don't see this as a problem - in particular not in
>the context of the proposed patch.
>
>The idea is that you can reliably tell what code base a certain
>executable image originates from. With that patch, you can

Only if you do an "svn update" immediately after *every* "svn 
commit".  Otherwise, the code base reflected will be a version *before* 
your changes.  This is fragile, since not everyone will know (or remember!) 
to do this.


> > It can also give you a false indicator of change, when nothing has in
> > fact changed.  Don't believe me?
>
>I believe that the version number changes. It is a false indicator only
>if you are unaware of that fact. To me, different version numbers don't
>indicate different code bases, because I know how subversion works.
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Exactly my point.  The proposed mechanism relies on an intimate 
understanding of how subversion and its revision numbers work, making it 
unnecessarily fragile when used by Python developers and the community at 
large.


> > I'm rather baffled as to why everyone seems so insistent on not using
> > "Last Changed Rev" and "-R"
>
>That's easy to tell: because it is expensive.

I doubt that's the actual reason, but it seems like a bad reason in any 
case; it seems to me the applicable Zen should be "never is often better 
than *right* now".  :)

That is, if you're going to rely on a number that can be falsely high or 
falsely low depending on the detailed development practices of developers 
working on the trunk or anywhere else, it seems like wasted effort.  Trying 
to diagnose a problem with wrong information is worse than having *no* 
information.

Thus, I'm -1 on including a revision number that will be frequently wrong 
(high *or* low) in practice.  If it's too "expensive" to do it right, it's 
*definitely* too expensive to do it wrong.  :)


From pje at telecommunity.com  Fri Dec 16 22:34:23 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 16 Dec 2005 16:34:23 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <20051216211603.GA951@code1.codespeak.net>
References: <5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>

At 10:16 PM 12/16/2005 +0100, Armin Rigo wrote:
>Hi Phillip,
>
>On Fri, Dec 16, 2005 at 11:33:00AM -0500, Phillip J. Eby wrote:
> > >Not if you're happy with "Last Changed Rev":
> > >
> > >     LC_ALL=C svn info | grep -i "last changed rev" | cut -f 4 -d " "
> >
> > You left off the all-important "-R" from "svn info", and the "sort -nr |
> > head -1" at the end.  The "Last Changed Rev" of the root is not 
> necessarily
> > the highest "Last Changed Rev", no matter how or where you update or check
> > out.  Try it and see.
>
>I was proposing this line as a slight extension of the one currently in
>the SF patch.  In accordance with Martin I am still unconvinced that
>'svn info -R' or more fancy tools are really useful here.
>
>If you meant that the following situation is possible:
>
>     trunk$  svn up
>     At revision xxx.
>     trunk$  svn info
>     Last Changed Rev: 10000
>     trunk$  cd Python
>     trunk/python$  svn info
>     Last Changed Rev: 10001
>
>then I object.  As far as I can tell this is not possible.

It is indeed possible for a file's "Last Changed Rev" to exceed that of the 
directory that contains it.  I'm not sure what you object to, 
though.  These are simply the facts of how Subversion operates, so 
objecting to anybody but the Subversion developers won't help.  ;)

I have not found any way to establish a stable "revision number" for a 
directory tree in Subversion except by using -R and "Last Changed Rev" (or 
the equivalent scanning of .svn/entries files).  Through my experience 
working on setuptools in the sandbox, it is clearly possible to *commit* 
changes without affecting a directory's "Revision" number, and updating a 
directory can cause its "Revision" to advance even when there has been no 
change to the source.



From fredrik at pythonware.com  Fri Dec 16 22:53:02 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri, 16 Dec 2005 22:53:02 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
References: <5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com><5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com><20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com><5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com><5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<20051216211603.GA951@code1.codespeak.net>
	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>
Message-ID: <dnvd00$dul$1@sea.gmane.org>

Phillip J. Eby wrote:
> I have not found any way to establish a stable "revision number" for a
> directory tree in Subversion except by using -R and "Last Changed Rev" (or
> the equivalent scanning of .svn/entries files).  Through my experience
> working on setuptools in the sandbox, it is clearly possible to *commit*
> changes without affecting a directory's "Revision" number

it's also possible to modify files and rebuild.  your approach doesn't
address that.

> and updating a directory can cause its "Revision" to advance even when
> there has been no  change to the source.

sure, but is that really relevant?  checking out that revision will give you the
same code base, and it's not exactly difficult to figure out what's changed be-
tween two given versions...

fwiw, the official way to do this is to use svnversion:

    http://subversion.tigris.org/faq.html#version-value-in-source

(this also looks for local changes).

</F>




From pje at telecommunity.com  Fri Dec 16 23:11:31 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 16 Dec 2005 17:11:31 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <dnvd00$dul$1@sea.gmane.org>
References: <5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<20051216211603.GA951@code1.codespeak.net>
	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>

At 10:53 PM 12/16/2005 +0100, Fredrik Lundh wrote:

>fwiw, the official way to do this is to use svnversion:
>
>     http://subversion.tigris.org/faq.html#version-value-in-source
>
>(this also looks for local changes).

It looks like using 'svnversion -c . | cut -f2 -d":"' would get the 
most-recent committed version, plus the letter "M" if there are local 
changes.  That sounds like what we should be using.

That way, a build with local revisions would include "M", thus nicely 
addressing that issue as well.


From bcannon at gmail.com  Fri Dec 16 23:50:36 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Fri, 16 Dec 2005 14:50:36 -0800
Subject: [Python-Dev] hashlib - faster md5/sha, adds sha256/512 support
In-Reply-To: <1f7befae0512161226k524646f6j3e8e9a78378e2f5a@mail.gmail.com>
References: <6.2.5.6.2.20051216102116.02c32d60@mit.edu>
	<1f7befae0512161226k524646f6j3e8e9a78378e2f5a@mail.gmail.com>
Message-ID: <bbaeab100512161450j1f6b3a2fh4af13c1127c96a15@mail.gmail.com>

On 12/16/05, Tim Peters <tim.peters at gmail.com> wrote:
[SNIP]
> python-dev'ers:  I failed to find anything in the trunk's NEWS file
> about this (neither about `hashlib`, nor about any of the specific new
> hash functions).  It's not like it isn't newsworthy ;-)

I have fixed the faux pas and added an entry.

-Brett

From martin at v.loewis.de  Sat Dec 17 10:35:59 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 17 Dec 2005 10:35:59 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <5.1.1.6.0.20051216161627.030ca518@mail.telecommunity.com>
References: <5.1.1.6.0.20051216145513.03184690@mail.telecommunity.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216145513.03184690@mail.telecommunity.com>
	<5.1.1.6.0.20051216161627.030ca518@mail.telecommunity.com>
Message-ID: <43A3DBFF.60103@v.loewis.de>

Phillip J. Eby wrote:
> Only if you do an "svn update" immediately after *every* "svn commit". 
> Otherwise, the code base reflected will be a version *before* your
> changes.  This is fragile, since not everyone will know (or remember!)
> to do this.

That's true. It would be fairly reliable only for people without
commit access (although they still could checkout different revisions
for, say, Objects and Modules).

A revision number can't take the fact into account that you may
have local modifications, anyway.

> Thus, I'm -1 on including a revision number that will be frequently
> wrong (high *or* low) in practice.  If it's too "expensive" to do it
> right, it's *definitely* too expensive to do it wrong.  :)

Ok, understood. I'm -0 for this patch, because of the subtleties.
I think I would be -1 for a patch that noticably increases the
build time just to get some "better" single-revision number:
that *still* won't tell you what precise sources had been used
to build the binary, as different files simply can be on different
revisions, and no single number, in whatever way computed, can
give you full information.

Regards,
Martin

From martin at v.loewis.de  Sat Dec 17 11:32:54 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 17 Dec 2005 11:32:54 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <20051216211603.GA951@code1.codespeak.net>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<20051216211603.GA951@code1.codespeak.net>
Message-ID: <43A3E956.1080604@v.loewis.de>

Armin Rigo wrote:
> If you meant that the following situation is possible:
> 
>     trunk$  svn up
>     At revision xxx.
>     trunk$  svn info
>     Last Changed Rev: 10000
>     trunk$  cd Python
>     trunk/python$  svn info
>     Last Changed Rev: 10001
> 
> then I object.  As far as I can tell this is not possible.

It is possible:
svn up -r 10000
cd Python
svn up -r 10001

(assuming Python changed in 10001).

Regards,
Martin

From arigo at tunes.org  Sat Dec 17 11:43:34 2005
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 17 Dec 2005 11:43:34 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <43A3E956.1080604@v.loewis.de>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<20051216211603.GA951@code1.codespeak.net>
	<43A3E956.1080604@v.loewis.de>
Message-ID: <20051217104334.GA18964@code1.codespeak.net>

Hi Martin,

On Sat, Dec 17, 2005 at 11:32:54AM +0100, "Martin v. L?wis" wrote:
> >     trunk$  svn up
> >     At revision xxx.
> >     trunk$  svn info
> >     Last Changed Rev: 10000
> >     trunk$  cd Python
> >     trunk/python$  svn info
> >     Last Changed Rev: 10001
> 
> It is possible:
> svn up -r 10000
> cd Python
> svn up -r 10001
> 
> (assuming Python changed in 10001).

No, I know about this.  I meant -- obviously, I'd have expected... --
precisely the sequence of commands that my example shows: the operation
of doing only a plain 'svn up' in the root directory and nothing more --
which is what most people should do -- cannot leave a subdirectory's
Last Changed Rev higher than the root's.


A bientot,

Armin

From martin at v.loewis.de  Sat Dec 17 13:00:27 2005
From: martin at v.loewis.de (martin@v.loewis.de)
Date: Sat, 17 Dec 2005 13:00:27 +0100
Subject: [Python-Dev] Test branch for ssize_t changes
Message-ID: <1134820827.43a3fddb2ad05@domainfactory-webmail.de>



I just created a branch for the ssize_t changes I had been working on
for a while. I hope to follow up with a PEP quickly.

Regards,
Martin



From mwh at python.net  Sat Dec 17 17:09:14 2005
From: mwh at python.net (Michael Hudson)
Date: Sat, 17 Dec 2005 16:09:14 +0000
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <43A3DBFF.60103@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Sat,
	17 Dec 2005 10:35:59 +0100")
References: <5.1.1.6.0.20051216145513.03184690@mail.telecommunity.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216145513.03184690@mail.telecommunity.com>
	<5.1.1.6.0.20051216161627.030ca518@mail.telecommunity.com>
	<43A3DBFF.60103@v.loewis.de>
Message-ID: <2mhd974set.fsf@starship.python.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> that *still* won't tell you what precise sources had been used
> to build the binary, as different files simply can be on different
> revisions, and no single number, in whatever way computed, can
> give you full information.

A tangential point is that presumably source distributions will be
built from an 'svn export' and thus won't have any revision
information in, which in turns means that a Python built from such a
distribution will not have any revision information.  This seems
rather far from ideal, as I'd guess most Pythons (e.g. Debian's) are
built this way. For a different approach, would it be possible to have
a subversion trigger put the revision number into some file in the
repository?

Cheers,
mwh

-- 
  On the other hand, the following areas are subject to boycott
  in reaction to the rampant impurity of design or execution, as
  determined after a period of study, in no particular order:
    ...                              http://www.naggum.no/profile.html

From martin at v.loewis.de  Sat Dec 17 17:46:20 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 17 Dec 2005 17:46:20 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <20051217104334.GA18964@code1.codespeak.net>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<20051216211603.GA951@code1.codespeak.net>
	<43A3E956.1080604@v.loewis.de>
	<20051217104334.GA18964@code1.codespeak.net>
Message-ID: <43A440DC.4000902@v.loewis.de>

Armin Rigo wrote:
> No, I know about this.  I meant -- obviously, I'd have expected... --
> precisely the sequence of commands that my example shows:

I still managed to miss that point :-( My fault, you are right.

Regards,
Martin

From martin at v.loewis.de  Sat Dec 17 18:29:27 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 17 Dec 2005 18:29:27 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <2mhd974set.fsf@starship.python.net>
References: <5.1.1.6.0.20051216145513.03184690@mail.telecommunity.com>	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>	<5.1.1.6.0.20051216145513.03184690@mail.telecommunity.com>	<5.1.1.6.0.20051216161627.030ca518@mail.telecommunity.com>	<43A3DBFF.60103@v.loewis.de>
	<2mhd974set.fsf@starship.python.net>
Message-ID: <43A44AF7.1020604@v.loewis.de>

Michael Hudson wrote:
> For a different approach, would it be possible to have
> a subversion trigger put the revision number into some file in the
> repository?

Not easily, to my knowledge. Assuming that the export will be made
from a tag, the revision at which the tag was created (along with
the path of the tag) could be added through a wrapper script to
perform the export.

Of course, if the policy to export from a tag is followed, *just*
having the tag name is also sufficient.

Regards,
Martin

From barry at python.org  Sat Dec 17 22:24:17 2005
From: barry at python.org (Barry Warsaw)
Date: Sat, 17 Dec 2005 16:24:17 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>
References: <5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<20051216211603.GA951@code1.codespeak.net>
	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>
	<5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>
Message-ID: <1134854657.10350.5.camel@geddy.wooz.org>

On Fri, 2005-12-16 at 17:11 -0500, Phillip J. Eby wrote:

> It looks like using 'svnversion -c . | cut -f2 -d":"' would get the 
> most-recent committed version, plus the letter "M" if there are local 
> changes.  That sounds like what we should be using.
> 
> That way, a build with local revisions would include "M", thus nicely 
> addressing that issue as well.

I didn't know about svnversion, but that does seem like the right thing
to use.  One downside is that it can take a long time on a big tree, but
in my own limited testing, that doesn't seem like a practical concern
for the Python source checkout.

AFAICT, the reason to use -c is so that changes outside the Python
source tree (i.e. in the sandbox) won't show up in Python's build
number.  That's fine although I wouldn't mind leaving off the -c since
you'll still get the same snapshot of code from a revisioned checkout
either way, and that's my primary interest.

Because the 'M' can show up in the build number (and is useful
information), then I'll change the C API and sys attribute to be a
string instead of an int.

Other than that, it sounds like this is a generally acceptable change to
make to the build process, so I'll make the above modifications and
commit it.

Thanks everyone,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051217/27251dd3/attachment.pgp

From martin at v.loewis.de  Sat Dec 17 23:48:38 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 17 Dec 2005 23:48:38 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <1134854657.10350.5.camel@geddy.wooz.org>
References: <5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>	<20051216211603.GA951@code1.codespeak.net>	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>	<5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>
	<1134854657.10350.5.camel@geddy.wooz.org>
Message-ID: <43A495C6.5040509@v.loewis.de>

Barry Warsaw wrote:
> AFAICT, the reason to use -c is so that changes outside the Python
> source tree (i.e. in the sandbox) won't show up in Python's build
> number.  That's fine although I wouldn't mind leaving off the -c since
> you'll still get the same snapshot of code from a revisioned checkout
> either way, and that's my primary interest.

I think the -c should be omitted, as should be any attempts to cut(1)
this information. It's there for a reason.

Regards,
Martin

From steven.bethard at gmail.com  Sun Dec 18 00:13:05 2005
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sat, 17 Dec 2005 16:13:05 -0700
Subject: [Python-Dev] DRAFT: python-dev summary for 2005-11-16 to 2005-11-31
Message-ID: <d11dcfba0512171513s5ac03353k8d901adcb901bfd1@mail.gmail.com>

Here's the summary for the first half of November -- sorry for the bit
of a delay.  As always, let me or Tony know if you have any
corrections!


=====================
Summary Announcements
=====================

--------------------------------------
Reminder: Python is now on Subversion!
--------------------------------------

Don't forget that the Python source code is now hosted on
svn.python.org as a Subversion (rather than CVS) repository.

Note that because of the way the subversion conversion was done,
by-date revision specifications for dates prior to the switchover
won't work.  To work around this, you can use svn diff (find the
changes since some date), svn up (check out revision a some date), and
svn annotate (aka svn blame).

Removing the CVS repository from sourceforge isn't possible without
hacks (as a result of their "open source never goes away" policy). 
However, it's no longer available from the project page, and the
repository is now filled with files pointing people to the new
repository.

Contributing threads:

- `Is some magic required to check out new files from svn?
<http://mail.python.org/pipermail/python-dev/2005-November/058168.html>`__
- `svn diff -r {2001-01-01}
<http://mail.python.org/pipermail/python-dev/2005-November/058269.html>`__
- `CVS repository mostly closed now
<http://mail.python.org/pipermail/python-dev/2005-November/058348.html>`__

[TAM]

=========
Summaries
=========

----------------------------
Memory management in the AST
----------------------------

Thomas Lee's attempt to implement `PEP 341`_ brought up some issues
about working with the new AST code. Because the AST code used its own
custom objects instead of PyObjects, it also introduced its own set of
allocation/deallocation functions instead of the existing Py_INCREF
and Py_DECREF. There was some discussion about how best to simplify
the scheme, with the two main suggestions being:

(1) Convert all AST objects into PyObjects so Py_INCREF and Py_DECREF work
(2) Create an arena API, where objects are added to the arena and then
can be freed in one shot when the arena is freed

Neal Norwitz presented an example from the current AST code using the
various asdl_*_free functions, and he, Greg Ewing and Martin v. L?wis
compared how the code would look with the various API suggestions. 
While using ref-counting had the benefit of being consistent with the
rest of Python, there were still some who felt that the arena API
would simplify things enough to make the extra learning curve
worthwhile.  It seemed likely that branches or patches for the various
APIs would appear shortly.

While the C API is still undergoing these changes, and thus the Python
API is still a ways off, a few implementations for the Python API were
suggested.  If the AST code ends up using PyObjects, these could be
passed directly to Python code, though they would probably have to be
immutable.  Brett Cannon suggested that another route would be a
simple PyString marshalling like the current parser module, so that
Python code and C code would never share the same objects.

.. _PEP 341: http://www.python.org/peps/pep-0341.html

Contributing threads:

- `Memory management in the AST parser &amp; compiler
<http://mail.python.org/pipermail/python-dev/2005-November/058156.html>`__
- `ast status, memory leaks, etc
<http://mail.python.org/pipermail/python-dev/2005-November/058220.html>`__
- `a Python interface for the AST (WAS: DRAFT: python-dev...)
<http://mail.python.org/pipermail/python-dev/2005-November/058294.html>`__

[SJB]

-----------------------
Profilers in the stdlib
-----------------------

Armin Rigo summarised the current Python profiler situation, which
includes profile.Profile (ages-old, slow, pure Python profiler with
limited support for profiling C calls), hotshot (Python 2.2+, faster
than profile.py, but very slow to convert the log file to the
pstats.Stats format, possibly inaccurate, doesn't know about C calls),
and `lsprof`_ (Brett Rosen, Ted Czotter, Michael Hudson, Armin Rigo;
doesn't support C calls, incompatible interface with
profile.py/hotshot, can record detailed stats about children).  He
suggested that lsprof be added to the standard library, keeping
profile.py as a pure Python implementation and replacing hotshot with
lsprof.

There was concern about maintenence of the library; however, since
Armin and Michael are core developers, this seems covered.  Martin
suggested that lsprof be distributed separately for some time, and
then included when it is more mature.  Many people were concerned
about having so many profilers included (with the preference for a
single profiler that would suit beginners, since advanced users can
easily install third-party modules, which could be referenced in the
documentation).

Tim Peters explained that the aim of hotshot wasn't to reduce total
time overhead, but to be less disruptive (than profile.py) to the code
being profiled, while that code is running, via tiny little C
functions that avoid memory allocation/deallocation.  Hotshot can do
much more than the minimalistic documentation says (e.g. it could be
used as the basis of a tracing tool to debug software, to measure test
coverage); you won't find them discussed in the documentation, which
makes user experience mostly negative, but you do find them in Tim's
e-mails.

Discussion centered around whether lsprof should be added to the
standard distribution, and whether hotshot and/or profile.py should be
removed.  Armin indicated that he favours removing hotshot, adding
lsprof, which would be added as "cProfile" (c.f cPickle/Pickle,
cStringIO/StringIO), and possibly rewriting profile.py as a pure
Python version of lsprof.

Floris Bruynooghe (for Google's Summer of Code) wrote a `replacement
for profile.py`_ that uses hotshot directory.  This replacement didn't
fix the problems with hotshot, but did result in pstats loading
hotshot data 30% faster, and would mean that profile.py could be
removed.

There was a little debate about whether any profiler should even be
included in the standard library, but there were several people who
opined that it was an important 'battery'.  A few people also liked
the idea of adding a statistical profiler to the standard library at
some point (e.g. http://wingolog.org/archives/2005/10/28/profiling).

Aahz suggested that Armin write a PEP for this, which seems the likely
way that this will progress.

Contributing thread:

- `s/hotshot/lsprof
<http://mail.python.org/pipermail/python-dev/2005-November/058212.html>`__

 .. _lsprof: http://codespeak.net/svn/user/arigo/hack/misc/lsprof
 .. _replacement for profile.py:  http://savannah.nongnu.org/projects/pyprof/

[TAM]

----------------------------------------------
The tp_free slot and multiple inheritance in C
----------------------------------------------

Travis Oliphant started a thread discussing a memory problem in some
new scipy core code where a huge number of objects were not being
freed.  Making the allocation code use malloc and free instead of
PyObject_New and PyObject_Del made these problems go away.  After an
intense discussion, Armin Rigo figured out that the problem arose in a
type that inherited both from int and from another scipy type.  The
tp_free slot of this type was being inherited from its second parent
(int) instead of its first parent (the scipy type), and thus
"deallocated" objects were put on the CPython free list of integers
instead of being freed.  It was unclear as to whether the code in
typeobject.c which made this decision could be "fixed", so Armin
suggested forcing the appropriate tp_alloc/tp_free functions in the
static types instead.

Contributing threads:

- `Problems with the Python Memory Manager
<http://mail.python.org/pipermail/python-dev/2005-November/058155.html>`__
- `Problems with mro for dual inheritance in C [Was: Problems with the
Python Memory Manager]
<http://mail.python.org/pipermail/python-dev/2005-November/058323.html>`__

[SJB]

--------------------------------------
Patches for porting Python to a new OS
--------------------------------------

Ben Decker asked for some feedback on patches porting Python to
DOS/DJGPP.  This lead to a discussion of what the requirements for
accepting a porting patch were.  Guido made it clear that he wanted
porting patches included in Python whenever reasonable so that the
various obscure ports would be able to upgrade to new versions of
Python when they were released.  The basic conditions were that the
submission came from a reputable platform maintainer, and that if the
patches caused problems in future Python versions, the maintainer
would either need to update the patch appropriately, or have it
removed from Python.

Contributing thread:

- `Patch Req. # 1351020 &amp; 1351036: PythonD modifications
<http://mail.python.org/pipermail/python-dev/2005-November/058177.html>`__

[SJB]

---------------------------------------
Making StringIO behave more like a file
---------------------------------------

Walter D?rwald identified a number of situations where StringIO (but
not cStringIO) does not behave like a normal file:

- next() after close() raises StopIteration instead of ValueError
- isatty() after close() returns False instead of raising ValueError
- truncate() with a negative argument doesn't raise an IOError

These were determined to be bugs in StringIO and will likely be fixed
in an upcoming Python release.

Contributing threads:

- `Iterating a closed StringIO
<http://mail.python.org/pipermail/python-dev/2005-November/058187.html>`__
- `isatty() on closed StringIO (was: Iterating a closed StringIO)
<http://mail.python.org/pipermail/python-dev/2005-November/058198.html>`__
- `Another StringIO/cStringIO discrepancy
<http://mail.python.org/pipermail/python-dev/2005-November/058199.html>`__
- `isatty() on closed StringIO
<http://mail.python.org/pipermail/python-dev/2005-November/058207.html>`__

[SJB]

-----------------------------------
User-defined data for logging calls
-----------------------------------

Vinay Sajip explained that on numerous occasions, requests have been
made for the ability to easily add user-defined data to logging
events. For example, a multi-threaded server application may want to
output specific information to a particular server thread (e.g. the
identity of the client, specific protocol options for the client
connection).

While this is currently possible, you have to subclass the Logger
class and override its makeRecord method to put custom attributes in
the LogRecord; the approach is usable but requires more work than
necessary.

Vinay proposed a simpler way of achieving the same result, which
requires use of an additional optional keyword argument ("extra") in
logging calls.  The "extra" argument will be passed to
Logger.makeRecord, which extend the logRecord's __dict__ with this
argument; however, if any of the keys are already present (values
calculated by the logging package), then a KeyError will be raised.

Contributing thread:

- `Proposed additional keyword argument in logging calls
<http://mail.python.org/pipermail/python-dev/2005-November/058286.html>`__

[TAM]

-------------------------------------
Updating urlparse to support RFC 3986
-------------------------------------

Paul Jimenez complained that urlparse uses a table of url schemes to
determine whether a protocol (e.g. http or ftp) supports specifying a
username and password in the url (e.g. https://user:pass at host:port). 
He suggested that all protocols should be capable of using this
format.

Guido pointed out that the main purpose of urlparse is to be
RFC-compliant.  Paul explained that the current code is valid
according to `RFC 1808`_ (1995-1998), but that this was superceded by
`RFC 2396`_ (1998-2004) and `RFC 3986`_ (2005-).  Guido was convinced,
and asked for a new API (for backwards compatibility) and a patch to
be submitted via sourceforge.

Contributing thread:

- `urlparse brokenness
<http://mail.python.org/pipermail/python-dev/2005-November/058301.html>`__

 .. _RFC 1808: http://www.ietf.org/rfc/rfc1808.txt
 .. _RFC 2396: http://www.ietf.org/rfc/rfc2396.txt
 .. _RFC 3986: http://www.ietf.org/rfc/rfc3986.txt

[TAM]

---------------------------------------------
Magic methods on the instance and on the type
---------------------------------------------

Nick Coghlan pointed out that the current semantics of `PEP 343`_ look
up methods on the instance instead of on the type, and noted that
slots are generally invoked as ``type(obj).__slot__(obj)`` instead. 
Guido explained that in general, using ``__xxxx__`` methods in an
undocumented way (e.g. relying on them being looked up in the
instance) was not supported, and code relying on that could be
expected to break if the ``__xxxx__`` method was ever upgraded to a
slot.  So, it was okay that the `PEP 343`_ support looked up methods
on the instance, but anyone depending on this behavior was asking for
trouble.

.. _PEP 343: http://www.python.org/peps/pep-0343.html

Contributing thread:

- `Metaclass problem in the &quot;with&quot; statement semantics in
PEP 343 <http://mail.python.org/pipermail/python-dev/2005-November/058360.html>`__

[SJB]

----------------------------------
Releasing the GIL in the re module
----------------------------------

Duncan Grisby has a multi-threaded program that does a lot of complex
regular expression searching, and has trouble with threads blocking
because the GIL is not released while the re engine is running.  He
wanted to know whether there was any fundamental reason why the re
engine could not release the interpreter lock.

Fredrik Lundh pointed out that SRE can operate on anything that
implements the buffer interface.  This means that the objects that the
engine is accessing might be mutable, which could cause problems.

Several people suggested that a better solution would be using more
efficient regular expressions; Duncan explained that the expressions
are user-entered, which makes this difficult.

Eric Noyau put together a `patch to release the GIL` when the engine
performs a low level search, if (and only if) the object searched is a
[unicode] string.

 .. _patch to release the GIL: http://python.org/sf/1366311

Contributing threads:

- `(no subject)
<http://mail.python.org/pipermail/python-dev/2005-November/058316.html>`__
- `Re: Regular expressions
<http://mail.python.org/pipermail/python-dev/2005-November/058318.html>`__
- `SRE should release the GIL (was: no subject)
<http://mail.python.org/pipermail/python-dev/2005-November/058329.html>`__
- `Regular expressions
<http://mail.python.org/pipermail/python-dev/2005-November/058333.html>`__

[TAM]

===============
Skipped Threads
===============

- `str.dedent <http://mail.python.org/pipermail/python-dev/2005-November/058148.html>`__
- `Behavoir question.
<http://mail.python.org/pipermail/python-dev/2005-November/058149.html>`__
- `Conclusion: Event loops, PyOS_InputHook, and Tkinter
<http://mail.python.org/pipermail/python-dev/2005-November/058150.html>`__
- `DRAFT: python-dev Summary for 2005-09-16 to 2005-09-30
<http://mail.python.org/pipermail/python-dev/2005-November/058178.html>`__
- `DRAFT: python-dev Summary for 2005-10-01 to 2005-10-15
<http://mail.python.org/pipermail/python-dev/2005-November/058179.html>`__
- `DRAFT: python-dev Summary for 2005-10-16 to 2005-10-31
<http://mail.python.org/pipermail/python-dev/2005-November/058180.html>`__
- `Coroutines (PEP 342)
<http://mail.python.org/pipermail/python-dev/2005-November/058192.html>`__
- `Enjoy a week without me
<http://mail.python.org/pipermail/python-dev/2005-November/058209.html>`__
- `Weekly Python Patch/Bug Summary
<http://mail.python.org/pipermail/python-dev/2005-November/058210.html>`__
- `How to stay almost backwards compatible with all these new cool
features <http://mail.python.org/pipermail/python-dev/2005-November/058211.html>`__
- `test_cmd_line on Windows
<http://mail.python.org/pipermail/python-dev/2005-November/058271.html>`__
- `Fwd: [Python-checkins] commit of r41497 - python/trunk/Lib/test
<http://mail.python.org/pipermail/python-dev/2005-November/058272.html>`__
- `[Python-checkins] commit of r41497 -python/trunk/Lib/test
<http://mail.python.org/pipermail/python-dev/2005-November/058275.html>`__
- `DRAFT: python-dev Summary for 2005-11-01 through 2005-11-15
<http://mail.python.org/pipermail/python-dev/2005-November/058281.html>`__
- `something is wrong with test___all__
<http://mail.python.org/pipermail/python-dev/2005-November/058292.html>`__
- `PEP 302, PEP 338 and imp.getloader (was Re: a Python interface for
the AST (WAS: DRAFT: python-dev...)
<http://mail.python.org/pipermail/python-dev/2005-November/058300.html>`__
- `registering unicode codecs
<http://mail.python.org/pipermail/python-dev/2005-November/058324.html>`__
- `reference leaks
<http://mail.python.org/pipermail/python-dev/2005-November/058330.html>`__
- `Bug bz2.BZ2File(...).seek(0,2) + patch
<http://mail.python.org/pipermail/python-dev/2005-November/058334.html>`__
- `Python 3 <http://mail.python.org/pipermail/python-dev/2005-November/058346.html>`__
- `For Python 3k, drop default/implicit hash, and comparison
<http://mail.python.org/pipermail/python-dev/2005-November/058349.html>`__
- `Bug day this Sunday?
<http://mail.python.org/pipermail/python-dev/2005-November/058367.html>`__
- `Short-circuiting iterators
<http://mail.python.org/pipermail/python-dev/2005-November/058423.html>`__
- `Standalone email package in the sandbox
<http://mail.python.org/pipermail/python-dev/2005-November/058431.html>`__

From barry at python.org  Sun Dec 18 02:28:17 2005
From: barry at python.org (Barry Warsaw)
Date: Sat, 17 Dec 2005 20:28:17 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <43A495C6.5040509@v.loewis.de>
References: <5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<20051216211603.GA951@code1.codespeak.net>
	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>
	<5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>
	<1134854657.10350.5.camel@geddy.wooz.org>
	<43A495C6.5040509@v.loewis.de>
Message-ID: <1134869297.10353.6.camel@geddy.wooz.org>

On Sat, 2005-12-17 at 23:48 +0100, "Martin v. L?wis" wrote:
> Barry Warsaw wrote:
> > AFAICT, the reason to use -c is so that changes outside the Python
> > source tree (i.e. in the sandbox) won't show up in Python's build
> > number.  That's fine although I wouldn't mind leaving off the -c since
> > you'll still get the same snapshot of code from a revisioned checkout
> > either way, and that's my primary interest.
> 
> I think the -c should be omitted, as should be any attempts to cut(1)
> this information. It's there for a reason.

Done. r41744.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051217/0ee00fee/attachment.pgp

From nnorwitz at gmail.com  Sun Dec 18 06:41:39 2005
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sat, 17 Dec 2005 21:41:39 -0800
Subject: [Python-Dev] [Python-checkins] commit of r41497
	-python/trunk/Lib/test
In-Reply-To: <dluigb$2as$1@sea.gmane.org>
References: <20051122051745.B440A1E400B@bag.python.org>
	<ee2a432c0511212120r2a4429c9lbda2ead70a3156f6@mail.gmail.com>
	<dluigb$2as$1@sea.gmane.org>
Message-ID: <ee2a432c0512172141s670315bbhf6f0bb96acac1d87@mail.gmail.com>

On 11/21/05, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Neal Norwitz wrote:
>
> > I just checked in the modification below.  I'm not sure if this
> > behaviour is on purpose or by accident.

[ /f shows diff on linux and windows ]

I checked in a fix for this so float('0x3') should not work on any platform now.

n

From nnorwitz at gmail.com  Sun Dec 18 06:49:58 2005
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sat, 17 Dec 2005 21:49:58 -0800
Subject: [Python-Dev] ref leak in element tree/pyexpat
Message-ID: <ee2a432c0512172149i3543ee24o1a7c5e2db6d35677@mail.gmail.com>

I'm not sure where the problem is, but this code leaks a reference:
   parser = ET.XMLParser() ; parser.feed('<x>text</x>')

You need this to set it up:
  from xmlcore.etree import cElementTree as ET

This isn't a memory leak according to valgrind.

Also, I noticed several places where errors where being ignored.  What
is the procedure for the element tree?  Are we supposed to modify it
in the python SVN or treat it as read-only?

The patch below fixes a few small problems.  There were some others I
noticed, but didn't add to the patch.

n
--
Index: Modules/_elementtree.c
===================================================================
--- Modules/_elementtree.c      (revision 41747)
+++ Modules/_elementtree.c      (working copy)
@@ -905,6 +905,9 @@
     }

     args = PyTuple_New(2);
+    if (!args)
+       return NULL;
+
     Py_INCREF(self); PyTuple_SET_ITEM(args, 0, (PyObject*) self);
     Py_INCREF(tag);  PyTuple_SET_ITEM(args, 1, (PyObject*) tag);

@@ -1641,8 +1644,8 @@
         PyObject* node = (PyObject*) self->last;
         res = PyTuple_New(2);
         if (res) {
-            Py_INCREF(action); PyTuple_SET_ITEM(res, 0, (PyObject*) action);
-            Py_INCREF(node);   PyTuple_SET_ITEM(res, 1, (PyObject*) node);
+            Py_INCREF(action); PyTuple_SET_ITEM(res, 0, action);
+            Py_INCREF(node);   PyTuple_SET_ITEM(res, 1, node);
             PyList_Append(self->events, res);
             Py_DECREF(res);
         } else
@@ -2586,10 +2589,14 @@
 #endif

     m = Py_InitModule("_elementtree", _functions);
+    if (!m)
+       return;

     /* python glue code */

     g = PyDict_New();
+    if (!g)
+       return;

     PyDict_SetItemString(g, "__builtins__", PyEval_GetBuiltins());

From martin at v.loewis.de  Sun Dec 18 11:09:54 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 18 Dec 2005 11:09:54 +0100
Subject: [Python-Dev] Incorporation of zlib sources into Python subversion
Message-ID: <43A53572.9020802@v.loewis.de>

Thomas (Heller) and I have been discussing whether the zlib
module should become builtin, atleast on Win32 (i.e. part
of python25.dll). This would simplify py2exe, which then could
bootstrap extraction from the compressed file just with
pythonxy.dll (clearly, zlib.pyd cannot be *in* the compressed
file).

We currently don't do this, because the pythoncore.vcproj would
then not be buildable anymore unless you also have the right
version of zlib on disk.

To solve this, Thomas has proposed that the Python release could
incorporate a copy of zlib, primarily for use on Windows
(with the project files appropriately adjusted). I'm in favour
of such a change: the library is fairly small, and it would
not only simplify py2exe, but also simplify the build process.

Whether or not this copy of zlib would be integrated in the
Unix build process, in case where the system does not provide
a zlib, is a separate question.

Regards,
Martin

From martin at v.loewis.de  Sun Dec 18 11:34:35 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 18 Dec 2005 11:34:35 +0100
Subject: [Python-Dev] ref leak in element tree/pyexpat
In-Reply-To: <ee2a432c0512172149i3543ee24o1a7c5e2db6d35677@mail.gmail.com>
References: <ee2a432c0512172149i3543ee24o1a7c5e2db6d35677@mail.gmail.com>
Message-ID: <43A53B3B.20200@v.loewis.de>

Neal Norwitz wrote:
> Also, I noticed several places where errors where being ignored.  What
> is the procedure for the element tree?  Are we supposed to modify it
> in the python SVN or treat it as read-only?

You should add a bug report on sf.net/projects/python, and assign
that to Fredrik Lundh. Likewise, if you have a patch.

Regards,
Martin

P.S. In principle, Fredrik agreed that others could also do bug fixes
on that tree (but no new features), so checking the fix in would be
allowed also.

From fredrik at pythonware.com  Sun Dec 18 14:51:05 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 18 Dec 2005 14:51:05 +0100
Subject: [Python-Dev] ref leak in element tree/pyexpat
References: <ee2a432c0512172149i3543ee24o1a7c5e2db6d35677@mail.gmail.com>
Message-ID: <do3pgc$foe$1@sea.gmane.org>

Neal Norwitz wrote:

> I'm not sure where the problem is, but this code leaks a reference:
>    parser = ET.XMLParser() ; parser.feed('<x>text</x>')
>
> You need this to set it up:
>   from xmlcore.etree import cElementTree as ET
>
> This isn't a memory leak according to valgrind.

looks like it's stealing more None references than it should.  here's a more
isolated test case:

    tree = ET.TreeBuilder()
    tree.start("x")
    tree.data("text")
    tree.end("x")

if you remove *either* the data or the end call, the leak goes away.  I'll
take a look.

> Also, I noticed several places where errors where being ignored.  What
> is the procedure for the element tree?  Are we supposed to modify it
> in the python SVN or treat it as read-only?

the overall goal is to be able to say that Python X.Y.Z ships with Element-
Tree A.B.C.  to achieve this, the Python SVN should be treated as mostly
read-only -- with the exception of critical errors (crashes seen in the wild,
serious leaks, security issues, etc) and build/portability issues.

> The patch below fixes a few small problems.  There were some others I
> noticed, but didn't add to the patch.

thanks.  I'll fix these (and a few others) in the next "official" release.

</F>




From fredrik at pythonware.com  Sun Dec 18 16:07:52 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 18 Dec 2005 16:07:52 +0100
Subject: [Python-Dev] fresh checkout won't build
Message-ID: <do3u0b$r3n$1@sea.gmane.org>

$ svn up
At revision 41759.
$ make
...
Python/compile.c: In function `PyNode_Compile':
Python/compile.c:301: parse error before `mod'
Python/compile.c:302: `mod' undeclared (first use in this function)
Python/compile.c:302: (Each undeclared identifier is reported only once
Python/compile.c:302: for each function it appears in.)
make: *** [Python/compile.o] Error 1

$ make distclean; ./configure; make
Python/compile.c: In function `PyNode_Compile':
Python/compile.c:301: parse error before `mod'
Python/compile.c:302: `mod' undeclared (first use in this function)
Python/compile.c:302: (Each undeclared identifier is reported only once
Python/compile.c:302: for each function it appears in.)
make: *** [Python/compile.o] Error 1

what am I missing ?

(old linux, old linux, built just fine yesterday)

</F>




From fredrik at pythonware.com  Sun Dec 18 16:47:39 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 18 Dec 2005 16:47:39 +0100
Subject: [Python-Dev] fresh checkout won't build
References: <do3u0b$r3n$1@sea.gmane.org>
Message-ID: <do40at$17g$1@sea.gmane.org>

> $ svn up
> At revision 41759.
> $ make
> ...
> Python/compile.c: In function `PyNode_Compile':
> Python/compile.c:301: parse error before `mod'
> Python/compile.c:302: `mod' undeclared (first use in this function)
> Python/compile.c:302: (Each undeclared identifier is reported only once
> Python/compile.c:302: for each function it appears in.)
> make: *** [Python/compile.o] Error 1
>
> $ make distclean; ./configure; make
> Python/compile.c: In function `PyNode_Compile':
> Python/compile.c:301: parse error before `mod'
> Python/compile.c:302: `mod' undeclared (first use in this function)
> Python/compile.c:302: (Each undeclared identifier is reported only once
> Python/compile.c:302: for each function it appears in.)
> make: *** [Python/compile.o] Error 1
>
> what am I missing ?

a C++ compiler, obviously,  the arena code used C++ (C99?) constructs
in a couple of places.  I've checked in a fix.

</F>




From arigo at tunes.org  Sun Dec 18 18:58:08 2005
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 18 Dec 2005 18:58:08 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <1134869297.10353.6.camel@geddy.wooz.org>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<20051216211603.GA951@code1.codespeak.net>
	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>
	<5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>
	<1134854657.10350.5.camel@geddy.wooz.org>
	<43A495C6.5040509@v.loewis.de>
	<1134869297.10353.6.camel@geddy.wooz.org>
Message-ID: <20051218175808.GA17467@code1.codespeak.net>

Hi Barry,

On Sat, Dec 17, 2005 at 08:28:17PM -0500, Barry Warsaw wrote:
> Done. r41744.

Doesn't appear to work for me: sys.build_number receives the value from
the buildno.  Looking at the Makefile, the reason is that I'm building
CPython in a separate directory (running '/some/path/configure; make').

Running 'svnversion .' by hand is quite fast if the whole tree of files
is in the cache.  My guess is that if you do 'svn up; make' then the
tree will indeed be in the cache, so the extra build time shouldn't be
noticeable in this common case (unless you are low on RAM).

Do we have any plan to make sys.build_number meaningful in the releases
as well (generally compiled from an svn export, as Michael pointed out),
or are we happy with a broken number in this case?

Should I propose / check-in a patch to expose sys.build_info instead
("CPython", "41761", "trunk"), as this got positive feedback so far?
It's also less surprizing than the current sys.build_number, which is a
string despite its name.


A bientot,

Armin

From fredrik at pythonware.com  Sun Dec 18 19:08:33 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 18 Dec 2005 19:08:33 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com><20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com><5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com><5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com><20051216211603.GA951@code1.codespeak.net><5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com><5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com><1134854657.10350.5.camel@geddy.wooz.org><43A495C6.5040509@v.loewis.de><1134869297.10353.6.camel@geddy.wooz.org>
	<20051218175808.GA17467@code1.codespeak.net>
Message-ID: <do48j3$onr$1@sea.gmane.org>

Armin Rigo wrote:

> Should I propose / check-in a patch to expose sys.build_info instead
> ("CPython", "41761", "trunk"), as this got positive feedback so far?
> It's also less surprizing than the current sys.build_number, which is a
> string despite its name.

fwiw, I'm still +1 on that.

</F>




From martin at v.loewis.de  Sun Dec 18 19:19:53 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 18 Dec 2005 19:19:53 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <20051218175808.GA17467@code1.codespeak.net>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<20051216211603.GA951@code1.codespeak.net>
	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>
	<5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>
	<1134854657.10350.5.camel@geddy.wooz.org>
	<43A495C6.5040509@v.loewis.de>
	<1134869297.10353.6.camel@geddy.wooz.org>
	<20051218175808.GA17467@code1.codespeak.net>
Message-ID: <43A5A849.1080801@v.loewis.de>

Armin Rigo wrote:
> Do we have any plan to make sys.build_number meaningful in the releases
> as well (generally compiled from an svn export, as Michael pointed out),
> or are we happy with a broken number in this case?

I'm actually a bit confused that Barry changed the meaning of "build
number" for that feature. The build number was meant to count builds,
not revisions (whether this is a useful feature or not).

So I would argue that it is "broken" when it is the result of
svnversion, and "good" when it actually counts builds.

It stopped counting builds on Windows quite some time ago; perhaps it
is best to drop the build number entirely?

> Should I propose / check-in a patch to expose sys.build_info instead
> ("CPython", "41761", "trunk"), as this got positive feedback so far?
> It's also less surprizing than the current sys.build_number, which is a
> string despite its name.

Propose first. I have the feeling that the feature will change forth
and back if everybody gets to say something. I would call it
sys.svnversion (because that's what it is).

What to put in in case of tagged builds is then yet another question.

Regards,
Martin

From reinhold-birkenfeld-nospam at wolke7.net  Sun Dec 18 19:28:31 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Sun, 18 Dec 2005 19:28:31 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <43A5A849.1080801@v.loewis.de>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>	<20051216211603.GA951@code1.codespeak.net>	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>	<5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>	<1134854657.10350.5.camel@geddy.wooz.org>	<43A495C6.5040509@v.loewis.de>	<1134869297.10353.6.camel@geddy.wooz.org>	<20051218175808.GA17467@code1.codespeak.net>
	<43A5A849.1080801@v.loewis.de>
Message-ID: <do49of$rpk$1@sea.gmane.org>

Martin v. L?wis wrote:

> Propose first. I have the feeling that the feature will change forth
> and back if everybody gets to say something. I would call it
> sys.svnversion (because that's what it is).

Perhaps it could make sense for sys.svnversion to exist only in a debug
build. This way people won't use it to test for release versions.

Reinhold


-- 
Mail address is perfectly valid!


From martin at v.loewis.de  Sun Dec 18 19:48:20 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 18 Dec 2005 19:48:20 +0100
Subject: [Python-Dev] fresh checkout won't build
In-Reply-To: <do40at$17g$1@sea.gmane.org>
References: <do3u0b$r3n$1@sea.gmane.org> <do40at$17g$1@sea.gmane.org>
Message-ID: <43A5AEF4.5050604@v.loewis.de>

Fredrik Lundh wrote:
>>what am I missing ?
> 
> 
> a C++ compiler, obviously,  the arena code used C++ (C99?) constructs
> in a couple of places.  I've checked in a fix.

C99 also allows to declare variables in the middle of a block.

Regards,
Martin

From fredrik at pythonware.com  Sun Dec 18 20:05:37 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 18 Dec 2005 20:05:37 +0100
Subject: [Python-Dev] fixing log messages
Message-ID: <do4bu3$2j2$1@sea.gmane.org>

just noticed an embarrasing misspelling in one of my recent checkins, only
to find that I cannot fix it:

$  svn propedit --revprop -r 41759 svn:log
svn: Repository has not been enabled to accept revision propchanges;
ask the administrator to create a pre-revprop-change hook
$

would it be a good idea to ask the administrator to do this ?

</F>




From mwh at python.net  Sun Dec 18 20:07:55 2005
From: mwh at python.net (Michael Hudson)
Date: Sun, 18 Dec 2005 19:07:55 +0000
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <43A5A849.1080801@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Sun,
	18 Dec 2005 19:19:53 +0100")
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<20051216211603.GA951@code1.codespeak.net>
	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>
	<5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>
	<1134854657.10350.5.camel@geddy.wooz.org>
	<43A495C6.5040509@v.loewis.de>
	<1134869297.10353.6.camel@geddy.wooz.org>
	<20051218175808.GA17467@code1.codespeak.net>
	<43A5A849.1080801@v.loewis.de>
Message-ID: <2m64pm441g.fsf@starship.python.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> It stopped counting builds on Windows quite some time ago; perhaps it
> is best to drop the build number entirely?

+1.  I don't see how the information it contributes is meaningful in
any way.

Cheers,
mwh

-- 
  Gullible editorial staff continues to post links to any and all
  articles that vaguely criticize Linux in any way.
         -- Reason #4 for quitting slashdot today, from
            http://www.cs.washington.edu/homes/klee/misc/slashdot.html

From martin at v.loewis.de  Sun Dec 18 20:30:19 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 18 Dec 2005 20:30:19 +0100
Subject: [Python-Dev] fixing log messages
In-Reply-To: <do4bu3$2j2$1@sea.gmane.org>
References: <do4bu3$2j2$1@sea.gmane.org>
Message-ID: <43A5B8CB.2090603@v.loewis.de>

Fredrik Lundh wrote:
> just noticed an embarrasing misspelling in one of my recent checkins, only
> to find that I cannot fix it:
> 
> $  svn propedit --revprop -r 41759 svn:log
> svn: Repository has not been enabled to accept revision propchanges;
> ask the administrator to create a pre-revprop-change hook
> $
> 
> would it be a good idea to ask the administrator to do this ?

I have now installed this hook to allow editing svn:log; please try
again.

Regards,
Martin

From fredrik at pythonware.com  Sun Dec 18 20:34:40 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 18 Dec 2005 20:34:40 +0100
Subject: [Python-Dev] fixing log messages
References: <do4bu3$2j2$1@sea.gmane.org> <43A5B8CB.2090603@v.loewis.de>
Message-ID: <do4dkh$7qm$1@sea.gmane.org>

Martin v. Löwis wrote:

> I have now installed this hook to allow editing svn:log; please try
> again.

$  svn propedit --revprop -r 41759 svn:log
Set new value for property 'svn:log' on revision 41759

thanks! /F




From Scott.Daniels at Acm.Org  Sun Dec 18 23:29:27 2005
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Sun, 18 Dec 2005 14:29:27 -0800
Subject: [Python-Dev] Incorporation of zlib sources into Python
	subversion
In-Reply-To: <43A53572.9020802@v.loewis.de>
References: <43A53572.9020802@v.loewis.de>
Message-ID: <do4nrq$64e$1@sea.gmane.org>

Martin v. L?wis wrote:
> Thomas (Heller) and I have been discussing whether the zlib
> module should become builtin, atleast on Win32 (i.e. part
> of python25.dll). This would simplify py2exe, which then could
> bootstrap extraction from the compressed file just with
> pythonxy.dll (clearly, zlib.pyd cannot be *in* the compressed
> file).

Question:
I am trying to enable other compression forms in zipfile, in
particular bzip2, but eventually extensible.  My primary intent
is to extend the useful life of .zips by allowing better
compression (and enabling reading and writing zip formats that
are starting to be created from other sources).  Would it make
sense to include bzip2 in here as well (if the zipfile changes
go in)?

--Scott David Daniels
Scott.Daniels at Acm.Org


From barry at python.org  Sun Dec 18 23:36:51 2005
From: barry at python.org (Barry Warsaw)
Date: Sun, 18 Dec 2005 17:36:51 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <20051218175808.GA17467@code1.codespeak.net>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<20051216211603.GA951@code1.codespeak.net>
	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>
	<5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>
	<1134854657.10350.5.camel@geddy.wooz.org>
	<43A495C6.5040509@v.loewis.de>
	<1134869297.10353.6.camel@geddy.wooz.org>
	<20051218175808.GA17467@code1.codespeak.net>
Message-ID: <1134945411.10357.40.camel@geddy.wooz.org>

On Sun, 2005-12-18 at 18:58 +0100, Armin Rigo wrote:
> On Sat, Dec 17, 2005 at 08:28:17PM -0500, Barry Warsaw wrote:
> > Done. r41744.
> 
> Doesn't appear to work for me: sys.build_number receives the value from
> the buildno.  Looking at the Makefile, the reason is that I'm building
> CPython in a separate directory (running '/some/path/configure; make').

Right.  That's easily fixable by prepending $(srcdir) in front of the
test path and for the svnversion command.  I'm testing that patch now.

> Do we have any plan to make sys.build_number meaningful in the releases
> as well (generally compiled from an svn export, as Michael pointed out),
> or are we happy with a broken number in this case?

Yes, here's my thought: I have a mod to Makefile.pre.in and
getbuildinfo.c so that when we don't find .svn directory, we don't
define the BUILD macro when we compile getbuildinfo.c.  Then, in that
file we have something like:

#ifndef BUILD
#define BUILD "$Revision$"
#endif

Py_GetBuildNumber() grows a bit of logic to yank out the revision number
from that string, but that's all pretty straightforward.  I don't think
svn has an equivalent of cvs's -kv switch.  I've tested most of this,
but I'll have to commit the new getbuildinfo.c to test the export part.

> Should I propose / check-in a patch to expose sys.build_info instead
> ("CPython", "41761", "trunk"), as this got positive feedback so far?
> It's also less surprizing than the current sys.build_number, which is a
> string despite its name.

I'm still unsure about this, so I won't check this change yet.  But I'm
okay with that if people want, though I think the banner should probably
still just contain the revision number.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051218/88bf7202/attachment.pgp

From barry at python.org  Sun Dec 18 23:39:27 2005
From: barry at python.org (Barry Warsaw)
Date: Sun, 18 Dec 2005 17:39:27 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <43A5A849.1080801@v.loewis.de>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<20051216211603.GA951@code1.codespeak.net>
	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>
	<5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>
	<1134854657.10350.5.camel@geddy.wooz.org>
	<43A495C6.5040509@v.loewis.de>
	<1134869297.10353.6.camel@geddy.wooz.org>
	<20051218175808.GA17467@code1.codespeak.net>
	<43A5A849.1080801@v.loewis.de>
Message-ID: <1134945567.10353.42.camel@geddy.wooz.org>

On Sun, 2005-12-18 at 19:19 +0100, "Martin v. L?wis" wrote:

> It stopped counting builds on Windows quite some time ago; perhaps it
> is best to drop the build number entirely?

I think so, because it doesn't really convey anything useful.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051218/abfd1636/attachment.pgp

From martin at v.loewis.de  Sun Dec 18 23:48:27 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 18 Dec 2005 23:48:27 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <1134945411.10357.40.camel@geddy.wooz.org>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>	
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>	
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>	
	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>	
	<20051216211603.GA951@code1.codespeak.net>	
	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>	
	<5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>	
	<1134854657.10350.5.camel@geddy.wooz.org>
	<43A495C6.5040509@v.loewis.de>	
	<1134869297.10353.6.camel@geddy.wooz.org>	
	<20051218175808.GA17467@code1.codespeak.net>
	<1134945411.10357.40.camel@geddy.wooz.org>
Message-ID: <43A5E73B.8050607@v.loewis.de>

Barry Warsaw wrote:
> Yes, here's my thought: I have a mod to Makefile.pre.in and
> getbuildinfo.c so that when we don't find .svn directory, we don't
> define the BUILD macro when we compile getbuildinfo.c.  Then, in that
> file we have something like:
> 
> #ifndef BUILD
> #define BUILD "$Revision$"
> #endif

What does that achieve? It will give you the latest revision at which
getbuildinfo was changed (currently 41744).

Regards,
Martin

From martin at v.loewis.de  Sun Dec 18 23:51:06 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 18 Dec 2005 23:51:06 +0100
Subject: [Python-Dev] Incorporation of zlib sources into
	Python	subversion
In-Reply-To: <do4nrq$64e$1@sea.gmane.org>
References: <43A53572.9020802@v.loewis.de> <do4nrq$64e$1@sea.gmane.org>
Message-ID: <43A5E7DA.40408@v.loewis.de>

Scott David Daniels wrote:
> I am trying to enable other compression forms in zipfile, in
> particular bzip2, but eventually extensible.  My primary intent
> is to extend the useful life of .zips by allowing better
> compression (and enabling reading and writing zip formats that
> are starting to be created from other sources).

That's a good plan; I hope you make it compatible with WinZIP.

> Would it make
> sense to include bzip2 in here as well (if the zipfile changes
> go in)?

I don't think so. People relying on the builtin compression
support of pythonxy.dll would have to make sure the files are
compressed with zlib.

Regards,
Martin

From barry at python.org  Sun Dec 18 23:57:11 2005
From: barry at python.org (Barry Warsaw)
Date: Sun, 18 Dec 2005 17:57:11 -0500
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <43A5E73B.8050607@v.loewis.de>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>
	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>
	<20051216211603.GA951@code1.codespeak.net>
	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>
	<5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>
	<1134854657.10350.5.camel@geddy.wooz.org>
	<43A495C6.5040509@v.loewis.de>
	<1134869297.10353.6.camel@geddy.wooz.org>
	<20051218175808.GA17467@code1.codespeak.net>
	<1134945411.10357.40.camel@geddy.wooz.org>
	<43A5E73B.8050607@v.loewis.de>
Message-ID: <1134946631.10357.44.camel@geddy.wooz.org>

On Sun, 2005-12-18 at 23:48 +0100, "Martin v. L?wis" wrote:

> What does that achieve? It will give you the latest revision at which
> getbuildinfo was changed (currently 41744).

Dunno.  It's better than nothing I guess.
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051218/0f64a52e/attachment.pgp

From martin at v.loewis.de  Mon Dec 19 08:06:18 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 19 Dec 2005 08:06:18 +0100
Subject: [Python-Dev] DRAFT: python-dev summary for 2005-11-16 to
	2005-11-31
In-Reply-To: <d11dcfba0512171513s5ac03353k8d901adcb901bfd1@mail.gmail.com>
References: <d11dcfba0512171513s5ac03353k8d901adcb901bfd1@mail.gmail.com>
Message-ID: <43A65BEA.5040805@v.loewis.de>

Steven Bethard wrote:
> Note that because of the way the subversion conversion was done,
> by-date revision specifications for dates prior to the switchover
> won't work.  To work around this, you can use svn diff (find the
> changes since some date), svn up (check out revision a some date), and
> svn annotate (aka svn blame).

It's actually different: you *cannot* use svn diff or svn up with a
date, since that is what isn't working. As a work-around, you need
a revision /number/ (instead of a date), and you can get them through
svn log or svn blame.

Regards,
Martin

From steve at holdenweb.com  Mon Dec 19 09:28:57 2005
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 19 Dec 2005 08:28:57 +0000
Subject: [Python-Dev] fixing log messages
In-Reply-To: <do4bu3$2j2$1@sea.gmane.org>
References: <do4bu3$2j2$1@sea.gmane.org>
Message-ID: <do5r0d$rgm$3@sea.gmane.org>

Fredrik Lundh wrote:
> just noticed an embarrasing misspelling in one of my recent checkins, only

That's "embarrassing", by the way. You're obviously having a bad 
spelling day :-)

not-throwing-stones-ly y'rs  - steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From steve at holdenweb.com  Mon Dec 19 09:25:09 2005
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 19 Dec 2005 08:25:09 +0000
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <1134945567.10353.42.camel@geddy.wooz.org>
References: <5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>	<20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>	<5.1.1.6.0.20051216105152.01affc40@mail.telecommunity.com>	<5.1.1.6.0.20051216113120.01dec9b8@mail.telecommunity.com>	<20051216211603.GA951@code1.codespeak.net>	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>	<5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>	<1134854657.10350.5.camel@geddy.wooz.org>	<43A495C6.5040509@v.loewis.de>	<1134869297.10353.6.camel@geddy.wooz.org>	<20051218175808.GA17467@code1.codespeak.net>	<43A5A849.1080801@v.loewis.de>
	<1134945567.10353.42.camel@geddy.wooz.org>
Message-ID: <do5qp8$rgm$2@sea.gmane.org>

Barry Warsaw wrote:
> On Sun, 2005-12-18 at 19:19 +0100, "Martin v. L?wis" wrote:
> 
> 
>>It stopped counting builds on Windows quite some time ago; perhaps it
>>is best to drop the build number entirely?
> 
> 
> I think so, because it doesn't really convey anything useful.
> 
I thought it was more succinct than the build-date when rebuilding 
continuously during testing, but I guess I'm only -0 on dropping it.

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From smiles at worksmail.net  Mon Dec 19 06:42:39 2005
From: smiles at worksmail.net (Chris or Leslie Smith)
Date: Sun, 18 Dec 2005 23:42:39 -0600
Subject: [Python-Dev] synchronized enumerate
Message-ID: <007001c6045f$584ec7c0$0b2c4fca@csmith>

I see that there is a thread of a similar topic that was posted recently ( enumerate with a start index  ) but thought I would start a new thread since what I am suggesting is a little different.

Whenever I use enumerate, I am doing so because I will use the index to access some other element in the list (the previous or next, usually) while also looking at the element that is returned from enumerate.  Several times, however, in the development phase of the work, I end up sending a subset of the list at hand and then get bitten by the fact that the indices returned by enumerate are not the indices of the original list, they are the indices of the slice that I sent. e.g. in the following, "0" is the first index but I wanted it to be 3

###
>>> start=3
>>> count=5
>>> for i, x in enumerate(range(10)[start:start+count]):
...  print i, x
...  
0 3
1 4
2 5
3 6
4 7
>>> 
###

What I would propose is an optional slice argument to the enumerate routine that would allow enumerate to return elements that are synchronized with the original list list/iterable elements. e.g.

def enum(l, slc=None):
    if slc==None:
        for i, dat in enumerate(l):
            yield i, dat
    else:
        if type(slc)<>slice:
            raise TypeError, "slc must be a valid slice"
        start, step = slc.start, slc.step
        # we need actual values for start and step, so check for None
        # and supply defaults
        if step==None:step=1
        if start==None:
            if step>0:
                start=0
            else:
                start=-1
        for i, dat in enumerate(l[slc]):
            j = i*step+start
            if j<0: j+=len(l)
            yield j, dat
###
>>> for i, x in enum(range(10), slice(start, start+count)):
...  print i, x
...  
3 3
4 4
5 5
6 6
7 7
>>> for i, j in enum(range(10), slice(None,None,-3)):
...  print i,j
...  
9 9
6 6
3 3
0 0
>>> 
###

An advantage to processing the iteratable with a slice argument is that then the slice information is given only once and it can do 2 things: slice the original iterable and provide the synchronized indices.

NOTE: the same thing that I am proposing could also be done with count and izip if count had a step argument, but it's more ackward and you have to supply the same information in two places:

>>> def count(start, step=1):
...  for i in itertools.count(start):
...   yield start+(i-start)*step
... 
>>> 
>>> start=3; stop=None; step=2
>>> for i,j in itertools.izip(count(start, step), itertools.islice(range(10), start, stop, step)):
...  print i,j
...  
3 3
5 5
7 7
9 9

A "P.S." question for this email is, was there a reason to leave step out of itertools.count? 

/c

From aahz at pythoncraft.com  Mon Dec 19 16:22:46 2005
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 19 Dec 2005 07:22:46 -0800
Subject: [Python-Dev] synchronized enumerate
In-Reply-To: <007001c6045f$584ec7c0$0b2c4fca@csmith>
References: <007001c6045f$584ec7c0$0b2c4fca@csmith>
Message-ID: <20051219152246.GA12150@panix.com>

On Sun, Dec 18, 2005, Chris or Leslie Smith wrote:
>
> What I would propose is an optional slice argument to the enumerate
> routine that would allow enumerate to return elements that are
> synchronized with the original list list/iterable elements. e.g.

python-dev is the wrong place to start discussions like this; please use
comp.lang.python.  Thanks!
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Don't listen to schmucks on USENET when making legal decisions.  Hire
yourself a competent schmuck."  --USENET schmuck (aka Robert Kern)

From john at clocksoft.com  Mon Dec 19 16:22:31 2005
From: john at clocksoft.com (John Pinner)
Date: Mon, 19 Dec 2005 15:22:31 +0000
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <mailman.19.1134990006.24967.python-dev@python.org>
References: <mailman.19.1134990006.24967.python-dev@python.org>
Message-ID: <43A6D037.6040406@clocksoft.com>

> 
> Barry Warsaw wrote:
> 
>>On Sun, 2005-12-18 at 19:19 +0100, "Martin v. L?wis" wrote:
>>
>>
>>
>>>It stopped counting builds on Windows quite some time ago; perhaps it
>>>is best to drop the build number entirely?
>>
>>
>>I think so, because it doesn't really convey anything useful.
>>
> 
> I thought it was more succinct than the build-date when rebuilding 
> continuously during testing, but I guess I'm only -0 on dropping it.

It's also the only thing that identifes the revision/build precisely, 
allowing reversion to a known state.

John
--

From jimjjewett at gmail.com  Mon Dec 19 19:51:12 2005
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 19 Dec 2005 13:51:12 -0500
Subject: [Python-Dev] Keep default comparisons - or add a second set?
Message-ID: <fb6fbf560512191051t336daa32pb5a7c0c6929f7e4c@mail.gmail.com>

PEP 3000 now suggests that dropping default comparison has become more
than an idle what-if.

Unfortunately, one very common use case of comparisons is to get a
canonical order.  If the order is sensible, all the better, but that
is not strictly required.  One of Python's selling points (especially
compared to Java) is that getting a canonical order "just works", even
if the objects being sorted are not carefully homogenized by hand. 
Python itself relies on this when comparing same-length dictionaries.

There are times when a specific comparison doesn't make sense (date vs
a datetime that it includes), but these are corner cases best handled
by the specific class that understands the specific requirements --
classes that already have to override the comparison operators anyhow.

Even the recently posted "get rid of default comparisons" use case is
really just an argument to make the canonical ordering work better. 
The problem Jim Fulton describes is that the (current default)
canonical order will change if objects are saved to a database and
then imported to a different session.  Removing default comparisons
wouldn't really help much; the errors would (sometimes) show up at
saving instead of (maybe) at loading, but the solution would still be
to handcode a default comparison for every single class individually.

I don't think anyone wants a smorgasbord of inconsistent error-prone
boilerplate code.  (X<Y but also Y<X, because they used different
default comparisons, so it matters which one gets asked.)  Even
careful coders will sometimes get burned by whether to sort based on
id(obj) or (obj.__class__, id(obj))

Guido has already rejected a standard mixin class.

Today's solution is to write the default comparison once and let
everything "inherit" it.

The persistence layers need an improved default comparison -- but they
can provide their own via wrapper objects, since they control the save
mechanism.  To be honest, this may be the only reasonable solution; if
objects are saved with their original id and they use that for
comparison, then you can continue a session, but you still have to
worry about having the id reused in a later session.  If ids are
timestamped, you need to worry about importing objects from multiple
original processes, or different machines, or ... and the comparison
gets slow, for something that won't provide any benefit to smaller
applications.

Go ahead and tweak the default comparisons; there is room to argue about
whether (1,2) < [3,4] < (5,6) should evaluate as true.  Go ahead and
include more information, if it doesn't slow down the common case; it
might help the simpler databases.  But please don't make that
expression throw an exception.  (Or, at the very least, promote a
*standard* way to say "just get me a canonical ordering of some sort"
-- which we can write today with "sorted()".)

---------------------
References:

Guido agrees to removing the default comparisons:

    http://www.python.org/peps/pep-3000.html
        """
        Comparisons other than == and != between disparate types will raise an
        exception unless explicitly supported by the type [6]
        """
    and the more recent reference at
        http://mail.python.org/pipermail/python-dev/2005-November/057938.html.

Jim Fulton says that the _current_ default breaks for persistence.
    http://mail.python.org/pipermail/python-dev/2005-November/057924.html

Guido rejects a comparison mixin
    http://mail.python.org/pipermail/python-dev/2005-November/057925.html

From p.f.moore at gmail.com  Mon Dec 19 21:19:58 2005
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 19 Dec 2005 20:19:58 +0000
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <fb6fbf560512191051t336daa32pb5a7c0c6929f7e4c@mail.gmail.com>
References: <fb6fbf560512191051t336daa32pb5a7c0c6929f7e4c@mail.gmail.com>
Message-ID: <79990c6b0512191219x1fa91b0evb5722116a91216a@mail.gmail.com>

On 12/19/05, Jim Jewett <jimjjewett at gmail.com> wrote:

> Unfortunately, one very common use case of comparisons is to get a
> canonical order.  If the order is sensible, all the better, but that
> is not strictly required.  One of Python's selling points (especially
> compared to Java) is that getting a canonical order "just works", even
> if the objects being sorted are not carefully homogenized by hand.
> Python itself relies on this when comparing same-length dictionaries.

While I agree that this is a useful property, and I'd like it to be
true, it isn't even today:

>>> import Numeric
>>> a = Numeric.array((1,2,3))
>>> a
array([1, 2, 3])
>>> sorted([a, (1,2,3), 1, 2, '123'])
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: function not supported for these types, and can't coerce to supported
 types
>>> a < '123'
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: function not supported for these types, and can't coerce to supported
 types
>>>

Numeric arrays are the canonical example of an "awkward case" when
discussing comparisons...

OTOH, I'm not sure I want to see more "awkward cases" added.

Paul.

From ncoghlan at gmail.com  Mon Dec 19 22:10:31 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 20 Dec 2005 07:10:31 +1000
Subject: [Python-Dev] synchronized enumerate
In-Reply-To: <007001c6045f$584ec7c0$0b2c4fca@csmith>
References: <007001c6045f$584ec7c0$0b2c4fca@csmith>
Message-ID: <43A721C7.3020101@gmail.com>

Chris or Leslie Smith wrote:
> Whenever I use enumerate, I am doing so because I will use the index to
> access some other element in the list (the previous or next, usually) while
> also looking at the element that is returned from enumerate.  Several
> times, however, in the development phase of the work, I end up sending a
> subset of the list at hand and then get bitten by the fact that the indices
> returned by enumerate are not the indices of the original list, they are
> the indices of the slice that I sent. e.g. in the following, "0" is the
> first index but I wanted it to be 3

a. What Aahz said (this is more a c.l.p./python-list question than a 
python-dev one)

b. It may be worth finding a way to use itertools.islice on the output of 
enumerate rather than slicing before the enumeration operation if you want the 
indices to line up with the original sequence rather than with the slice.

Cheers,
Nick.

P.S. I don't personally track c.l.p. (other than some of the threads that make 
it to python-url) so cc' my email address if you want to follow up on point b :)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From martin at v.loewis.de  Mon Dec 19 23:22:10 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 19 Dec 2005 23:22:10 +0100
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <43A6D037.6040406@clocksoft.com>
References: <mailman.19.1134990006.24967.python-dev@python.org>
	<43A6D037.6040406@clocksoft.com>
Message-ID: <43A73292.4020900@v.loewis.de>

John Pinner wrote:
> It's also the only thing that identifes the revision/build precisely,
> allowing reversion to a known state.

How so?
- It doesn't identify a build precisely: you may have dynamically-loaded
  modules that get rebuild even though the build number doesn't change.
  So a single build number may refer to different sets of code.
- it doesn't allow reversion to a known state: I can't even find a
  meaningful interpretation of such a claim. How would the build allow
  to revert anything? And what is that anything that it would allow to
  revert? Reverting doesn't work for source changes, nor for configure
  options (svn revert allows to do the former, in a limited way; svk
  allows that in a broader way).

Regards,
Martin

From raymond.hettinger at verizon.net  Mon Dec 19 23:33:35 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Mon, 19 Dec 2005 17:33:35 -0500
Subject: [Python-Dev] synchronized enumerate
In-Reply-To: <007001c6045f$584ec7c0$0b2c4fca@csmith>
Message-ID: <018501c604ec$4089bf40$94bd2c81@oemcomputer>

[Chris or Leslie Smith]
> I see that there is a thread of a similar topic that was posted
recently (
> enumerate with a start index  ) but thought I would start a new thread
> since what I am suggesting is a little different.

Try rolling your own with izip() and count():
   
   izip(count(start), someslice)


Raymond Hettinger


From greg.ewing at canterbury.ac.nz  Tue Dec 20 02:37:06 2005
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 20 Dec 2005 14:37:06 +1300
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <fb6fbf560512191051t336daa32pb5a7c0c6929f7e4c@mail.gmail.com>
References: <fb6fbf560512191051t336daa32pb5a7c0c6929f7e4c@mail.gmail.com>
Message-ID: <43A76042.2080406@canterbury.ac.nz>

Jim Jewett wrote:
 > Or, at the very least, promote a
> *standard* way to say "just get me a canonical ordering of some sort"

That would be my preference. Comparison for canonical
ordering should be a distinct operation with its
own spelling. Then Guido's

>         Comparisons other than == and != between disparate types will raise an
>         exception unless explicitly supported by the type

can be true without precluding the existence of a
canonical ordering.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From murman at gmail.com  Tue Dec 20 04:29:33 2005
From: murman at gmail.com (Michael Urman)
Date: Mon, 19 Dec 2005 21:29:33 -0600
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <43A76042.2080406@canterbury.ac.nz>
References: <fb6fbf560512191051t336daa32pb5a7c0c6929f7e4c@mail.gmail.com>
	<43A76042.2080406@canterbury.ac.nz>
Message-ID: <dcbbbb410512191929k604a1532r5d465555ae4deaff@mail.gmail.com>

On 12/19/05, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> That would be my preference. Comparison for canonical
> ordering should be a distinct operation with its
> own spelling.

Such as sorted(stuff, key=id)?

Michael
--
Michael Urman  http://www.tortall.net/mu/blog

From jcarlson at uci.edu  Tue Dec 20 05:29:13 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 19 Dec 2005 20:29:13 -0800
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <dcbbbb410512191929k604a1532r5d465555ae4deaff@mail.gmail.com>
References: <43A76042.2080406@canterbury.ac.nz>
	<dcbbbb410512191929k604a1532r5d465555ae4deaff@mail.gmail.com>
Message-ID: <20051219202756.946C.JCARLSON@uci.edu>


Michael Urman <murman at gmail.com> wrote:
> 
> On 12/19/05, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> > That would be my preference. Comparison for canonical
> > ordering should be a distinct operation with its
> > own spelling.
> 
> Such as sorted(stuff, key=id)?

I believe that ideally, canonical orderings would be persistant across
sessions.

 - Josiah


From ldlandis at gmail.com  Tue Dec 20 10:00:54 2005
From: ldlandis at gmail.com (LD 'Gus' Landis)
Date: Tue, 20 Dec 2005 03:00:54 -0600
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <43A5E73B.8050607@v.loewis.de>
References: <20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<20051216211603.GA951@code1.codespeak.net>
	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>
	<5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>
	<1134854657.10350.5.camel@geddy.wooz.org>
	<43A495C6.5040509@v.loewis.de>
	<1134869297.10353.6.camel@geddy.wooz.org>
	<20051218175808.GA17467@code1.codespeak.net>
	<1134945411.10357.40.camel@geddy.wooz.org>
	<43A5E73B.8050607@v.loewis.de>
Message-ID: <a1ddf57e0512200100w3b1d4d0do375ae52ecead42f0@mail.gmail.com>

Hi,

  So, what does one get when built on a system not connected to
  the net? say from a tarball?  It can happen, ya know.

Cheers,
  --ldl

On 12/18/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Barry Warsaw wrote:
> > Yes, here's my thought: I have a mod to Makefile.pre.in and
> > getbuildinfo.c so that when we don't find .svn directory, we don't
> > define the BUILD macro when we compile getbuildinfo.c.  Then, in that
> > file we have something like:
> >
> > #ifndef BUILD
> > #define BUILD "$Revision$"
> > #endif
>
> What does that achieve? It will give you the latest revision at which
> getbuildinfo was changed (currently 41744).
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ldlandis%40gmail.com
>


--
LD Landis - N0YRQ - from the St Paul side of Minneapolis

From mwh at python.net  Tue Dec 20 10:13:14 2005
From: mwh at python.net (Michael Hudson)
Date: Tue, 20 Dec 2005 09:13:14 +0000
Subject: [Python-Dev] Expose Subversion revision number to Python
In-Reply-To: <a1ddf57e0512200100w3b1d4d0do375ae52ecead42f0@mail.gmail.com> (LD
	Landis's message of "Tue, 20 Dec 2005 03:00:54 -0600")
References: <20051216074215.4fbqijt1mhkw8wcc@login.werra.lunarpages.com>
	<20051216211603.GA951@code1.codespeak.net>
	<5.1.1.6.0.20051216162743.01dddb60@mail.telecommunity.com>
	<5.1.1.6.0.20051216170838.01def980@mail.telecommunity.com>
	<1134854657.10350.5.camel@geddy.wooz.org>
	<43A495C6.5040509@v.loewis.de>
	<1134869297.10353.6.camel@geddy.wooz.org>
	<20051218175808.GA17467@code1.codespeak.net>
	<1134945411.10357.40.camel@geddy.wooz.org>
	<43A5E73B.8050607@v.loewis.de>
	<a1ddf57e0512200100w3b1d4d0do375ae52ecead42f0@mail.gmail.com>
Message-ID: <2mvexk2kt1.fsf@starship.python.net>

"LD 'Gus' Landis" <ldlandis at gmail.com> writes:

> Hi,
>
>   So, what does one get when built on a system not connected to
>   the net? say from a tarball?

Um, the two things you mention are unrelated.  svnversion doesn't
access the network, so the first thing is moot.  For the second, we've
talked about that already, though I'm not entirely sure what the
conclusion was...

>   It can happen, ya know.

Yes, we know!

Cheers,
mwh

-- 
  Windows installation day one.  Getting rid of the old windows 
  was easy - they fell apart quite happily, and certainly wont 
  be re-installable anywhere else.   -- http://www.linux.org.uk/diary/
                                       (not *that* sort of windows...)

From fredrik at pythonware.com  Tue Dec 20 11:40:15 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 20 Dec 2005 11:40:15 +0100
Subject: [Python-Dev] fixing log messages
References: <do4bu3$2j2$1@sea.gmane.org> <do5r0d$rgm$3@sea.gmane.org>
Message-ID: <do8n2g$1s5$1@sea.gmane.org>

Steve Holden wrote:

> That's "embarrassing", by the way. You're obviously having a bad
> spelling day :-)

I'd say that any spelling with more than 500,000 google hits is perfectly valid...

</F> 




From fredrik at pythonware.com  Tue Dec 20 11:55:19 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 20 Dec 2005 11:55:19 +0100
Subject: [Python-Dev] status of development documentation
Message-ID: <do8nun$76d$1@sea.gmane.org>

the "Documentation Development" page at

    http://www.python.org/dev/doc/

contains a link to a "SVN trunk" version which was last updated nearly
four months ago.  what would it take to automatically update the trunk
docs, say, once a day or so ?

or is it time to move away from the current "odd-format-with-extensive-
semantic-markup-that-nobody-uses-and-only-a-few-people-fully-under-
stand" approach to something quicker and dirtier [1] ?

</F>

1) http://microformats.org/wiki/microformats 




From steve at holdenweb.com  Tue Dec 20 12:12:29 2005
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 20 Dec 2005 11:12:29 +0000
Subject: [Python-Dev] fixing log messages
In-Reply-To: <do8n2g$1s5$1@sea.gmane.org>
References: <do4bu3$2j2$1@sea.gmane.org> <do5r0d$rgm$3@sea.gmane.org>
	<do8n2g$1s5$1@sea.gmane.org>
Message-ID: <do8ouu$bds$2@sea.gmane.org>

Fredrik Lundh wrote:
> Steve Holden wrote:
> 
> 
>>That's "embarrassing", by the way. You're obviously having a bad
>>spelling day :-)
> 
> 
> I'd say that any spelling with more than 500,000 google hits is perfectly valid...
> 
Anything you say, Frederick ...

thirteen-million-google-hits-can't-possibly-be-wrong-ly y'rs  - steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From jim at zope.com  Tue Dec 20 12:54:38 2005
From: jim at zope.com (Jim Fulton)
Date: Tue, 20 Dec 2005 06:54:38 -0500
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <fb6fbf560512191051t336daa32pb5a7c0c6929f7e4c@mail.gmail.com>
References: <fb6fbf560512191051t336daa32pb5a7c0c6929f7e4c@mail.gmail.com>
Message-ID: <43A7F0FE.3050303@zope.com>

Jim Jewett wrote:
> PEP 3000 now suggests that dropping default comparison has become more
> than an idle what-if.
> 
> Unfortunately, one very common use case of comparisons is to get a
> canonical order.  If the order is sensible, all the better, but that
> is not strictly required.  One of Python's selling points (especially
> compared to Java) is that getting a canonical order "just works", even
> if the objects being sorted are not carefully homogenized by hand. 
> Python itself relies on this when comparing same-length dictionaries.
> 
> There are times when a specific comparison doesn't make sense (date vs
> a datetime that it includes), but these are corner cases best handled
> by the specific class that understands the specific requirements --
> classes that already have to override the comparison operators anyhow.
> 
> Even the recently posted "get rid of default comparisons" use case is
> really just an argument to make the canonical ordering work better. 
> The problem Jim Fulton describes is that the (current default)
> canonical order will change if objects are saved to a database and
> then imported to a different session.  Removing default comparisons
> wouldn't really help much; the errors would (sometimes) show up at
> saving instead of (maybe) at loading, but the solution would still be
> to handcode a default comparison for every single class individually.

I think you need to do a much better job of defining canonical ordering.

You've given two properties:

- It need not make sense. :)

- It must be consistent accross sessions

   Does this also mean accross different versions of Python?

   How about different operating systems and hardware?

   If I create and pickle a BTree with a bunch of object keys
   and reload that pickle in a different session, with a
   later version of Python on a different OS and Hardware
   architecture, will the keys still have the same order?

   I consider (obviously) this second property to be crucial.

Do you have any proposal for how to achieve these properties?

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From fredrik at pythonware.com  Tue Dec 20 14:36:29 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 20 Dec 2005 14:36:29 +0100
Subject: [Python-Dev] fixing log messages
References: <do4bu3$2j2$1@sea.gmane.org>
	<do5r0d$rgm$3@sea.gmane.org><do8n2g$1s5$1@sea.gmane.org>
	<do8ouu$bds$2@sea.gmane.org>
Message-ID: <do91cu$8c1$1@sea.gmane.org>

Steve Holden wrote:

>> I'd say that any spelling with more than 500,000 google hits is perfectly valid...
>>
> Anything you say, Frederick ...
>
> thirteen-million-google-hits-can't-possibly-be-wrong-ly y'rs  - steve

oh, frederick is a perfectly valid english spelling of the germanic name "Friedrich",
but only ~200 google hits use that name to refer to me.

(but alright, as long as you don't call me "Fred"...)

</F> 




From steve at holdenweb.com  Tue Dec 20 15:42:18 2005
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 20 Dec 2005 14:42:18 +0000
Subject: [Python-Dev] fixing log messages
In-Reply-To: <do91cu$8c1$1@sea.gmane.org>
References: <do4bu3$2j2$1@sea.gmane.org>	<do5r0d$rgm$3@sea.gmane.org><do8n2g$1s5$1@sea.gmane.org>	<do8ouu$bds$2@sea.gmane.org>
	<do91cu$8c1$1@sea.gmane.org>
Message-ID: <do958b$lki$1@sea.gmane.org>

Fredrik Lundh wrote:
> Steve Holden wrote:
> 
> 
>>>I'd say that any spelling with more than 500,000 google hits is perfectly valid...
>>>
>>
>>Anything you say, Frederick ...
>>
>>thirteen-million-google-hits-can't-possibly-be-wrong-ly y'rs  - steve
> 
> 
> oh, frederick is a perfectly valid english spelling of the germanic name "Friedrich",
> but only ~200 google hits use that name to refer to me.
> 
> (but alright, as long as you don't call me "Fred"...)
> 

Did I *ever* do that? That would have been an embarrassing slip ;-)

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From murman at gmail.com  Tue Dec 20 15:59:04 2005
From: murman at gmail.com (Michael Urman)
Date: Tue, 20 Dec 2005 08:59:04 -0600
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <20051219202756.946C.JCARLSON@uci.edu>
References: <43A76042.2080406@canterbury.ac.nz>
	<dcbbbb410512191929k604a1532r5d465555ae4deaff@mail.gmail.com>
	<20051219202756.946C.JCARLSON@uci.edu>
Message-ID: <dcbbbb410512200659j2767bf3ckfa3319d50ec0f716@mail.gmail.com>

On 12/19/05, Josiah Carlson <jcarlson at uci.edu> wrote:
>
> Michael Urman <murman at gmail.com> wrote:
> > Such as sorted(stuff, key=id)?
>
> I believe that ideally, canonical orderings would be persistant across
> sessions.

Erm, yes, I totally missed that in Jim's original preferred
requirements. And I nearly wrote another response ignoring Jim's use
case of persistence, as I'm having trouble thinking of any (others)
where order matters yet comparability doesn't.

Michael
--
Michael Urman  http://www.tortall.net/mu/blog

From munna_tank at yahoo.co.in  Tue Dec 20 18:27:40 2005
From: munna_tank at yahoo.co.in (RASHMI TANK)
Date: Tue, 20 Dec 2005 17:27:40 +0000 (GMT)
Subject: [Python-Dev] (no subject)
Message-ID: <20051220172740.15717.qmail@web8207.mail.in.yahoo.com>

sir i have taken softwer from icashline.com
  that is mass mailing softwere is worldcast
  that is not running it is showing that I/O PROBLEM
  OR MEDIA PRO BLEM.
   
  PLEASE HELP ME.

Send instant messages to your online friends http://in.messenger.yahoo.com 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20051220/2034aa2b/attachment.html

From theller at python.net  Tue Dec 20 18:54:14 2005
From: theller at python.net (Thomas Heller)
Date: Tue, 20 Dec 2005 18:54:14 +0100
Subject: [Python-Dev] Build failure and problem on Windows
In-Reply-To: <hd93u0lr.fsf@python.net> (Thomas Heller's message of "Tue, 20
	Dec 2005 18:42:08 +0100")
References: <hd93u0lr.fsf@python.net>
Message-ID: <8xufu01l.fsf@python.net>

Thomas Heller <theller at python.net> writes:

> Building the svn trunk on Windows fails because Python\pyarena.c is
> missing in the pythoncore.vcproj file (I'm not yet up to speed with svn,
> otherwise I would have checked in a fix for this myself).
>
> Worse, when running the built exe it segfaults in Py_GetBuildInfo(),
> because it is picking up somehow a definition of #define BUILD 'b' (from
> cPickle.c? Could that be?)

I should have known better, but BUILD is defined in the MSVC project
file as BUILD=60.

Thomas



From theller at python.net  Tue Dec 20 18:42:08 2005
From: theller at python.net (Thomas Heller)
Date: Tue, 20 Dec 2005 18:42:08 +0100
Subject: [Python-Dev] Build failure and problem on Windows
Message-ID: <hd93u0lr.fsf@python.net>


Building the svn trunk on Windows fails because Python\pyarena.c is
missing in the pythoncore.vcproj file (I'm not yet up to speed with svn,
otherwise I would have checked in a fix for this myself).

Worse, when running the built exe it segfaults in Py_GetBuildInfo(),
because it is picking up somehow a definition of #define BUILD 'b' (from
cPickle.c? Could that be?)

Thomas





From theller at python.net  Tue Dec 20 20:23:04 2005
From: theller at python.net (Thomas Heller)
Date: Tue, 20 Dec 2005 20:23:04 +0100
Subject: [Python-Dev] os.startfile with optional second parameter
Message-ID: <oe3bshd3.fsf@python.net>

Would a patch be accepted that implemented an optional second parameter
for the os.startfile function on Windows?

Sometimes I missed the possibility to write

  os.startfile("mydocs.pdf", "print")

Thomas



From nnorwitz at gmail.com  Tue Dec 20 20:45:30 2005
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 20 Dec 2005 11:45:30 -0800
Subject: [Python-Dev] [OT] Fwd: a new python port: iPod
In-Reply-To: <20051220192552.GA6547@pascal.jepsi.net>
References: <20051220192552.GA6547@pascal.jepsi.net>
Message-ID: <ee2a432c0512201145x682718bfofb0a5dffd2b7bee2@mail.gmail.com>

I know this is OT, but I thought y'all might find this interesting. -- n

---------- Forwarded message ----------
From: jack <jack at gulcas.org>
To: webmaster at python.org

Hi,
 I saw in http://www.python.org/download/download_other.html that people notify
 python ports to other platforms/devices.
 I ported python to iPod, and would be great that this port will appear in this
 section too.
 For more info about python port to Apple iPod see:
 http://www.ciberjacobo.com/en/linux_on_ipod.html

Thanks.

--
Jacobo Avariento Gimeno
http://ciberjacobo.com
OpenPGP key: http://ciberjacobo.com/key.pem

From jcarlson at uci.edu  Tue Dec 20 21:22:52 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 20 Dec 2005 12:22:52 -0800
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <43A7F0FE.3050303@zope.com>
References: <fb6fbf560512191051t336daa32pb5a7c0c6929f7e4c@mail.gmail.com>
	<43A7F0FE.3050303@zope.com>
Message-ID: <20051220120244.9470.JCARLSON@uci.edu>


Jim Fulton <jim at zope.com> wrote:
> 
> Jim Jewett wrote:
> > PEP 3000 now suggests that dropping default comparison has become more
> > than an idle what-if.
> > 
> > Unfortunately, one very common use case of comparisons is to get a
> > canonical order.  If the order is sensible, all the better, but that
> > is not strictly required.  One of Python's selling points (especially
> > compared to Java) is that getting a canonical order "just works", even
> > if the objects being sorted are not carefully homogenized by hand. 
> > Python itself relies on this when comparing same-length dictionaries.
> > 
> > There are times when a specific comparison doesn't make sense (date vs
> > a datetime that it includes), but these are corner cases best handled
> > by the specific class that understands the specific requirements --
> > classes that already have to override the comparison operators anyhow.
> > 
> > Even the recently posted "get rid of default comparisons" use case is
> > really just an argument to make the canonical ordering work better. 
> > The problem Jim Fulton describes is that the (current default)
> > canonical order will change if objects are saved to a database and
> > then imported to a different session.  Removing default comparisons
> > wouldn't really help much; the errors would (sometimes) show up at
> > saving instead of (maybe) at loading, but the solution would still be
> > to handcode a default comparison for every single class individually.
> 
> I think you need to do a much better job of defining canonical ordering.
> 
> You've given two properties:
> 
> - It need not make sense. :)
> 
> - It must be consistent accross sessions
> 
>    Does this also mean accross different versions of Python?
> 
>    How about different operating systems and hardware?
> 
>    If I create and pickle a BTree with a bunch of object keys
>    and reload that pickle in a different session, with a
>    later version of Python on a different OS and Hardware
>    architecture, will the keys still have the same order?
> 
>    I consider (obviously) this second property to be crucial.
> 
> Do you have any proposal for how to achieve these properties?

New superclasses for all built-in types (except for string and unicode,
which already subclass from basestring).

int, float, complex (long) : subclass from basenumber
tuple, list, set : subclass from basesequence
dict : subclass from basemapping

The idea is that each of the above classes define a group in which items
are comparable.  If you end up in a situation in which the base classes
of the compared object differ (and hence are not comparable directly by
value), you compare their base class name.  Because their base classes
differ, you always get a reliable differentiation between groups.


What about comparisons between user-defined classes (classic or subclass
of object)?  Presumably if a user wanted something to be compared
against integers, floats, or complex, the user would subclass from
basenumber, etc.  If the user only wanted their objects to compare
against objects of its own type, they compose their own __cmp__ or
related methods on their class, and they get this behavior 'for free'.

The only thing necessary for canonical ordering persistancy is that the
content of an object define its behavior in comparison operators, and
that pickle knows how to save and restore this content reliably.


Note that one can remove the superclass requirement with a smart cmp()
builtin to automatically choose the comparable group.


This is not perfect, but it is an idea, and it would allow a reliable
canonical ordering.

 - Josiah


From mal at egenix.com  Tue Dec 20 21:49:23 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 20 Dec 2005 21:49:23 +0100
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <20051220120244.9470.JCARLSON@uci.edu>
References: <fb6fbf560512191051t336daa32pb5a7c0c6929f7e4c@mail.gmail.com>	<43A7F0FE.3050303@zope.com>
	<20051220120244.9470.JCARLSON@uci.edu>
Message-ID: <43A86E53.4000007@egenix.com>

Josiah Carlson wrote:
> Jim Fulton <jim at zope.com> wrote:
>> Jim Jewett wrote:
>>> PEP 3000 now suggests that dropping default comparison has become more
>>> than an idle what-if.
>>>
>>> Unfortunately, one very common use case of comparisons is to get a
>>> canonical order.  If the order is sensible, all the better, but that
>>> is not strictly required.  One of Python's selling points (especially
>>> compared to Java) is that getting a canonical order "just works", even
>>> if the objects being sorted are not carefully homogenized by hand. 
>>> Python itself relies on this when comparing same-length dictionaries.
>>>
>>> There are times when a specific comparison doesn't make sense (date vs
>>> a datetime that it includes), but these are corner cases best handled
>>> by the specific class that understands the specific requirements --
>>> classes that already have to override the comparison operators anyhow.
>>>
>>> Even the recently posted "get rid of default comparisons" use case is
>>> really just an argument to make the canonical ordering work better. 
>>> The problem Jim Fulton describes is that the (current default)
>>> canonical order will change if objects are saved to a database and
>>> then imported to a different session.  Removing default comparisons
>>> wouldn't really help much; the errors would (sometimes) show up at
>>> saving instead of (maybe) at loading, but the solution would still be
>>> to handcode a default comparison for every single class individually.
>> I think you need to do a much better job of defining canonical ordering.
>>
>> You've given two properties:
>>
>> - It need not make sense. :)
>>
>> - It must be consistent accross sessions
>>
>>    Does this also mean accross different versions of Python?
>>
>>    How about different operating systems and hardware?
>>
>>    If I create and pickle a BTree with a bunch of object keys
>>    and reload that pickle in a different session, with a
>>    later version of Python on a different OS and Hardware
>>    architecture, will the keys still have the same order?
>>
>>    I consider (obviously) this second property to be crucial.
>>
>> Do you have any proposal for how to achieve these properties?
> 
> New superclasses for all built-in types (except for string and unicode,
> which already subclass from basestring).
> 
> int, float, complex (long) : subclass from basenumber
> tuple, list, set : subclass from basesequence
> dict : subclass from basemapping

set should be under basemapping.

> The idea is that each of the above classes define a group in which items
> are comparable.  If you end up in a situation in which the base classes
> of the compared object differ (and hence are not comparable directly by
> value), you compare their base class name.  Because their base classes
> differ, you always get a reliable differentiation between groups.

Python already uses this "trick" based on the type name.
If that still doesn't help, id(object) is used which is
what JimF is criticizing (I presume).

> What about comparisons between user-defined classes (classic or subclass
> of object)?  Presumably if a user wanted something to be compared
> against integers, floats, or complex, the user would subclass from
> basenumber, etc. 

... and get all kinds of weird side-effects. A user
probably doesn't want these :-)

> If the user only wanted their objects to compare
> against objects of its own type, they compose their own __cmp__ or
> related methods on their class, and they get this behavior 'for free'.
> 
> The only thing necessary for canonical ordering persistancy is that the
> content of an object define its behavior in comparison operators, and
> that pickle knows how to save and restore this content reliably.

Actually, the only thing necessary for *persisting* order is
making sure that the persistence logic maintains order across
pickling.

Note that this is a completely different requirement
than making sure that the outcome of list.sort() is the same
across platforms and sessions.

> Note that one can remove the superclass requirement with a smart cmp()
> builtin to automatically choose the comparable group.
> 
> 
> This is not perfect, but it is an idea, and it would allow a reliable
> canonical ordering.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 20 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From aahz at pythoncraft.com  Tue Dec 20 23:31:24 2005
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 20 Dec 2005 14:31:24 -0800
Subject: [Python-Dev]  Sets are mappings?
In-Reply-To: <43A86E53.4000007@egenix.com>
References: <fb6fbf560512191051t336daa32pb5a7c0c6929f7e4c@mail.gmail.com>
	<43A7F0FE.3050303@zope.com> <20051220120244.9470.JCARLSON@uci.edu>
	<43A86E53.4000007@egenix.com>
Message-ID: <20051220223124.GB6402@panix.com>

On Tue, Dec 20, 2005, M.-A. Lemburg wrote:
> Josiah Carlson wrote:
>> 
>> New superclasses for all built-in types (except for string and unicode,
>> which already subclass from basestring).
>> 
>> int, float, complex (long) : subclass from basenumber
>> tuple, list, set : subclass from basesequence
>> dict : subclass from basemapping
> 
> set should be under basemapping.

Are you sure?  Sets are not actually a mapping; they consist only of
keys.  The Python docs do not include sets under maps, and sets do not
support some of the standard mapping methods (notably keys()).  Raymond
Hettinger has also talked about switching to a different internal
structure for sets.

(Should this discussion move to c.l.py?  Normally I'd think so, but I
think it's critical that the core developers agree about this.  It's
also critical for me to know because I'm writing a book, but that's not
reason enough to stick with python-dev. ;-)
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Don't listen to schmucks on USENET when making legal decisions.  Hire
yourself a competent schmuck."  --USENET schmuck (aka Robert Kern)

From mal at egenix.com  Wed Dec 21 00:52:55 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 21 Dec 2005 00:52:55 +0100
Subject: [Python-Dev] Sets are mappings?
In-Reply-To: <20051220223124.GB6402@panix.com>
References: <fb6fbf560512191051t336daa32pb5a7c0c6929f7e4c@mail.gmail.com>	<43A7F0FE.3050303@zope.com>
	<20051220120244.9470.JCARLSON@uci.edu>	<43A86E53.4000007@egenix.com>
	<20051220223124.GB6402@panix.com>
Message-ID: <43A89957.9040709@egenix.com>

Aahz wrote:
> On Tue, Dec 20, 2005, M.-A. Lemburg wrote:
>> Josiah Carlson wrote:
>>> New superclasses for all built-in types (except for string and unicode,
>>> which already subclass from basestring).
>>>
>>> int, float, complex (long) : subclass from basenumber
>>> tuple, list, set : subclass from basesequence
>>> dict : subclass from basemapping
>> set should be under basemapping.
> 
> Are you sure?  Sets are not actually a mapping; they consist only of
> keys. 

You're right, sets should really have a separate base class.

However, in reality they behave mostly like dictionaries
using (and hiding) a common value of all keys.

> The Python docs do not include sets under maps, and sets do not
> support some of the standard mapping methods (notably keys()).  Raymond
> Hettinger has also talked about switching to a different internal
> structure for sets.

basestring is an abstract class in the sense that it
doesn't provide any interface on its own. I guess the others
should use the same approach.

They are usually only used for quickly checking for an
interface or "type property".

Note that unicode and strings don't share a common implementation
either - they just happen to expose a rather similar interface.

> (Should this discussion move to c.l.py?  Normally I'd think so, but I
> think it's critical that the core developers agree about this.  It's
> also critical for me to know because I'm writing a book, but that's not
> reason enough to stick with python-dev. ;-)

Not sure about others. I rarely read c.l.p. Even pydev has
enough traffic these days to require filtering.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 21 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From martin at v.loewis.de  Wed Dec 21 01:31:45 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 21 Dec 2005 01:31:45 +0100
Subject: [Python-Dev] status of development documentation
In-Reply-To: <do8nun$76d$1@sea.gmane.org>
References: <do8nun$76d$1@sea.gmane.org>
Message-ID: <43A8A271.1020506@v.loewis.de>

Fredrik Lundh wrote:
> or is it time to move away from the current "odd-format-with-extensive-
> semantic-markup-that-nobody-uses-and-only-a-few-people-fully-under-
> stand" approach to something quicker and dirtier [1] ?

If you just want to know what your changes look like: type "make html"
in the Doc directory, and wait a moment for it to complete. I get
xml.etree as section 13.13.

Regards,
Martin

From ncoghlan at gmail.com  Wed Dec 21 09:24:20 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 21 Dec 2005 18:24:20 +1000
Subject: [Python-Dev] Sets are mappings?
In-Reply-To: <20051220223124.GB6402@panix.com>
References: <fb6fbf560512191051t336daa32pb5a7c0c6929f7e4c@mail.gmail.com>	<43A7F0FE.3050303@zope.com>
	<20051220120244.9470.JCARLSON@uci.edu>	<43A86E53.4000007@egenix.com>
	<20051220223124.GB6402@panix.com>
Message-ID: <43A91134.3060103@gmail.com>

Aahz wrote:
> On Tue, Dec 20, 2005, M.-A. Lemburg wrote:
>> Josiah Carlson wrote:
>>> New superclasses for all built-in types (except for string and unicode,
>>> which already subclass from basestring).
>>>
>>> int, float, complex (long) : subclass from basenumber
>>> tuple, list, set : subclass from basesequence
>>> dict : subclass from basemapping
>> set should be under basemapping.
> 
> Are you sure?  Sets are not actually a mapping; they consist only of
> keys.  The Python docs do not include sets under maps, and sets do not
> support some of the standard mapping methods (notably keys()).  Raymond
> Hettinger has also talked about switching to a different internal
> structure for sets.
> 
> (Should this discussion move to c.l.py?  Normally I'd think so, but I
> think it's critical that the core developers agree about this.  It's
> also critical for me to know because I'm writing a book, but that's not
> reason enough to stick with python-dev. ;-)

Close enough to on-topic to stay here, I think. However, I tend to think of 
the taxonomy as a little less flat:

basecontainer (anything with __len__)
   - set
   - basemapping (anything with __getitem__)
     - dict
     - basesequence (anything which understands x[0:0])
         - list
         - tuple
         - string
         - unicode
     - basearray (anything which understands x[0:0,])
         - Numeric.array/scipy.array

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From mcherm at mcherm.com  Wed Dec 21 14:45:38 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed, 21 Dec 2005 05:45:38 -0800
Subject: [Python-Dev] Sets are mappings?
Message-ID: <20051221054538.pd9rhja1gldwgggs@login.werra.lunarpages.com>

Josiah Carlson writes:
> New superclasses for all built-in types (except for string and unicode,
> which already subclass from basestring).
>
> int, float, complex (long) : subclass from basenumber
> tuple, list, set : subclass from basesequence
> dict : subclass from basemapping
>
> The idea is that each of the above classes define a group in which items
> are comparable.


Nick Coghlan writes:
> Close enough to on-topic to stay here, I think. However, I tend to think of
> the taxonomy as a little less flat:
>
> basecontainer (anything with __len__)
>    - set
>    - basemapping (anything with __getitem__)
>      - dict
>      - basesequence (anything which understands x[0:0])
>          - list
>          - tuple
>          - string
>          - unicode
>      - basearray (anything which understands x[0:0,])
>          - Numeric.array/scipy.array


Hold on a sec folks!

I really don't understand why we are trying to build a taxonomy of
container classes. There are some languages which have rather elaborate
taxonomys of container classes. The STL comes to mind, Smalltalk (I
think), even Java's collection classes are somewhat elaborate. But this
is NOT how things have been done in the Python world. We believe that
flat is better than nested. We believe in one simple-and-obvious way
to do things. For goodness sakes, we don't even have a basic
linked-list type because we figure it's simpler to make people just use
the single well-tuned array-list implementation.

Furthermore, I AGREE with this choice. I realize that in THEORY, a list
is simply a bag with the extra feature of ordering, and that a list you
can iterate backward is just an iterate-only-forwards list with an extra
feature. But I have never found it USEFUL in practice. In languages that
support it, I hardly ever find myself saying "well, I'm planning to pass
a list, but this method really only needs a bag... it doesn't matter
whether it is ordered", then later finding that this made it easy to
re-use the method when I had some other bag implementation. Frankly, I
find this sort of re-use MORE likely in Python simply because of support
for duck typing.

So I have a counter-proposal. Let's NOT create a hierarchy of abstract
base types for the elementary types of Python. (Even basestring feels
like a minor wart to me, although for now it seems like we need it.) If
the core problem is "how do you create a canonical ordering for objects
that survives serialization and deserialization into a different VM?",
then somehow abstract base types doesn't seem like the most obvious
solution. And if that's not the problem we're trying to solve here, then
what IS? Because I don't know of very many ACTUAL (as opposed to
theoretical) use cases for abstract base classes of fundamental types.

-- Michael Chermside

From ncoghlan at gmail.com  Wed Dec 21 15:32:26 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 22 Dec 2005 00:32:26 +1000
Subject: [Python-Dev] Sets are mappings?
In-Reply-To: <20051221054538.pd9rhja1gldwgggs@login.werra.lunarpages.com>
References: <20051221054538.pd9rhja1gldwgggs@login.werra.lunarpages.com>
Message-ID: <43A9677A.2080305@gmail.com>

Michael Chermside wrote:
> Nick Coghlan writes:
>> Close enough to on-topic to stay here, I think. However, I tend to think of
>> the taxonomy as a little less flat:
>>
>> basecontainer (anything with __len__)
>>    - set
>>    - basemapping (anything with __getitem__)
>>      - dict
>>      - basesequence (anything which understands x[0:0])
>>          - list
>>          - tuple
>>          - string
>>          - unicode
>>      - basearray (anything which understands x[0:0,])
>>          - Numeric.array/scipy.array

<snip>
> So I have a counter-proposal. Let's NOT create a hierarchy of abstract
> base types for the elementary types of Python. (Even basestring feels
> like a minor wart to me, although for now it seems like we need it.)

Sorry - I meant to indicate that I didn't think the base classes were 
necessary because the relevant checks already existed in a "does it behave 
like one" sense:

   def is_container(x):
     try:
         len(x)
         return True
     except (TypeError, AttributeError):
         return False

   def is_mapping(x):
     return hasattr(x, "__getitem__")

   def is_sequence(x):
     try:
         x[0:0]
         return True
     except LookupError:
         return False

   def is_multiarray(x):
     try:
         x[0:0,]
         return True
     except LookupError:
         return False

I agree it's a definite tangent to the original topic :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From mcherm at mcherm.com  Wed Dec 21 16:00:32 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed, 21 Dec 2005 07:00:32 -0800
Subject: [Python-Dev] Sets are mappings?
Message-ID: <20051221070032.zfgu4qxhouz4cog4@login.werra.lunarpages.com>

Nick Coghlan writes:
> Sorry - I meant to indicate that I didn't think the base classes were
> necessary because the relevant checks already existed in a "does it behave
> like one" sense:
>
>    def is_container(x):
        [...]
>    def is_mapping(x):
        [...]
>    def is_sequence(x):
        [...]
>    def is_multiarray(x):
        [...]

That sounds much more reasonable to me, although I'd also mention
that it is unusual to need to test for the "protocol support" as
you describe. Instead, it usually suffices to just USE the darn
thing and handle failures in an except clause. This is MORE powerful
than the hierarchy you describe, because it winds up testing for
only the features actually needed rather than testing for adherence
to some abstract base class.

An example should make it easy to understand. It is perfectly
reasonable for a container to support __getitem__, but not support
__len__. Perhaps the container uses an algorithm to generate the
items and is effectively of infinite size. In your hierarchy, this
wouldn't even be a basecontainer (and thus, clearly not a
basesequence). But if all you want to do is to use __getitem__ then
it ought to work fine.

-- Michael Chermside


From aahz at pythoncraft.com  Wed Dec 21 17:04:32 2005
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 21 Dec 2005 08:04:32 -0800
Subject: [Python-Dev] Sets are mappings?
In-Reply-To: <20051221054538.pd9rhja1gldwgggs@login.werra.lunarpages.com>
References: <20051221054538.pd9rhja1gldwgggs@login.werra.lunarpages.com>
Message-ID: <20051221160432.GA9293@panix.com>

On Wed, Dec 21, 2005, Michael Chermside wrote:
>
> So I have a counter-proposal. Let's NOT create a hierarchy of abstract
> base types for the elementary types of Python. (Even basestring feels
> like a minor wart to me, although for now it seems like we need
> it.) If the core problem is "how do you create a canonical ordering
> for objects that survives serialization and deserialization into a
> different VM?", then somehow abstract base types doesn't seem like
> the most obvious solution. And if that's not the problem we're trying
> to solve here, then what IS? Because I don't know of very many ACTUAL
> (as opposed to theoretical) use cases for abstract base classes of
> fundamental types.

You've got a good point, but the documentation issue still exists; that's
what I was more interested in.  Clearly lists, tuples, and strings are
sequences; clearly dicts are a mapping; the question is whether sets get
tossed in with dicts.  Overall, I think it's pretty clear that the answer
is "no", particularly given that sets don't support __getitem__().
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Don't listen to schmucks on USENET when making legal decisions.  Hire
yourself a competent schmuck."  --USENET schmuck (aka Robert Kern)

From fredrik at pythonware.com  Wed Dec 21 17:10:24 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 21 Dec 2005 17:10:24 +0100
Subject: [Python-Dev] status of development documentation
References: <do8nun$76d$1@sea.gmane.org> <43A8A271.1020506@v.loewis.de>
Message-ID: <dobupj$f9i$1@sea.gmane.org>

Martin v. Löwis wrote:

> If you just want to know what your changes look like: type "make html"
> in the Doc directory, and wait a moment for it to complete. I get
> xml.etree as section 13.13.

provided you have all the right stuff on your machine, that is:

$ make html
TEXINPUTS=...
+++ TEXINPUTS=...
+++ latex api
*** Session transcript and error messages are in .../Python-2.5/Doc/html/api/api.how.
*** Exited with status 127.
The relevant lines from the transcript are:
------------------------------------------------------------------------
+++ latex api
sh: latex: command not found
*** Session transcript and error messages are in .../Python-2.5/Doc/html/api/api.how.
*** Exited with status 127.
make: *** [html/api/api.html] Error 127

I'm not sure I have enough time to sort this out...

my original questions remain:

- could a cronjob that does this be set up on some python.org machine
(or on some volunteer's machine)

- is it perhaps time to start investigating using "lighter" tools for the core
documentation ?

(as I hinted, I'd prefer HTML with microformat annotations as the main format;
with roundtripping to markdown or rest (etc) for people who prefer to author in
that, and tidy->xhtml->python tools for the HTML generation)

</F>




From theller at python.net  Wed Dec 21 17:32:22 2005
From: theller at python.net (Thomas Heller)
Date: Wed, 21 Dec 2005 17:32:22 +0100
Subject: [Python-Dev] os.startfile with optional second parameter
References: <oe3bshd3.fsf@python.net>
Message-ID: <wthyqull.fsf@python.net>

Thomas Heller <theller at python.net> writes:

> Would a patch be accepted that implemented an optional second parameter
> for the os.startfile function on Windows?
>
> Sometimes I missed the possibility to write
>
>   os.startfile("mydocs.pdf", "print")

The other possibility would be to extend the subprocess module with this
functionality.

Thomas


From theller at python.net  Wed Dec 21 17:31:50 2005
From: theller at python.net (Thomas Heller)
Date: Wed, 21 Dec 2005 17:31:50 +0100
Subject: [Python-Dev] Build failure and problem on Windows
References: <hd93u0lr.fsf@python.net> <8xufu01l.fsf@python.net>
Message-ID: <3bkms96x.fsf@python.net>

Thomas Heller <theller at python.net> writes:

> Thomas Heller <theller at python.net> writes:
>
>> Building the svn trunk on Windows fails because Python\pyarena.c is
>> missing in the pythoncore.vcproj file (I'm not yet up to speed with svn,
>> otherwise I would have checked in a fix for this myself).
>>
>> Worse, when running the built exe it segfaults in Py_GetBuildInfo(),
>> because it is picking up somehow a definition of #define BUILD 'b' (from
>> cPickle.c? Could that be?)
>
> I should have known better, but BUILD is defined in the MSVC project
> file as BUILD=60.

I've committed a fix for both (Hope these comments aren't off-topic
nowadays for python-dev).

Thomas


From steve at holdenweb.com  Wed Dec 21 18:34:24 2005
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 21 Dec 2005 17:34:24 +0000
Subject: [Python-Dev] status of development documentation
In-Reply-To: <dobupj$f9i$1@sea.gmane.org>
References: <do8nun$76d$1@sea.gmane.org> <43A8A271.1020506@v.loewis.de>
	<dobupj$f9i$1@sea.gmane.org>
Message-ID: <doc3n2$2ll$1@sea.gmane.org>

Fredrik Lundh wrote:

> 
> - is it perhaps time to start investigating using "lighter" tools for the core
> documentation ?
> 
+1

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From jcarlson at uci.edu  Wed Dec 21 19:02:33 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 21 Dec 2005 10:02:33 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <doc3n2$2ll$1@sea.gmane.org>
References: <dobupj$f9i$1@sea.gmane.org> <doc3n2$2ll$1@sea.gmane.org>
Message-ID: <20051221095628.BE79.JCARLSON@uci.edu>


Steve Holden <steve at holdenweb.com> wrote:
> 
> Fredrik Lundh wrote:
> 
> > 
> > - is it perhaps time to start investigating using "lighter" tools for the core
> > documentation ?
> > 
> +1

+1 for using ReST.
+0 for sticking with latex.
-1 for choosing something not ReST or latex.

+10 for any language we can generate from the latex sources so that a
complete rewrite is unnecessary.

 - Josiah


From pje at telecommunity.com  Wed Dec 21 19:21:01 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Dec 2005 13:21:01 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <dobupj$f9i$1@sea.gmane.org>
References: <do8nun$76d$1@sea.gmane.org>
 <43A8A271.1020506@v.loewis.de>
Message-ID: <5.1.1.6.0.20051221131334.0279fa90@mail.telecommunity.com>

At 05:10 PM 12/21/2005 +0100, Fredrik Lundh wrote:
>- is it perhaps time to start investigating using "lighter" tools for the core
>documentation ?
>
>(as I hinted, I'd prefer HTML with microformat annotations as the main format;
>with roundtripping to markdown or rest (etc) for people who prefer to 
>author in
>that, and tidy->xhtml->python tools for the HTML generation)

I don't see how HTML is any "lighter" than LaTeX - to me it feels a lot 
heavier, even if you only consider the number of shifted keystrokes needed 
to type it.  And attempting to roundtrip HTML back to reST would lose far 
too much information, like trying to decompile Python bytecode.  I'm +0.5 
for reST, but -1000 for HTML as an authoring format.

The reason I'm only +0.5 for reST is that *any* change from the status quo, 
with so much documentation in existence, has a very high standard to 
meet.  If there were no existing docs to convert, I'd be +1 on reST.


From fredrik at pythonware.com  Wed Dec 21 19:22:24 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 21 Dec 2005 19:22:24 +0100
Subject: [Python-Dev] status of development documentation
References: <dobupj$f9i$1@sea.gmane.org> <doc3n2$2ll$1@sea.gmane.org>
	<20051221095628.BE79.JCARLSON@uci.edu>
Message-ID: <doc6h2$dsi$1@sea.gmane.org>

Josiah Carlson wrote:

> -1 for choosing something not ReST or latex.

yeah, because using something that everyone else uses would of course
not be the python way.

</F>




From greg at electricrain.com  Wed Dec 21 19:26:00 2005
From: greg at electricrain.com (Gregory P. Smith)
Date: Wed, 21 Dec 2005 10:26:00 -0800
Subject: [Python-Dev] hashlib - faster md5/sha, adds sha256/512 support
In-Reply-To: <bbaeab100512161450j1f6b3a2fh4af13c1127c96a15@mail.gmail.com>
References: <6.2.5.6.2.20051216102116.02c32d60@mit.edu>
	<1f7befae0512161226k524646f6j3e8e9a78378e2f5a@mail.gmail.com>
	<bbaeab100512161450j1f6b3a2fh4af13c1127c96a15@mail.gmail.com>
Message-ID: <20051221182600.GC25059@zot.electricrain.com>

On Fri, Dec 16, 2005 at 02:50:36PM -0800, Brett Cannon wrote:
> On 12/16/05, Tim Peters <tim.peters at gmail.com> wrote:
> [SNIP]
> > python-dev'ers:  I failed to find anything in the trunk's NEWS file
> > about this (neither about `hashlib`, nor about any of the specific new
> > hash functions).  It's not like it isn't newsworthy ;-)
> 
> I have fixed the faux pas and added an entry.

thanks :)


From greg at electricrain.com  Wed Dec 21 19:33:24 2005
From: greg at electricrain.com (Gregory P. Smith)
Date: Wed, 21 Dec 2005 10:33:24 -0800
Subject: [Python-Dev] Incorporation of zlib sources into Python
	subversion
In-Reply-To: <43A53572.9020802@v.loewis.de>
References: <43A53572.9020802@v.loewis.de>
Message-ID: <20051221183324.GE25059@zot.electricrain.com>

On Sun, Dec 18, 2005 at 11:09:54AM +0100, "Martin v. L?wis" wrote:
> Thomas (Heller) and I have been discussing whether the zlib
> module should become builtin, atleast on Win32 (i.e. part
> of python25.dll). This would simplify py2exe, which then could
> bootstrap extraction from the compressed file just with
> pythonxy.dll (clearly, zlib.pyd cannot be *in* the compressed
> file).

That makes sense.

One note of caution... zlib has has several security vulnerabilities
revealed in the past.  zlib 1.1.x (4?) seems to have had less than the
more recent 1.2.x zlibs so it may be prudent to play conservative and
stick with the older one to avoid chances of having to release a
python security update when zlib bugs are found.

(i don't know what version python uses today maybe this is a non issue?)

> Whether or not this copy of zlib would be integrated in the
> Unix build process, in case where the system does not provide
> a zlib, is a separate question.

scary to think of a system without zlib.  tsk tsk on whoever makes those.

-g

From fdrake at acm.org  Wed Dec 21 19:35:09 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 21 Dec 2005 13:35:09 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <20051221095628.BE79.JCARLSON@uci.edu>
References: <dobupj$f9i$1@sea.gmane.org> <doc3n2$2ll$1@sea.gmane.org>
	<20051221095628.BE79.JCARLSON@uci.edu>
Message-ID: <200512211335.09433.fdrake@acm.org>

[Copied to the Doc-SIG list.]

On Wednesday 21 December 2005 13:02, Josiah Carlson wrote:
 > +1 for using ReST.
 > +0 for sticking with latex.

I'll try and spend a little time on this issue this week, but time is hard to 
come by these days.

ReST (as implemented in docutils) at this point does *not* support nested 
markup constructs, unless something has changed in the last few months.  I 
think this is a significant limitation.

LaTeX, for all the tool requirements, is a fairly light-weight markup 
language.  Yes, it has too many special characters.  But someone else 
invented it, and I'm not keen on inventing any more than we have to.

There is the matter of all the semantic markup we're doing in the LaTeX 
sources; some people think it's fine, and others think using a specialized 
semantic markup is either a bad idea or at the least a barrier to 
contributions (though I've pointed out that contributing just plain text is 
fine many, many times).  Alternatives to the semantic markup that I expect to 
see suggested include:

nothing special, just using presentation markup directly:
  This prevents even simple information re-use.  Conventions can help, but
  require a careful eye on the part of editors (possibly with tools to help).

something like HTML, but with "microformat" style annotations:
  More reasonable, especially if we rely on conventions and stylesheets for
  presentation.  I expect the markup will actually be much heavier than the
  current markup, though it will be somewhat more familiar to someone when
  they first look at it.  Adding in the annotations changes that a bit.

docbook, because others use that:
  This is really heavy, but tools exist.  The last I looked at the OOP
  extensions, they were fairly simple, but not well matched to Python.

ReST, possibly with additional interpreted text roles:
  This has been explored in the past, and would likely not be a bad approach.
  As noted above, I expect non-support for nested markup in docutils to be a
  problem that will become evident fairly quickly.

All that said, I think this discussion belongs on the Doc-SIG; I've CC'd that 
list.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From fredrik at pythonware.com  Wed Dec 21 19:33:41 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 21 Dec 2005 19:33:41 +0100
Subject: [Python-Dev] status of development documentation
References: <do8nun$76d$1@sea.gmane.org> <43A8A271.1020506@v.loewis.de>
	<dobupj$f9i$1@sea.gmane.org>
	<5.1.1.6.0.20051221131334.0279fa90@mail.telecommunity.com>
Message-ID: <doc767$gen$1@sea.gmane.org>

Phillip J. Eby wrote:

> >(as I hinted, I'd prefer HTML with microformat annotations as the main format;
> >with roundtripping to markdown or rest (etc) for people who prefer to
> >author in that, and tidy->xhtml->python tools for the HTML generation)
>
> I don't see how HTML is any "lighter" than LaTeX - to me it feels a lot
> heavier, even if you only consider the number of shifted keystrokes needed
> to type it.

umm.  I was thinking "light" in terms of

 - tools required for the processing chain
 - the chance that someone new to python actually knows the stuff
 - support for the format in widely used word processing tools

and you're talking about

 - number of keystrokes in a vintage text editor with no syntax support

since I prefer to avoid "whitespace vs. braces" arguments, let's leave it
there.

> And attempting to roundtrip HTML back to reST would lose far too much
> information

in a less dogmatic Python universe, that would be considered a major
design flaw in ReST.

</F>




From greg at electricrain.com  Wed Dec 21 19:28:10 2005
From: greg at electricrain.com (Gregory P. Smith)
Date: Wed, 21 Dec 2005 10:28:10 -0800
Subject: [Python-Dev] hashlib - faster md5/sha, adds sha256/512 support
In-Reply-To: <1f7befae0512161226k524646f6j3e8e9a78378e2f5a@mail.gmail.com>
References: <6.2.5.6.2.20051216102116.02c32d60@mit.edu>
	<1f7befae0512161226k524646f6j3e8e9a78378e2f5a@mail.gmail.com>
Message-ID: <20051221182810.GD25059@zot.electricrain.com>

> A new core `hashlib` module will be included in Python 2.5, but will
> not be backported to older Python versions.  It includes new
> implementations for SHA-224, -256, -384 and -512.  The code and tests
> are already written, and can be gotten from Python's SVN trunk.

Another thing I intended to do is package hashlib as standalone to
make it available as an addon for python 2.3 and 2.4 users.  Obviously
I haven't gotten around to that yet but it remains on my TODO list.

-g

From amk at amk.ca  Wed Dec 21 20:37:05 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Wed, 21 Dec 2005 14:37:05 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <dobupj$f9i$1@sea.gmane.org>
References: <do8nun$76d$1@sea.gmane.org> <43A8A271.1020506@v.loewis.de>
	<dobupj$f9i$1@sea.gmane.org>
Message-ID: <20051221193705.GA12847@rogue.amk.ca>

On Wed, Dec 21, 2005 at 05:10:24PM +0100, Fredrik Lundh wrote:
> (as I hinted, I'd prefer HTML with microformat annotations as the
> main format; with roundtripping to markdown or rest (etc) for people
> who prefer to author in that, and tidy->xhtml->python tools for the
> HTML generation)

I don't see how HTML can be used to support printed versions of the
docs (e.g. PostScript, PDF).  Even if you generated one big HTML file
instead of a zillion section-by-section files, web browsers are
terrible at printing.  I don't know how you could get a table of
contents that refers you to the actual pages, for example.  Are there
any HTML-to-print converters that are better?

reST is a possibility, though I don't think anyone has worked on
building the required toolchain.  Fred has a LaTeX-to-XML-format
converter kicking around somewhere, but the toolchain has never gotten
good enough to permit making that final transition.

--amk

From walter at livinglogic.de  Wed Dec 21 19:55:42 2005
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed, 21 Dec 2005 19:55:42 +0100
Subject: [Python-Dev] status of development documentation
In-Reply-To: <20051221193705.GA12847@rogue.amk.ca>
References: <do8nun$76d$1@sea.gmane.org>
	<43A8A271.1020506@v.loewis.de>	<dobupj$f9i$1@sea.gmane.org>
	<20051221193705.GA12847@rogue.amk.ca>
Message-ID: <43A9A52E.8080601@livinglogic.de>

A.M. Kuchling wrote:

> On Wed, Dec 21, 2005 at 05:10:24PM +0100, Fredrik Lundh wrote:
> 
>>(as I hinted, I'd prefer HTML with microformat annotations as the
>>main format; with roundtripping to markdown or rest (etc) for people
>>who prefer to author in that, and tidy->xhtml->python tools for the
>>HTML generation)
> 
> I don't see how HTML can be used to support printed versions of the
> docs (e.g. PostScript, PDF).  Even if you generated one big HTML file
> instead of a zillion section-by-section files, web browsers are
> terrible at printing.  I don't know how you could get a table of
> contents that refers you to the actual pages, for example.  Are there
> any HTML-to-print converters that are better?

Why not use our own XML format? The element names could be the same as 
the names of the LaTeX macros. Converting to HTML and DocBook should be 
semi-trivial.

> reST is a possibility, though I don't think anyone has worked on
> building the required toolchain.  Fred has a LaTeX-to-XML-format
> converter kicking around somewhere,

Is this available somewhere?

> but the toolchain has never gotten
> good enough to permit making that final transition.

Bye,
    Walter D?rwald

From fredrik at pythonware.com  Wed Dec 21 19:59:08 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 21 Dec 2005 19:59:08 +0100
Subject: [Python-Dev] status of development documentation
References: <dobupj$f9i$1@sea.gmane.org>
	<doc3n2$2ll$1@sea.gmane.org><20051221095628.BE79.JCARLSON@uci.edu>
	<200512211335.09433.fdrake@acm.org>
Message-ID: <doc8lu$lla$1@sea.gmane.org>

Fred L. Drake, Jr. wrote:

> LaTeX, for all the tool requirements, is a fairly light-weight markup
> language.  Yes, it has too many special characters.  But someone else
> invented it, and I'm not keen on inventing any more than we have to.

"someone else invented it" is of course why I'm advocating an HTML-
based format.  There's a huge infrastructure, both on the tool side and
on the spec side, that deals with (X)HTML.  And *everyone* knows
how to write HTML.

> nothing special, just using presentation markup directly:
>   This prevents even simple information re-use.  Conventions can help, but
>   require a careful eye on the part of editors (possibly with tools to help).
>
> something like HTML, but with "microformat" style annotations:
>   More reasonable, especially if we rely on conventions and stylesheets for
>   presentation.  I expect the markup will actually be much heavier than the
>   current markup, though it will be somewhat more familiar to someone when
>   they first look at it.  Adding in the annotations changes that a bit.

Light annotations plus simple conventions (with corresponding simple tools)
should be more than good enough to match the current level.

> docbook, because others use that:
>   This is really heavy, but tools exist.  The last I looked at the OOP
>   extensions, they were fairly simple, but not well matched to Python.
>
> ReST, possibly with additional interpreted text roles:
>   This has been explored in the past, and would likely not be a bad approach.
>   As noted above, I expect non-support for nested markup in docutils to be a
>   problem that will become evident fairly quickly.
>
> All that said, I think this discussion belongs on the Doc-SIG; I've CC'd that
> list.

The doc-sig didn't look too active when I checked the archives, but maybe
it's time to change that.

</F>




From pje at telecommunity.com  Wed Dec 21 20:10:09 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Dec 2005 14:10:09 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <doc767$gen$1@sea.gmane.org>
References: <do8nun$76d$1@sea.gmane.org> <43A8A271.1020506@v.loewis.de>
	<dobupj$f9i$1@sea.gmane.org>
	<5.1.1.6.0.20051221131334.0279fa90@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051221134931.03d3c240@mail.telecommunity.com>

At 07:33 PM 12/21/2005 +0100, Fredrik Lundh wrote:
> > And attempting to roundtrip HTML back to reST would lose far too much
> > information
>
>in a less dogmatic Python universe, that would be considered a major
>design flaw in ReST.

Since when is having a more expressive source language than HTML a flaw?  :)


From fredrik at pythonware.com  Wed Dec 21 20:08:45 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 21 Dec 2005 20:08:45 +0100
Subject: [Python-Dev] status of development documentation
References: <do8nun$76d$1@sea.gmane.org>
	<43A8A271.1020506@v.loewis.de><dobupj$f9i$1@sea.gmane.org>
	<20051221193705.GA12847@rogue.amk.ca>
Message-ID: <doc97v$np9$1@sea.gmane.org>

A.M. Kuchling wrote:

> I don't see how HTML can be used to support printed versions of the
> docs (e.g. PostScript, PDF).  Even if you generated one big HTML file
> instead of a zillion section-by-section files, web browsers are
> terrible at printing.  I don't know how you could get a table of
> contents that refers you to the actual pages, for example.  Are there
> any HTML-to-print converters that are better?

http://www.openoffice.org/

</F>




From jcarlson at uci.edu  Wed Dec 21 20:10:23 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 21 Dec 2005 11:10:23 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <doc6h2$dsi$1@sea.gmane.org>
References: <20051221095628.BE79.JCARLSON@uci.edu> <doc6h2$dsi$1@sea.gmane.org>
Message-ID: <20051221105132.BE7C.JCARLSON@uci.edu>


"Fredrik Lundh" <fredrik at pythonware.com> wrote:
> 
> Josiah Carlson wrote:
> 
> > -1 for choosing something not ReST or latex.
> 
> yeah, because using something that everyone else uses would of course
> not be the python way.

No, because ReST is significantly easier to learn and use than basically
every other markup language I've gotten my hands on.  Also, considering
that we are talking about documenting Python, perhaps using Perl or Ruby
for the generation of Python documentation would be right out, but
Python is perfectly reasonable - regardless of what 'everyone else uses'
(which is a poor reason to use a tool).

So far our alternatives to latex or ReST have been html, docbook, or our
own XML. Though docbook and XML (thankfully) leave formatting up to the
converter, all suffer from ML-itis (hard to write, hard to read, hard to
maintain, syntax highlighting matters, ...), though has the benefit that
it can at least be partially generated from the latex source - Walter
just mentioned Fred's latex->XML converter.  Depending on the output of
this coverter, it may be very reasonable to convert it to ReST, or
perhaps some other markup that is determined to be the rightful
destination.


 - Josiah


From skip at pobox.com  Wed Dec 21 20:24:56 2005
From: skip at pobox.com (skip@pobox.com)
Date: Wed, 21 Dec 2005 13:24:56 -0600
Subject: [Python-Dev] status of development documentation
In-Reply-To: <doc8lu$lla$1@sea.gmane.org>
References: <dobupj$f9i$1@sea.gmane.org> <doc3n2$2ll$1@sea.gmane.org>
	<20051221095628.BE79.JCARLSON@uci.edu>
	<200512211335.09433.fdrake@acm.org> <doc8lu$lla$1@sea.gmane.org>
Message-ID: <17321.44040.97252.894883@montanaro.dyndns.org>


    Fredrik> "someone else invented it" is of course why I'm advocating an
    Fredrik> HTML- based format. 

Of course, someone also invented HTML and TeX+LaTeX predates HTML by quite a
bit.

    Fredrik> And *everyone* knows how to write HTML.

That's debatable.  Maybe most people in the python-dev community know how.
Even within this communitiy I suspect there are at least a few people who
normally use something else (like Word) to generate HTML for them.  I
suspect to use the microformat stuff you'd have to restrict your authoring
toolchain substantially, perhaps restricting it to plain old text editors.

Skip

From amk at amk.ca  Wed Dec 21 21:25:32 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Wed, 21 Dec 2005 15:25:32 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <43A9A52E.8080601@livinglogic.de>
References: <do8nun$76d$1@sea.gmane.org> <43A8A271.1020506@v.loewis.de>
	<dobupj$f9i$1@sea.gmane.org> <20051221193705.GA12847@rogue.amk.ca>
	<43A9A52E.8080601@livinglogic.de>
Message-ID: <20051221202532.GA12920@rogue.amk.ca>

On Wed, Dec 21, 2005 at 07:55:42PM +0100, Walter D?rwald wrote:
> >reST is a possibility, though I don't think anyone has worked on
> >building the required toolchain.  Fred has a LaTeX-to-XML-format
> >converter kicking around somewhere,
> 
> Is this available somewhere?

Docs/tools/sgmlconv/, I think.  The code's age is apparent from the
README saying "Python 2.0 is required."

--amk

From fredrik at pythonware.com  Wed Dec 21 20:21:23 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 21 Dec 2005 20:21:23 +0100
Subject: [Python-Dev] status of development documentation
References: <do8nun$76d$1@sea.gmane.org>
	<43A8A271.1020506@v.loewis.de><dobupj$f9i$1@sea.gmane.org><5.1.1.6.0.20051221131334.0279fa90@mail.telecommunity.com>
	<doc767$gen$1@sea.gmane.org>
	<5.1.1.6.0.20051221134931.03d3c240@mail.telecommunity.com>
Message-ID: <doc9vl$qfr$1@sea.gmane.org>

Phillip J. Eby wrote:

> > > And attempting to roundtrip HTML back to reST would lose far too much
> > > information
> >
> >in a less dogmatic Python universe, that would be considered a major
> >design flaw in ReST.
>
> Since when is having a more expressive source language than HTML a flaw?  :)

more syntax != more expressive.

</F>




From fredrik at pythonware.com  Wed Dec 21 20:36:12 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 21 Dec 2005 20:36:12 +0100
Subject: [Python-Dev] status of development documentation
References: <20051221095628.BE79.JCARLSON@uci.edu> <doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu>
Message-ID: <docare$tjs$1@sea.gmane.org>

Josiah Carlson wrote:

> > yeah, because using something that everyone else uses would of course
> > not be the python way.
>
> No, because ReST is significantly easier to learn and use than basically
> every other markup language I've gotten my hands on.

I'm not really interested in optimizing for you, I'm interested in optimizing
for everyone else.  They already know HTML.  They don't know ReST, and
I doubt they care about it (how many blogs accept ReST for comments?)

</F>




From ianb at colorstudy.com  Wed Dec 21 20:43:21 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 21 Dec 2005 13:43:21 -0600
Subject: [Python-Dev] status of development documentation
In-Reply-To: <17321.44040.97252.894883@montanaro.dyndns.org>
References: <dobupj$f9i$1@sea.gmane.org>
	<doc3n2$2ll$1@sea.gmane.org>	<20051221095628.BE79.JCARLSON@uci.edu>	<200512211335.09433.fdrake@acm.org>
	<doc8lu$lla$1@sea.gmane.org>
	<17321.44040.97252.894883@montanaro.dyndns.org>
Message-ID: <43A9B059.9060900@colorstudy.com>

skip at pobox.com wrote:
>     Fredrik> And *everyone* knows how to write HTML.
> 
> That's debatable.  Maybe most people in the python-dev community know how.
> Even within this communitiy I suspect there are at least a few people who
> normally use something else (like Word) to generate HTML for them.  I
> suspect to use the microformat stuff you'd have to restrict your authoring
> toolchain substantially, perhaps restricting it to plain old text editors.

If we were using a microformat, it is likely that the CSS class would be 
used to mark content.  At least that's what I've noticed in some recent 
microformat specs, and there's lots of good reasons to follow that. 
Tool support for adding classes to elements is relatively good; not 
great from what I can tell, but good.  Not that I use a lot of these 
editing tools, so I might be wrong.

Still, the output of WYSIWYG tools remains very poor.  Because not 
everyone will be using WYSIWYG tools, it is likely that any such output 
will be to be cleaned -- reindented, and probably with any unrecognized 
styling removed.  But this isn't that hard.

Also, I assume that most documentation maintainers will continue to use 
text editors, because programmers use text editors, and this is 
programer documentation.  I think it is very reasonable to expect people 
to know HTML; I find it unlikely that many people will enjoy authoring 
HTML.  I know HTML quite well, I continue to write lots of it, and I've 
never enjoyed writing programming documentation in HTML.  I guess in 
practice I write very little HTML *content*, just structure, and when 
I'm writing structure I don't mind the markup.  But when I want to focus 
on content the markup is very distracting, and even moreso when writing 
about programming (where ASCII, newlines, and whitespace is the native 
layout technique).

To me, using HTML feels like sacrificing the authoring experience for 
expedient tools.  This doesn't seem like a big step forward from LaTeX.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Wed Dec 21 20:54:23 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Dec 2005 14:54:23 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <doc9vl$qfr$1@sea.gmane.org>
References: <do8nun$76d$1@sea.gmane.org> <43A8A271.1020506@v.loewis.de>
	<dobupj$f9i$1@sea.gmane.org>
	<5.1.1.6.0.20051221131334.0279fa90@mail.telecommunity.com>
	<doc767$gen$1@sea.gmane.org>
	<5.1.1.6.0.20051221134931.03d3c240@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051221144132.03d1ce98@mail.telecommunity.com>

At 08:21 PM 12/21/2005 +0100, Fredrik Lundh wrote:
>Phillip J. Eby wrote:
>
> > > > And attempting to roundtrip HTML back to reST would lose far too much
> > > > information
> > >
> > >in a less dogmatic Python universe, that would be considered a major
> > >design flaw in ReST.
> >
> > Since when is having a more expressive source language than HTML a 
> flaw?  :)
>
>more syntax != more expressive.

reST is more expressive than HTML in terms of allowing meaningful choices 
for readability and *human* expression.  In reST, I have the choice of 
inlining a URL or deferring it to later, according to what's readable.  I 
can give links friendly names, and so on.

Your statement that more syntax != more expressive is true, but also 
irrelevant, because it doesn't imply any useful conclusions.

Python is more expressive than Java because of the syntax it adds, relative 
to Java.  Specialized syntax for lists and dictionaries, mappings, sequence 
iteration, etc. are precisely the things that make it more expressive for 
the human reader or writer of code.

But the thing that makes it more expressive is not the quantity of syntax, 
but the balanced selection of *task-appropriate* syntax for *human* 
use.  More syntax doesn't always mean more expressiveness or readability, 
but less syntax can often mean less expressiveness, readability, and usability.


From pje at telecommunity.com  Wed Dec 21 21:07:08 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Dec 2005 15:07:08 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <docare$tjs$1@sea.gmane.org>
References: <20051221095628.BE79.JCARLSON@uci.edu> <doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu>
Message-ID: <5.1.1.6.0.20051221145613.03d1d6e8@mail.telecommunity.com>

At 08:36 PM 12/21/2005 +0100, Fredrik Lundh wrote:
>Josiah Carlson wrote:
>
> > > yeah, because using something that everyone else uses would of course
> > > not be the python way.
> >
> > No, because ReST is significantly easier to learn and use than basically
> > every other markup language I've gotten my hands on.
>
>I'm not really interested in optimizing for you, I'm interested in optimizing
>for everyone else.  They already know HTML.  They don't know ReST, and
>I doubt they care about it (how many blogs accept ReST for comments?)

I think you're asking the wrong question.  A better one is, how many blogs 
require valid HTML for comments, without offering any user-friendly bits 
like converting line feeds and paragraph breaks to BR and P for you?  How 
many blogs offer other humane formats like Textile and Markdown?  (Neither 
of which is very different from a stripped-down and underspecified version 
of reST.)

If anything, I'd think that the fact that blogs found it necessary to 
invent reST-like formats implies that far more people can deal with 
reST-like formats than with unadulterated HTML!

In addition to the syntaxes with names like Markdown and Textile and reST, 
I've seen lots of comment systems with their own primitive markups using 
similar approaches.  So, using the infrequent availability of one 
particular humane format in blogging comment software as an argument for 
HTML is missing the forest for the tree.  If you want to use blog comments 
as a test case, the evidence is overwhelming that people *don't* know HTML 
and/or find it hard to use.  Sure, they have to type it in a text box.  But 
you're the one who picked blog comments as an example.

In any case, blog comments rarely need the full expressiveness of 
reST.  You're not going to need section headings and intra-document links, 
file inclusion, footnotes, etc. in a blog comment, so it's natural that 
anybody inventing their own format is either going to try and make HTML 
more humane, or invent a reST-like mini-markup ala Textile or Markdown.




From pje at telecommunity.com  Wed Dec 21 21:10:37 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Dec 2005 15:10:37 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <43A9B059.9060900@colorstudy.com>
References: <17321.44040.97252.894883@montanaro.dyndns.org>
	<dobupj$f9i$1@sea.gmane.org> <doc3n2$2ll$1@sea.gmane.org>
	<20051221095628.BE79.JCARLSON@uci.edu>
	<200512211335.09433.fdrake@acm.org> <doc8lu$lla$1@sea.gmane.org>
	<17321.44040.97252.894883@montanaro.dyndns.org>
Message-ID: <5.1.1.6.0.20051221150745.03d1aa38@mail.telecommunity.com>

At 01:43 PM 12/21/2005 -0600, Ian Bicking wrote:
>  But when I want to focus
>on content the markup is very distracting, and even moreso when writing
>about programming (where ASCII, newlines, and whitespace is the native
>layout technique).

And where characters like '<' and '>' occur frequently as part of the text, 
especially in showing Python interactions like this:

     >>> print "hello world"
     hello world

I can't imagine trying to author the above in an HTML/XML based format, 
whereas in reST (or even LaTeX) I can just copy and paste it from an 
interpreter window.


From jcarlson at uci.edu  Wed Dec 21 21:15:10 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 21 Dec 2005 12:15:10 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <docare$tjs$1@sea.gmane.org>
References: <20051221105132.BE7C.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
Message-ID: <20051221120532.BE87.JCARLSON@uci.edu>


"Fredrik Lundh" <fredrik at pythonware.com> wrote:
> 
> Josiah Carlson wrote:
> 
> > > yeah, because using something that everyone else uses would of course
> > > not be the python way.
> >
> > No, because ReST is significantly easier to learn and use than basically
> > every other markup language I've gotten my hands on.
> 
> I'm not really interested in optimizing for you, I'm interested in optimizing
> for everyone else.  They already know HTML.  They don't know ReST, and
> I doubt they care about it (how many blogs accept ReST for comments?)

I'm not suggesting that anyone optimize for me.  Re-read my comment. 
Did you re-read it?  Off the top of my head, I can't think of an easier
markup to learn or use that provides a variety of output.  Can you?  Can
anyone?  If so, I'm ready to listen.  Until then, I'm standing by my
opinion that ReST is the easiest language to learn and use for right now,
which is MY criteria for selecting a documentation language.  Not yours? 
Ok, we just have different criteria for selecting a language for
documentation, so please stop suggesting that I want everyone to
"optimize for [me]".

Now, this is documentation for a language and its standard library.  But
since you brought up blogs, should we be offering LJ tags (in use by ~4
million active LJ users), BBCode (used by 10s of millions), or wiki
syntax for markup? In my opinion, marketshare means close to nothing. If
we were going by marketshare, we'd be documenting Python with Java, and
only developing on Windows.


 - Josiah


From barry at python.org  Wed Dec 21 21:16:02 2005
From: barry at python.org (Barry Warsaw)
Date: Wed, 21 Dec 2005 15:16:02 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <docare$tjs$1@sea.gmane.org>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<doc6h2$dsi$1@sea.gmane.org> <20051221105132.BE7C.JCARLSON@uci.edu>
	<docare$tjs$1@sea.gmane.org>
Message-ID: <1135196162.14495.15.camel@geddy.wooz.org>

On Wed, 2005-12-21 at 20:36 +0100, Fredrik Lundh wrote:

> I'm not really interested in optimizing for you, I'm interested in optimizing
> for everyone else.  They already know HTML.  They don't know ReST, and
> I doubt they care about it (how many blogs accept ReST for comments?)

Sorry, but HTML and (even more so) XML are not human-writable. :)  Yeah,
we can all do the simple stuff, but I absolutely hate authoring in HTML,
and it would be a nightmare if the documentation production system
didn't handle lots and lots of magic for you (like weaving in the right
footers, css, etc. -- oh wait, that's ht2html!).

reST is a fine language but it seems more suitable to simpler linear
documents like wiki pages and PEPs, rather than those with complicated
nested structure.

Maybe it's just because I came in late on this thread, but what exactly
is broken about the current LaTeX documentation?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051221/2ad4a68d/attachment.pgp

From bcannon at gmail.com  Wed Dec 21 21:35:02 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Wed, 21 Dec 2005 12:35:02 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <1135196162.14495.15.camel@geddy.wooz.org>
References: <20051221095628.BE79.JCARLSON@uci.edu> <doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
Message-ID: <bbaeab100512211235h26a5c8ads8009cc8178e15676@mail.gmail.com>

On 12/21/05, Barry Warsaw <barry at python.org> wrote:
[SNIP]
> Maybe it's just because I came in late on this thread, but what exactly
> is broken about the current LaTeX documentation?
>

Well, the toolchain is not necessarily installed on everyone's
computer.  Plus not everyone knows LaTeX comparative to other possible
markup languages we could be using.

Personally I am fine with LaTeX, but that is because I *learned* LaTeX
to be able to edit the Python docs and have continued to use it for my
school assignments.

-Brett

From steve at holdenweb.com  Wed Dec 21 21:49:26 2005
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 21 Dec 2005 20:49:26 +0000
Subject: [Python-Dev] status of development documentation
In-Reply-To: <200512212026.jBLKQJWZ027961@theraft.strakt.com>
References: <dobupj$f9i$1@sea.gmane.org>
	<doc3n2$2ll$1@sea.gmane.org>	<20051221095628.BE79.JCARLSON@uci.edu>	<200512211335.09433.fdrake@acm.org>
	<doc8lu$lla$1@sea.gmane.org>	<17321.44040.97252.894883@montanaro.dyndns.org>	<43A9B059.9060900@colorstudy.com>
	<200512212026.jBLKQJWZ027961@theraft.strakt.com>
Message-ID: <docf4r$clu$1@sea.gmane.org>

Laura Creighton wrote:
> Whenever people have demanded that I write documentation in html
> I have always done this:
> 
> <pre>
> all my documentation, as output from a text editor.
> 
> All subsequent formatting to be done by somebody else who doesn't
> find dealing with html as excruciatingly painful as I do.
> </pre>
> 
> I suspect there are lots of people who have concluded that this
> is all the html that you really need.  The question is, are you
> willing to put up with documentation like this from people?
> 
Well the existing system can cope with that style, but for some reason 
the oft-repeated advice that plain text markup is an acceptable format 
for documentation contributions doesn't seem to have escaped the gravity 
field. So that's just as good for the existing docs as anything that 
replaces them (if anything does).

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From bronger at physik.rwth-aachen.de  Wed Dec 21 21:52:04 2005
From: bronger at physik.rwth-aachen.de (Torsten Bronger)
Date: Wed, 21 Dec 2005 21:52:04 +0100
Subject: [Python-Dev] status of development documentation
References: <do8nun$76d$1@sea.gmane.org> <43A8A271.1020506@v.loewis.de>
	<dobupj$f9i$1@sea.gmane.org> <20051221193705.GA12847@rogue.amk.ca>
Message-ID: <87d5jqw4uj.fsf@wilson.rwth-aachen.de>

Hall?chen!

"A.M. Kuchling" <amk at amk.ca> writes:

> On Wed, Dec 21, 2005 at 05:10:24PM +0100, Fredrik Lundh wrote:
>
>> (as I hinted, I'd prefer HTML with microformat annotations as the
>> main format; with roundtripping to markdown or rest (etc) for
>> people who prefer to author in that, and tidy->xhtml->python
>> tools for the HTML generation)
>
> I don't see how HTML can be used to support printed versions of the
> docs (e.g. PostScript, PDF).

I've used XSLT heavily for converting XML/XHTML to PDF.  It was
pretty easy, and the result was of very high typographic quality.
The only disadvantage is that XSLT is *slow*.

My standard approach was to convert XML to LaTeX and to substitute
all unicodes with LaTeX commands.  Thus, the depenencies are LaTeX,
an XSLT processor (Saxon), and a tiny program for the substitutions.
(The latter can be avoided by LaTeX's Unicode package; however,
expect problems in some cases.)

> [...]  Are there any HTML-to-print converters that are better?

I don't understand exactly how the HTML is to be used for Python but
I assume that not everything could be done via CSS, so own
converters will be necessary for perfect output.  Alternatively, you
can use XSLT so that the browser can convert the original document
to a printable document (with table of contents, index etc).  For
perfect typography you need LaTeX though.

> reST is a possibility, though I don't think anyone has worked on
> building the required toolchain.

I used reST last spring for a small package project.  Although I
love its goals (reST as well as Wiki languages are a perfect
"front-end" for the XML family), I was disappointed with its rather
small semantic vocabulary.  I felt forced to use visual markup
tricks and things like that.  If nothing significant has changed, I
think that reST is too young for a really big project.

Tsch?,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus            ICQ 264-296-646


From fredrik at pythonware.com  Wed Dec 21 21:53:30 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 21 Dec 2005 21:53:30 +0100
Subject: [Python-Dev] status of development documentation
References: <17321.44040.97252.894883@montanaro.dyndns.org><dobupj$f9i$1@sea.gmane.org>
	<doc3n2$2ll$1@sea.gmane.org><20051221095628.BE79.JCARLSON@uci.edu><200512211335.09433.fdrake@acm.org>
	<doc8lu$lla$1@sea.gmane.org><17321.44040.97252.894883@montanaro.dyndns.org>
	<43A9B059.9060900@colorstudy.com>
	<5.1.1.6.0.20051221150745.03d1aa38@mail.telecommunity.com>
Message-ID: <docfcc$dit$1@sea.gmane.org>

Phillip J. Eby wrote:

> And where characters like '<' and '>' occur frequently as part of the text,
> especially in showing Python interactions like this:
>
>      >>> print "hello world"
>      hello world
>
> I can't imagine trying to author the above in an HTML/XML based format,

it's spelled

    >>> print "hello world"
    hello world

in HTML.

</F>




From pje at telecommunity.com  Wed Dec 21 22:00:07 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Dec 2005 16:00:07 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <1135196162.14495.15.camel@geddy.wooz.org>
References: <docare$tjs$1@sea.gmane.org> <20051221095628.BE79.JCARLSON@uci.edu>
	<doc6h2$dsi$1@sea.gmane.org> <20051221105132.BE7C.JCARLSON@uci.edu>
	<docare$tjs$1@sea.gmane.org>
Message-ID: <5.1.1.6.0.20051221152616.0209a2f8@mail.telecommunity.com>

At 03:16 PM 12/21/2005 -0500, Barry Warsaw wrote:
>Maybe it's just because I came in late on this thread, but what exactly
>is broken about the current LaTeX documentation?

As far as I can tell from his comments:

1. Fredrik doesn't want to have to install a LaTeX toolchain in order to 
get an HTML version of the documentation

2. Fredrik likes using whatever tools he has for editing HTML better than 
whatever he has for editing LaTeX

3. Fredrik believes that more people would participate in updating Python 
documentation if it didn't require a LaTeX toolchain or LaTeX-friendly editor.

(Of course, these are equally arguments for using other formats besides 
HTML, especially formats that are closer to plain text.)

By the way, I'm not sure I see what the problem with authoring Python 
documentation with reST would be.  I've written fairly sizable documents 
(at least the size of a large library reference chapter (section?)) with 
both the pythondoc toolchain and with reST.  It seems to me that even the 
largest Python manual is composed of chunks that are that size or smaller, 
so I don't  think I see what constructs would be missing.


From trentm at ActiveState.com  Wed Dec 21 22:37:17 2005
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 21 Dec 2005 13:37:17 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <dobupj$f9i$1@sea.gmane.org>
References: <do8nun$76d$1@sea.gmane.org> <43A8A271.1020506@v.loewis.de>
	<dobupj$f9i$1@sea.gmane.org>
Message-ID: <20051221213717.GA24349@ActiveState.com>

[Fredrik Lundh wrote]
> $ make html
> TEXINPUTS=...
> +++ TEXINPUTS=...
> +++ latex api
> *** Session transcript and error messages are in .../Python-2.5/Doc/html/api/api.how.
> *** Exited with status 127.
> The relevant lines from the transcript are:
> ------------------------------------------------------------------------
> +++ latex api
> sh: latex: command not found
> *** Session transcript and error messages are in .../Python-2.5/Doc/html/api/api.how.
> *** Exited with status 127.
> make: *** [html/api/api.html] Error 127
> 
> I'm not sure I have enough time to sort this out...

For the record... I remember way back that I hit a limitation in
latex2html that disallowed having any hyphens in the path to where the
docs were being built. So your hyphen in "Python-2.5" might be
confounding latex2html there.

Trent

-- 
Trent Mick
TrentM at ActiveState.com

From fredrik at pythonware.com  Wed Dec 21 22:40:45 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 21 Dec 2005 22:40:45 +0100
Subject: [Python-Dev] status of development documentation
References: <20051221095628.BE79.JCARLSON@uci.edu><doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu><docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
Message-ID: <doci50$nom$1@sea.gmane.org>

Barry Warsaw wrote:

> Sorry, but HTML and (even more so) XML are not human-writable. :)  Yeah,
> we can all do the simple stuff, but I absolutely hate authoring in HTML,
> and it would be a nightmare if the documentation production system
> didn't handle lots and lots of magic for you (like weaving in the right
> footers, css, etc. -- oh wait, that's ht2html!).

Sure, and some people hate using whitespace for block structure.

> Maybe it's just because I came in late on this thread, but what exactly
> is broken about the current LaTeX documentation?

Checked the python-list archives lately?  If you google c.l.python for the
word "documentation", you'll find recent megathreads with subjects like
"bitching about the documentation", "opensource documentation problems"
and "python documentation should be better" among the top hits.  But if
you check the bug and patch trackers, you don't find many contributions.
Something's definitely broken.

</F>




From trentm at ActiveState.com  Wed Dec 21 22:42:03 2005
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 21 Dec 2005 13:42:03 -0800
Subject: [Python-Dev] Incorporation of zlib sources into Python
	subversion
In-Reply-To: <20051221183324.GE25059@zot.electricrain.com>
References: <43A53572.9020802@v.loewis.de>
	<20051221183324.GE25059@zot.electricrain.com>
Message-ID: <20051221214203.GB24349@ActiveState.com>

[Gregory P. Smith wrote]
> (i don't know what version python uses today maybe this is a non issue?)

    $ svn cat http://svn.python.org/projects/python/trunk/PCbuild/zlib.vcproj | grep "zlib-"
    ...

zlib 1.2.3



Trent

-- 
Trent Mick
TrentM at ActiveState.com

From barry at python.org  Wed Dec 21 23:15:38 2005
From: barry at python.org (Barry Warsaw)
Date: Wed, 21 Dec 2005 17:15:38 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <doci50$nom$1@sea.gmane.org>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<doc6h2$dsi$1@sea.gmane.org> <20051221105132.BE7C.JCARLSON@uci.edu>
	<docare$tjs$1@sea.gmane.org> <1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org>
Message-ID: <1135203338.14488.29.camel@geddy.wooz.org>

On Wed, 2005-12-21 at 22:40 +0100, Fredrik Lundh wrote:

> > Sorry, but HTML and (even more so) XML are not human-writable. :)  Yeah,
> > we can all do the simple stuff, but I absolutely hate authoring in HTML,
> > and it would be a nightmare if the documentation production system
> > didn't handle lots and lots of magic for you (like weaving in the right
> > footers, css, etc. -- oh wait, that's ht2html!).
> 
> Sure, and some people hate using whitespace for block structure.

A more proper analogy would be people who hate braces and parentheses.
You have to type so many more < and > characters (not to mention &s
and ;s) to make happy-joy html than you have to type \s and {s and }s to
make nice-nice latex.

> > Maybe it's just because I came in late on this thread, but what exactly
> > is broken about the current LaTeX documentation?
> 
> Checked the python-list archives lately?  

That's a joke, right?

> If you google c.l.python for the
> word "documentation", you'll find recent megathreads with subjects like
> "bitching about the documentation", "opensource documentation problems"
> and "python documentation should be better" among the top hits.  But if
> you check the bug and patch trackers, you don't find many contributions.
> Something's definitely broken.

I'm not convinced it's the toolchain though.  People hate writing
documentation.  Getting people to contribute documentation is worse than
pulling teeth.  If people can't install the required toolchain and
they're still highly motivated to write Python documentation, then we
already recommend they just write it in plain text and "someone" will
mark it up.

Heck, I wouldn't mind an xml2latex converter so those that like a
different kind of pain (writing xml vs. installing latex) can still
contribute documentation and we can convert it to back to latex.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051221/1b3d2861/attachment.pgp

From fredrik at pythonware.com  Wed Dec 21 23:33:55 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 21 Dec 2005 23:33:55 +0100
Subject: [Python-Dev] status of development documentation
References: <20051221095628.BE79.JCARLSON@uci.edu><doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu><docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org><doci50$nom$1@sea.gmane.org>
	<1135203338.14488.29.camel@geddy.wooz.org>
Message-ID: <docl8k$2gp$1@sea.gmane.org>

Barry Warsaw wrote:

> > Sure, and some people hate using whitespace for block structure.
>
> A more proper analogy would be people who hate braces and parentheses.
> You have to type so many more < and > characters (not to mention &s
> and ;s) to make happy-joy html than you have to type \s and {s and }s to
> make nice-nice latex.

so what's *your* excuse for not using emacs? ;-)

(if you don't have sgml/html support in your editor, I recommend that you
borrow my swedish keyboard, and see if you really prefer \ { } ` etc over
< > & ...)

> > If you google c.l.python for the
> > word "documentation", you'll find recent megathreads with subjects like
> > "bitching about the documentation", "opensource documentation problems"
> > and "python documentation should be better" among the top hits.  But if
> > you check the bug and patch trackers, you don't find many contributions.
> > Something's definitely broken.
>
> I'm not convinced it's the toolchain though.  People hate writing
> documentation.  Getting people to contribute documentation is worse
> than pulling teeth.

fwiw, I seem to get more contributions to effbot.org via my really silly
HTML useredit feature than python.org gets via the patch tracker...

> If people can't install the required toolchain and they're still highly
> motivated to write Python documentation, then we already recommend
> they just write it in plain text and "someone" will mark it up.

and how motivating is it to have to wait days or weeks to be able to
see how your contribution looks after formatting?

"I had to get up in the morning at four o'clock, travel on train for eight
hours with my punch cards in a shoebox, wait twenty-nine hours for the
control data mainframe to finish, and drink a cup of sulphuric acid, ..."

</F>




From walter at livinglogic.de  Thu Dec 22 00:20:32 2005
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu, 22 Dec 2005 00:20:32 +0100
Subject: [Python-Dev] status of development documentation
In-Reply-To: <20051221202532.GA12920@rogue.amk.ca>
References: <do8nun$76d$1@sea.gmane.org>
	<43A8A271.1020506@v.loewis.de>	<dobupj$f9i$1@sea.gmane.org>
	<20051221193705.GA12847@rogue.amk.ca>	<43A9A52E.8080601@livinglogic.de>
	<20051221202532.GA12920@rogue.amk.ca>
Message-ID: <43A9E340.8030103@livinglogic.de>

A.M. Kuchling wrote:
> On Wed, Dec 21, 2005 at 07:55:42PM +0100, Walter D?rwald wrote:
>>> reST is a possibility, though I don't think anyone has worked on
>>> building the required toolchain.  Fred has a LaTeX-to-XML-format
>>> converter kicking around somewhere,
>> Is this available somewhere?
> 
> Docs/tools/sgmlconv/, I think.  The code's age is apparent from the
> README saying "Python 2.0 is required."

Hmm, I get the following:

$ make -f tools/sgmlconv/Makefile
for DIR in api dist ext lib mac ref ; do \
     (cd $DIR && make -f ../tools/sgmlconv/make.rules TOOLSDIR=../tools 
xml) || exit $? ; done
../tools/sgmlconv/latex2esis.py  abstract.tex abstract.esis1
../tools/sgmlconv/docfixer.py abstract.esis1 abstract.esis
Traceback (most recent call last):
   File "../tools/sgmlconv/docfixer.py", line 1073, in ?
     main()
   File "../tools/sgmlconv/docfixer.py", line 1064, in main
     convert(ifp, ofp)
   File "../tools/sgmlconv/docfixer.py", line 1012, in convert
     fixup_descriptors(doc, fragment)
   File "../tools/sgmlconv/docfixer.py", line 168, in fixup_descriptors
     find_and_fix_descriptors(doc, section)
   File "../tools/sgmlconv/docfixer.py", line 177, in 
find_and_fix_descriptors
     rewrite_descriptor(doc, child)
   File "../tools/sgmlconv/docfixer.py", line 242, in rewrite_descriptor
     sig = methodline_to_signature(doc, children[pos])
   File "../tools/sgmlconv/docfixer.py", line 276, in 
methodline_to_signature
     methodline.removeAttribute("name")
   File 
"/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/_xmlplus/dom/minidom.py", 
line 762, in removeAttribute
     raise xml.dom.NotFoundErr()
xml.dom.NotFoundErr: Node does not exist in this context


Applying the following patch:
===================================================================
--- tools/sgmlconv/docfixer.py  (revision 41780)
+++ tools/sgmlconv/docfixer.py  (working copy)
@@ -273,7 +273,10 @@
      signature.appendChild(doc.createTextNode("\n    "))
      name = doc.createElement("name")
      name.appendChild(doc.createTextNode(methodline.getAttribute("name")))
-    methodline.removeAttribute("name")
+    try:
+       methodline.removeAttribute("name")
+    except xml.dom.NotFoundErr:
+       pass
      signature.appendChild(name)
      if len(methodline.childNodes):
          args = doc.createElement("args")

gives me this error:

Traceback (most recent call last):
   File "../tools/sgmlconv/docfixer.py", line 1076, in ?
     main()
   File "../tools/sgmlconv/docfixer.py", line 1067, in main
     convert(ifp, ofp)
   File "../tools/sgmlconv/docfixer.py", line 1044, in convert
     write_esis(fragment, ofp, knownempty)
   File "../tools/sgmlconv/docfixer.py", line 978, in write_esis
     write_esis(node, ofp, knownempty)
   File "../tools/sgmlconv/docfixer.py", line 978, in write_esis
     write_esis(node, ofp, knownempty)
   File "../tools/sgmlconv/docfixer.py", line 978, in write_esis
     write_esis(node, ofp, knownempty)
   File "../tools/sgmlconv/docfixer.py", line 978, in write_esis
     write_esis(node, ofp, knownempty)
   File "../tools/sgmlconv/docfixer.py", line 968, in write_esis
     raise ValueError, \
ValueError: declared-empty node <cmemberline> has children

Commenting out the node.hasChildNodes() check in docfixer.write_esis() 
gives me:

Traceback (most recent call last):
   File "../tools/sgmlconv/docfixer.py", line 1076, in ?
     main()
   File "../tools/sgmlconv/docfixer.py", line 1067, in main
     convert(ifp, ofp)
   File "../tools/sgmlconv/docfixer.py", line 1032, in convert
     if fragment.lastChild.data[-1:] == "\n":
AttributeError: Element instance has no attribute 'data'

Is there any change of getting this to work?

Bye,
    Walter D?rwald


From martin at v.loewis.de  Thu Dec 22 01:33:18 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 22 Dec 2005 01:33:18 +0100
Subject: [Python-Dev] status of development documentation
In-Reply-To: <dobupj$f9i$1@sea.gmane.org>
References: <do8nun$76d$1@sea.gmane.org> <43A8A271.1020506@v.loewis.de>
	<dobupj$f9i$1@sea.gmane.org>
Message-ID: <43A9F44E.9090004@v.loewis.de>

Fredrik Lundh wrote:
> - could a cronjob that does this be set up on some python.org machine
> (or on some volunteer's machine)

My understanding is: not easily. Somebody would have to invest time, of
course. And then there is the issue of the build failing due to syntax
errors in the input.

> - is it perhaps time to start investigating using "lighter" tools for the core
> documentation ?

Not my time, definitely. It's a larger task than I could afford to
tackle for the next, say, five years.

Regards,
Martin


From martin at v.loewis.de  Thu Dec 22 01:40:36 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 22 Dec 2005 01:40:36 +0100
Subject: [Python-Dev] status of development documentation
In-Reply-To: <5.1.1.6.0.20051221152616.0209a2f8@mail.telecommunity.com>
References: <docare$tjs$1@sea.gmane.org>
	<20051221095628.BE79.JCARLSON@uci.edu>	<doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu>	<docare$tjs$1@sea.gmane.org>
	<5.1.1.6.0.20051221152616.0209a2f8@mail.telecommunity.com>
Message-ID: <43A9F604.2050303@v.loewis.de>

Phillip J. Eby wrote:
> 1. Fredrik doesn't want to have to install a LaTeX toolchain in order to 
> get an HTML version of the documentation
> 
> 2. Fredrik likes using whatever tools he has for editing HTML better than 
> whatever he has for editing LaTeX
> 
> 3. Fredrik believes that more people would participate in updating Python 
> documentation if it didn't require a LaTeX toolchain or LaTeX-friendly editor.
> 
> (Of course, these are equally arguments for using other formats besides 
> HTML, especially formats that are closer to plain text.)

Except, of course, for any other format (than HTML), you would have to
substitute "Fredrik" by somebody promoting that other format.

> By the way, I'm not sure I see what the problem with authoring Python 
> documentation with reST would be.

Really not? How do we get from where we are to where you would like
us to be? With this, I mean both technically (but perhaps I'm unaware
of some tool that does the conversion automatically and lossless)
and emotionally (but perhaps everybody but Fredrik and Barry could agree
to switch to reST).

Regards,
Martin

From pje at telecommunity.com  Thu Dec 22 02:24:42 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Dec 2005 20:24:42 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <43A9F604.2050303@v.loewis.de>
References: <5.1.1.6.0.20051221152616.0209a2f8@mail.telecommunity.com>
	<docare$tjs$1@sea.gmane.org> <20051221095628.BE79.JCARLSON@uci.edu>
	<doc6h2$dsi$1@sea.gmane.org> <20051221105132.BE7C.JCARLSON@uci.edu>
	<docare$tjs$1@sea.gmane.org>
	<5.1.1.6.0.20051221152616.0209a2f8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051221200154.020e9d98@mail.telecommunity.com>

At 01:40 AM 12/22/2005 +0100, Martin v. L?wis wrote:
>Phillip J. Eby wrote:
> > 1. Fredrik doesn't want to have to install a LaTeX toolchain in order to
> > get an HTML version of the documentation
> >
> > 2. Fredrik likes using whatever tools he has for editing HTML better than
> > whatever he has for editing LaTeX
> >
> > 3. Fredrik believes that more people would participate in updating Python
> > documentation if it didn't require a LaTeX toolchain or LaTeX-friendly 
> editor.
> >
> > (Of course, these are equally arguments for using other formats besides
> > HTML, especially formats that are closer to plain text.)
>
>Except, of course, for any other format (than HTML), you would have to
>substitute "Fredrik" by somebody promoting that other format.

To be clear: I don't advocate a switch; I'm okay with the current tools, 
since I have managed to get LaTeX to work on both Cygwin and Linux, which 
is enough for my needs.  I'm endeavoring only to point out that the 
arguments being advanced for HTML seem shaky to me.


> > By the way, I'm not sure I see what the problem with authoring Python
> > documentation with reST would be.
>
>Really not? How do we get from where we are to where you would like
>us to be?

Again, I'm not advocating a switch.  I'm only questioning the statements 
people have brought up about reST not being adequate.  I'm curious to know 
what features are lacking, and whether this is an accurate assessment or 
just a general impression.  If there are specific issues with reST, it 
would be good to know what they are.


>With this, I mean both technically (but perhaps I'm unaware
>of some tool that does the conversion automatically and lossless)
>and emotionally (but perhaps everybody but Fredrik and Barry could agree
>to switch to reST).

I don't advocate a switch, for precisely the reasons you are bringing up 
here.  Fredrik is the one advocating a switch.  If there *is* to be a 
switch, however, I would advocate that reST be the format in the absence of 
compelling reasons otherwise.

Since Barry and I think one other person mentioned issues with reST, I 
would like to know what they are.  I don't think it's appropriate to have a 
"reST isn't adequate" meme being propagated without some definition of 
*how* it is considered inadequate, such as what features are missing or 
what misfeatures are present.  This would be helpful for the docutils team, 
I'm sure, and in any case in the event there was a PEP to decide on a new 
format, it would need to specifically address any rationale for why reST 
should *not* be used.

And I'm personally just curious as well.  I've done some fairly substantive 
work in both the existing LaTeX-based tools:

http://svn.eby-sarna.com/*checkout*/PyProtocols/docs/ref/libprotocols.tex?rev=184&content-type=text%2Fplain

and using reST:

http://svn.python.org/projects/sandbox/trunk/setuptools/setuptools.txt

And I didn't encounter any deficiencies of reST, so I'm genuinely curious 
to know what it is I'm missing.  It's true that the simplest standalone 
reST tools don't support very sophisticated indexing, but I had the 
impression that the more advanced tools (and certainly the docutils 
libraries themselves) had considerable flexibility in this regard.

If someone has examples of actual "Pythondoc" markup that don't translate 
to reST, I'd be really interested in seeing them, just for my own 
education.  Of course, I'd also be curious how common such constructs are.


From skip at pobox.com  Thu Dec 22 02:24:35 2005
From: skip at pobox.com (skip@pobox.com)
Date: Wed, 21 Dec 2005 19:24:35 -0600
Subject: [Python-Dev] status of development documentation
In-Reply-To: <doci50$nom$1@sea.gmane.org>
References: <20051221095628.BE79.JCARLSON@uci.edu> <doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org>
Message-ID: <17322.83.459862.800945@montanaro.dyndns.org>


    Fredrik> If you google c.l.python for the word "documentation", you'll
    Fredrik> find recent megathreads with subjects like "bitching about the
    Fredrik> documentation", "opensource documentation problems" and "python
    Fredrik> documentation should be better" among the top hits.  But if you
    Fredrik> check the bug and patch trackers, you don't find many
    Fredrik> contributions.  Something's definitely broken.

People find it easier to complain than to contribute.  Maybe we should fix
that problem...

Skip

From fdrake at acm.org  Thu Dec 22 02:33:12 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 21 Dec 2005 20:33:12 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <1135203338.14488.29.camel@geddy.wooz.org>
References: <20051221095628.BE79.JCARLSON@uci.edu> <doci50$nom$1@sea.gmane.org>
	<1135203338.14488.29.camel@geddy.wooz.org>
Message-ID: <200512212033.13136.fdrake@acm.org>

On Wednesday 21 December 2005 17:15, Barry Warsaw wrote:
 > I'm not convinced it's the toolchain though.  People hate writing
 > documentation.  Getting people to contribute documentation is worse than
 > pulling teeth.

I don't think it's the toolchain either.  While most people don't have it, 
it's easier and easier to get a decent toolchain on Linux; TeX just isn't as 
hard to have around as it used to be.

I suspect that part of the problem is that there's no need to write 
documentation to scratch itches: once you know what to write, your itch has 
been scratched (you're already able to make the changes needed to your own 
code); nobody is relying on the updated documentation to be released to use 
what they figured out, even if they noted that the documentation was lacking 
to start with.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From pedronis at strakt.com  Thu Dec 22 02:24:46 2005
From: pedronis at strakt.com (Samuele Pedroni)
Date: Thu, 22 Dec 2005 02:24:46 +0100
Subject: [Python-Dev] Next PyPy Sprint: Palma de Mallorca (Spain) 23rd -
 29th January 2006
Message-ID: <43AA005E.80300@strakt.com>

Palma de Mallorca  PyPy Sprint: 23rd - 29th January 2006

============================================================



The next PyPy sprint is scheduled to take place January 2006 in

Palma De Mallorca, Balearic Isles, Spain. We'll give newcomer-friendly

introductions and the focus will mainly be on current JIT work, garbage

collection, alternative threading models, logic programming and on

improving the interface with external functions. To learn more about the

new Python-in-Python implementation look here:



     http://codespeak.net/pypy



Goals and topics of the sprint

------------------------------



In Gothenburg we have made some first forays into the interesting topics

of Just-in-Time compilation. In Mallorca we will continue that

and have the following ideas:



   - Further work/experimentation toward Just-In-Time Compiler generation,

     which was initiated with the Abstract Interpreter started in
Gothenburg.



   - Integrating our garbage collection toolkit with the backends and the

     code generation.



   - Heading into the direction of adding logic programming to PyPy.



   - Optimization work: our threading implementation is still incredibly

     slow, we need to work on that. Furthermore there are still quite

     some slow places in the interpreter that could be improved.



   - getting the socket module to a more complete state (it is

     already improved but still far from complete)



   - generally improving the way we interface with external functions.



   - whatever participants want to do with PyPy (please send

     suggestions to the mailing list before to allow us to plan

     and give feedback)





Location & Accomodation

------------------------



The sprint will be held at the Palma University (UIB - Universitat de

les Illes Balears), in their GNU/Linux lab

(http://mnm.uib.es/phpwiki/AulaLinux).  We are hosted by the Computer

Science department and Ricardo Galli is our contact person there,

helping with arranging facilities.



The University is located 7 km away from the central Palma. Busses to

the University departs from "Plaza de Espa?a" (which is a very central

location in Palma). Take bus 19 to the UIB campus.  A ticket for one

urban trip costs 1 euro. You can also buy a card that is valid for 10

trips and costs 7.51 euros.  Information about bus timetables and routes

can be found on:



     http://www.a-palma.es



A map over the UIB campus are can be found on:



     http://www.uib.es/imagenes/planoCampus.html



The actual address is: 3r pis de l'Anselm Turmeda which can be found on

the UIB Campus map.



At "Plaza de Espa?a" there is a hostel (Hostal Residencia Terminus)

which has been recommended to us. It's cheap (ca 50 euros/double room

with bathroom). Some more links to accomodations (flats, student homes

and hotels):



http://www.lodging-in-spain.com/hotel/town/Islas_Baleares,Mallorca,Palma_de_Mallorca,1/



http://www.uib.es/fuguib/residencia/english/index.html



http://www.homelidays.com/EN-Holidays-Rental/110_Search/SearchList.asp?DESTINATION=Palma%20de%20Mallorca&ADR_PAYS=ES&ADR_

LOCALISATION=ES%20ISLASBALEARES%20MALLORCA



If you want to find a given street, you can search here:



     http://www.callejeando.com/Pueblos/pueblo7_1.htm



To get to Palma De Mallorca almost all low fare airlines and travel

agencies have cheap tickets to get there. Information about Mallorca and

Palma (maps, tourist information, local transports, recommended air

lines, ferries and much more) can be found on:



     http://www.palmademallorca.es/portalPalma/home.jsp



Comments on the weather: In January it is cold and wet on Mallorca



Average temperature: 8,4 degrees Celsius

Lowest temperature: 2 degrees Celsius

Highest temperature: 14,5 degrees Celsius

Average humidity rate: 77,6 %



So more time for coding and less time for sunbathing and beaches ;-)



Exact times

-----------



The public PyPy sprint is held Monday 23rd - Sunday 29th January 2006.

Hours will be from 10:00 until people have had enough.  It's a good idea

to arrive a day before the sprint starts and leave a day later.  In the

middle of the sprint there usually is a break day and it's usually ok to

take half-days off if you feel like it.



For this particular break day, Thursday, we are invited to the studio of

Gin?s Qui?onero, a local artist and painter. Gin?s have also been the

person helping us getting connections to UIB and providing much

appreciated help regarding accommodation and other logistical

information.



For those of you interested - here is his website where there also are

paintings showing his studio:



     http://www.hermetex4.com/damnans/



For those interested in playing collectable card games, this will also

be an opportunity to get aquainted with V:TES which will be demoed by

Gin?s and Beatrice and Sten D?ring. For more information on this

cardgame - see: http://www.white-wolf.com/vtes/index.php.  (The Mallorca

sprint was organized through contacts within the V:TES community).



Network, Food, currency

------------------------



Currency is Euro.



Food is available in the UIB Campus area as well as cheap restaurants in

Palma.



You normally need a wireless network card to access the network, but we

can provide a wireless/ethernet bridge.



230V AC plugs are used in Mallorca.



Registration etc.pp.

--------------------



Please subscribe to the `PyPy sprint mailing list`_, introduce yourself

and post a note that you want to come.  Feel free to ask any questions

there!  There also is a separate `Mallorca people`_ page tracking who is

already thought to come.  If you have commit rights on codespeak then

you can modify yourself a checkout of




http://codespeak.net/svn/pypy/extradoc/sprintinfo/mallorca-2006/people.txt



.. _`PyPy sprint mailing list`:
http://codespeak.net/mailman/listinfo/pypy-sprint

.. _`Mallorca people`:
http://codespeak.net/pypy/extradoc/sprintinfo/mallorca-2006/people.html

From trentm at ActiveState.com  Thu Dec 22 05:33:24 2005
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 21 Dec 2005 20:33:24 -0800
Subject: [Python-Dev]  status of development documentation
Message-ID: <20051222043324.GA16420@ActiveState.com>

[Fredrik wrote]
> - could a cronjob that does this be set up on some python.org machine
> (or on some volunteer's machine)

I bit:

    http://trentm.com/python/

Cheers,
Trent

-- 
Trent Mick
TrentM at ActiveState.com

From jcarlson at uci.edu  Thu Dec 22 05:36:38 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 21 Dec 2005 20:36:38 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <docl8k$2gp$1@sea.gmane.org>
References: <1135203338.14488.29.camel@geddy.wooz.org>
	<docl8k$2gp$1@sea.gmane.org>
Message-ID: <20051221203346.BE93.JCARLSON@uci.edu>


"Fredrik Lundh" <fredrik at pythonware.com> wrote:
> 
> Barry Warsaw wrote:
> 
> > > Sure, and some people hate using whitespace for block structure.
> >
> > A more proper analogy would be people who hate braces and parentheses.
> > You have to type so many more < and > characters (not to mention &s
> > and ;s) to make happy-joy html than you have to type \s and {s and }s to
> > make nice-nice latex.
> 
> so what's *your* excuse for not using emacs? ;-)
> 
> (if you don't have sgml/html support in your editor, I recommend that you
> borrow my swedish keyboard, and see if you really prefer \ { } ` etc over
> < > & ...)

Speaking of optimizing documentation for an individual ;)


 - Josiah


From ianb at colorstudy.com  Thu Dec 22 06:17:30 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 21 Dec 2005 23:17:30 -0600
Subject: [Python-Dev] status of development documentation
In-Reply-To: <doci50$nom$1@sea.gmane.org>
References: <20051221095628.BE79.JCARLSON@uci.edu><doc6h2$dsi$1@sea.gmane.org>	<20051221105132.BE7C.JCARLSON@uci.edu><docare$tjs$1@sea.gmane.org>	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org>
Message-ID: <43AA36EA.2090302@colorstudy.com>

Fredrik Lundh wrote:
>>Maybe it's just because I came in late on this thread, but what exactly
>>is broken about the current LaTeX documentation?
> 
> 
> Checked the python-list archives lately?  If you google c.l.python for the
> word "documentation", you'll find recent megathreads with subjects like
> "bitching about the documentation", "opensource documentation problems"
> and "python documentation should be better" among the top hits.  But if
> you check the bug and patch trackers, you don't find many contributions.
> Something's definitely broken.

This is somewhat tangential to this discussion, but I did have the 
Python documentation in mind as a potential future target for 
Commentary: http://pythonpaste.org/comment/commentary/ -- which would 
allow more casual contributions that seem to work well for other projects.

-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org

From fredrik at pythonware.com  Thu Dec 22 09:16:57 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 22 Dec 2005 09:16:57 +0100
Subject: [Python-Dev] status of development documentation
References: <20051222043324.GA16420@ActiveState.com>
Message-ID: <dodnds$r32$1@sea.gmane.org>

Trent Mick wrote:

> > - could a cronjob that does this be set up on some python.org machine
> > (or on some volunteer's machine)
>
> I bit:
>
>     http://trentm.com/python/

you rule!

thanks /F




From fredrik at pythonware.com  Thu Dec 22 09:31:42 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 22 Dec 2005 09:31:42 +0100
Subject: [Python-Dev] status of development documentation
References: <do8nun$76d$1@sea.gmane.org>
	<43A8A271.1020506@v.loewis.de><dobupj$f9i$1@sea.gmane.org>
	<20051221193705.GA12847@rogue.amk.ca>
	<87d5jqw4uj.fsf@wilson.rwth-aachen.de>
Message-ID: <dodo9f$t7m$1@sea.gmane.org>

Torsten Bronger wrote:

> > [...]  Are there any HTML-to-print converters that are better?
>
> I don't understand exactly how the HTML is to be used for Python but
> I assume that not everything could be done via CSS, so own
> converters will be necessary for perfect output.

If done right, it should be possible to get a "usable" rendering from the raw
HTML+microformat file, but a real online version would of course need some
preprocessing (e.g basic templating and navigation fixup).  Not more than
you could do on the fly, or in a simple cgi script...

For publication work, you need more preprocessing, of course (but I'm not
sure the typical python user cares much about the subtle differences be-
tween latex and openoffice/word formatting...)

> I was disappointed with its rather small semantic vocabulary.

I sometimes doubt that the rest proponents understand the phrase "semantic
vocabulary".  They do sound a lot like Perl proponents, though...

(and strangely enough, there seems to be an almost perfect inverse relation-
ship between the ReST zealousness and the amount of text and code they
have contributed to the core distribution.  oh well.)

</F>




From fredrik at pythonware.com  Thu Dec 22 09:55:30 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 22 Dec 2005 09:55:30 +0100
Subject: [Python-Dev] status of development documentation
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<doci50$nom$1@sea.gmane.org><1135203338.14488.29.camel@geddy.wooz.org>
	<200512212033.13136.fdrake@acm.org>
Message-ID: <dodpm3$up$1@sea.gmane.org>

Fred L. Drake, Jr. wrote:

>  > I'm not convinced it's the toolchain though.  People hate writing
>  > documentation.  Getting people to contribute documentation is worse than
>  > pulling teeth.
>
> I don't think it's the toolchain either.  While most people don't have it,
> it's easier and easier to get a decent toolchain on Linux; TeX just isn't as
> hard to have around as it used to be.
>
> I suspect that part of the problem is that there's no need to write
> documentation to scratch itches: once you know what to write, your itch has
> been scratched (you're already able to make the changes needed to your own
> code);

If an ordinary user finds a minor issue, a type, or an error in the documentation,
the current user workflow is:

    1) (optionally) cut and paste the text to an editor, edit, and save to disk
    2) go to the sourceforge site, and locate the python project
    3) (optionally) sign up for a sourceforge account
    4) log in to your sourceforge account
    5) open a new bug or patch issue, and attach your suggestion
    6) wait 3-6 months for someone to pick up your patch, and for the
       next documentation release to appear on the site

If the documentation had been placed in a wiki:

    1) click edit, fix the text, and click save

If the documentation had been connected to a discussion board (PHP-style)

    1) click post new message, write a note, and click save

With a "user edit" mechanism (connected either to a mailing list, or roundup),
and documentation regularily updated from the trunk, the workflow is:

    1) click edit, update the text, preview, and click submit
    2) wait a few days for someone to pick up your patch, and a day for
       the documentation to be regenerated.

On the maintainer side, wikis and discussion boards require regular monitoring
to avoid abuse.  A user edit mechanism requires about the same work as today
(except that an edit mechanism with preview tends to result in patches that are
a lot more "ready for use", in my experience).

> nobody is relying on the updated documentation to be released to use what
> they figured out, even if they noted that the documentation was lacking to
> start with.

I know what you mean here, but read the wrong way, that sentence is so com-
pletely off the track so I don't know where to start.  People love to contribute
bits of information, especially when they get feedback (this is of course what
powers places like python-list, not to mention the entire blog universe).  Let's
use this human feature to our advantage.

</F>




From ncoghlan at gmail.com  Thu Dec 22 09:57:39 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 22 Dec 2005 18:57:39 +1000
Subject: [Python-Dev] Sets are mappings?
In-Reply-To: <20051221160432.GA9293@panix.com>
References: <20051221054538.pd9rhja1gldwgggs@login.werra.lunarpages.com>
	<20051221160432.GA9293@panix.com>
Message-ID: <43AA6A83.8020405@gmail.com>

Aahz wrote:
> On Wed, Dec 21, 2005, Michael Chermside wrote:
>> So I have a counter-proposal. Let's NOT create a hierarchy of abstract
>> base types for the elementary types of Python. (Even basestring feels
>> like a minor wart to me, although for now it seems like we need
>> it.) If the core problem is "how do you create a canonical ordering
>> for objects that survives serialization and deserialization into a
>> different VM?", then somehow abstract base types doesn't seem like
>> the most obvious solution. And if that's not the problem we're trying
>> to solve here, then what IS? Because I don't know of very many ACTUAL
>> (as opposed to theoretical) use cases for abstract base classes of
>> fundamental types.
> 
> You've got a good point, but the documentation issue still exists; that's
> what I was more interested in.  Clearly lists, tuples, and strings are
> sequences; clearly dicts are a mapping; the question is whether sets get
> tossed in with dicts.  Overall, I think it's pretty clear that the answer
> is "no", particularly given that sets don't support __getitem__().

Like Aahz, my interest is more pedagogic than practical. Python's slightly 
unusual in that the behaviour of sequences and multi-dimensional arrays (or 
any kind of mapping, really) is more a matter of convention than anything 
enforced by the language  - whether or not a container understands slices or a 
tuple of slices is the closest thing I've found to a reliable indicator as to 
whether or not something is a sequence or multiarray rather than a simple mapping.

So in looking for a defining characteristic for those two terms (sequence, in 
particular, is a term that gets thrown around a lot without being really well 
defined), those are the main features I'd pick.

In practice, as MC said in his other email, "just try it and see what happens" 
is generally a far better approach. To answer MC's other point in that email, 
I actually agree it's perfectly possible to have a mapping which is not a 
container, so the structure of the taxonomy should be eliminated entirely. 
Whether or not something is a container and whether or not it is a mapping are 
independent questions.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From steve at holdenweb.com  Thu Dec 22 10:27:06 2005
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 22 Dec 2005 09:27:06 +0000
Subject: [Python-Dev] status of development documentation
In-Reply-To: <17322.83.459862.800945@montanaro.dyndns.org>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<doc6h2$dsi$1@sea.gmane.org>	<20051221105132.BE7C.JCARLSON@uci.edu>
	<docare$tjs$1@sea.gmane.org>	<1135196162.14495.15.camel@geddy.wooz.org>	<doci50$nom$1@sea.gmane.org>
	<17322.83.459862.800945@montanaro.dyndns.org>
Message-ID: <dodrhe$4g0$1@sea.gmane.org>

skip at pobox.com wrote:
>     Fredrik> If you google c.l.python for the word "documentation", you'll
>     Fredrik> find recent megathreads with subjects like "bitching about the
>     Fredrik> documentation", "opensource documentation problems" and "python
>     Fredrik> documentation should be better" among the top hits.  But if you
>     Fredrik> check the bug and patch trackers, you don't find many
>     Fredrik> contributions.  Something's definitely broken.
> 
> People find it easier to complain than to contribute.  Maybe we should fix
> that problem...
> 
I very much agree that we should, and *not* by making it more difficult 
to complain ;-)

Could the PSF help here by offering annual prizes for the best 
contributions to the documentation, or wouldn't that be an adequate 
motivator?

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From walter at livinglogic.de  Thu Dec 22 10:27:00 2005
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu, 22 Dec 2005 10:27:00 +0100
Subject: [Python-Dev] status of development documentation
In-Reply-To: <5.1.1.6.0.20051221200154.020e9d98@mail.telecommunity.com>
References: <5.1.1.6.0.20051221152616.0209a2f8@mail.telecommunity.com>	<docare$tjs$1@sea.gmane.org>
	<20051221095628.BE79.JCARLSON@uci.edu>	<doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu>	<docare$tjs$1@sea.gmane.org>	<5.1.1.6.0.20051221152616.0209a2f8@mail.telecommunity.com>
	<5.1.1.6.0.20051221200154.020e9d98@mail.telecommunity.com>
Message-ID: <43AA7164.8040308@livinglogic.de>

Phillip J. Eby wrote:

> [...] 
> 
> If someone has examples of actual "Pythondoc" markup that don't translate 
> to reST, I'd be really interested in seeing them, just for my own 
> education.  Of course, I'd also be curious how common such constructs are.

I'm using XML markup for our packages. Examples can be found at 
http://www.livinglogic.de/Python/xist/xsc/index_module.py (for 
docstrings) or at 
http://www.livinglogic.de/viewcvs/index.cgi/LivingLogic/Python/xist/HOWTO.xml?rev=2.110&content-type=text/vnd.viewcvs-markup 
for doc files. Possible output is:

* HTML: http://www.livinglogic.de/Python/xist/Howto.html

* Plain text (by piping a special HTML output through w3m): 
http://www.livinglogic.de/Python/xist/Howto.txt. It might probably be 
possible to extend this, so that the output is reST.

* XSL-FO: http://www.livinglogic.de/Python/xist/Howto.fo

* PDF (generated with FOP): http://www.livinglogic.de/Python/xist/Howto.pdf

The source is definitely wordier than reST, but adding new markup is 
trivial. Take a look at 
http://www.livinglogic.de/Python/xist/Download.html and at the source at 
http://www.livinglogic.de/Python/xist/Download.htmlxsc. The download 
element automatically determines the size of the package. Source can be 
found here 
http://www.livinglogic.de/viewcvs/index.cgi/LivingLogic/WWW-Python/site/Python_xmlns.py?rev=1.43&content-type=text/vnd.viewcvs-markup
(search for "class download"). Would something like this be possible 
with reST?

Bye,
    Walter D?rwald


From skip at pobox.com  Thu Dec 22 10:51:07 2005
From: skip at pobox.com (skip@pobox.com)
Date: Thu, 22 Dec 2005 03:51:07 -0600
Subject: [Python-Dev] status of development documentation
In-Reply-To: <dodnds$r32$1@sea.gmane.org>
References: <20051222043324.GA16420@ActiveState.com>
	<dodnds$r32$1@sea.gmane.org>
Message-ID: <17322.30475.650292.348665@montanaro.dyndns.org>

    >> http://trentm.com/python/

    Fredrik> you rule!

Actually, I think Trent rocks.  Guido rules. <wink>

Skip

From mwh at python.net  Thu Dec 22 11:14:53 2005
From: mwh at python.net (Michael Hudson)
Date: Thu, 22 Dec 2005 10:14:53 +0000
Subject: [Python-Dev] Sets are mappings?
In-Reply-To: <20051221054538.pd9rhja1gldwgggs@login.werra.lunarpages.com>
	(Michael Chermside's message of "Wed, 21 Dec 2005 05:45:38 -0800")
References: <20051221054538.pd9rhja1gldwgggs@login.werra.lunarpages.com>
Message-ID: <2m8xud30bm.fsf@starship.python.net>

Michael Chermside <mcherm at mcherm.com> writes:

> So I have a counter-proposal. Let's NOT create a hierarchy of abstract
> base types for the elementary types of Python.

+1

Cheers,
mwh

-- 
  <bruce> how are the jails in israel?
  <itamar> well, the one I was in was pretty nice
                                                -- from Twisted.Quotes

From mwh at python.net  Thu Dec 22 12:51:40 2005
From: mwh at python.net (Michael Hudson)
Date: Thu, 22 Dec 2005 11:51:40 +0000
Subject: [Python-Dev] status of development documentation
In-Reply-To: <doci50$nom$1@sea.gmane.org> (Fredrik Lundh's message of "Wed,
	21 Dec 2005 22:40:45 +0100")
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<doc6h2$dsi$1@sea.gmane.org> <20051221105132.BE7C.JCARLSON@uci.edu>
	<docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org>
Message-ID: <2m4q512vub.fsf@starship.python.net>

"Fredrik Lundh" <fredrik at pythonware.com> writes:

> Checked the python-list archives lately?  If you google c.l.python for the
> word "documentation", you'll find recent megathreads with subjects like
> "bitching about the documentation", "opensource documentation problems"
> and "python documentation should be better" among the top hits.  But if
> you check the bug and patch trackers, you don't find many contributions.
> Something's definitely broken.

Hmm, it's this discussion again!  Let me make my point again!

Writing good documentation is hard.

And sometimes the problem is that the document isn't really structured
right, or it has been hastily updated to cover too many changes that
it's a dogs breakfast, or some other 'global' problem and these
*really* take time to fix.  I really, really don't think the
formatting tools make that much difference in the grand scheme of
things.  I think streamlining the process of getting a patch in would
help a lot more (and not just for the documentation, obviously) but
still not *that* much.

Cheers,
mwh
(another one in the 'hates editing HTML' camp, if anyone's counting)

-- 
  The ability to quote is a serviceable substitute for wit.
                                                -- W. Somerset Maugham

From fredrik at pythonware.com  Thu Dec 22 12:51:44 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 22 Dec 2005 12:51:44 +0100
Subject: [Python-Dev] status of development documentation
References: <20051221095628.BE79.JCARLSON@uci.edu><doc6h2$dsi$1@sea.gmane.org>	<20051221105132.BE7C.JCARLSON@uci.edu><docare$tjs$1@sea.gmane.org>	<1135196162.14495.15.camel@geddy.wooz.org><doci50$nom$1@sea.gmane.org>
	<43AA36EA.2090302@colorstudy.com>
Message-ID: <doe40g$u8e$1@sea.gmane.org>

Ian Bicking wrote:

> This is somewhat tangential to this discussion, but I did have the
> Python documentation in mind as a potential future target for
> Commentary: http://pythonpaste.org/comment/commentary/ -- which would
> allow more casual contributions that seem to work well for other projects.

indeed.

Commentary worked better this time than last time I tinkered with it.  a few
notes:

- it would be nice to be able to cancel a new note by double-clicking again
in the same spot (at least as long as the note is empty)

- IE support seems to be a little shaky; klicking and entering text works fine,
but when I click "save", nothing happens.  in IE, that is.  if I look at the site in
Firefox, the note is there.

- if you click "edit this comment" on an existing note, and then click cancel,
the note disappears (in Firefox).  double-clicking on the associated block no
longer works, after that, so the note is still in there somewhere...

- many notes added to the same place may squeeze the original text into a
very narrow column.  note sure how to address that, but some kind of "mini-
mize" or "hide" feature could be nice.

</F> 




From amk at amk.ca  Thu Dec 22 15:22:06 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 22 Dec 2005 09:22:06 -0500
Subject: [Python-Dev] documentation comments
In-Reply-To: <dodrhe$4g0$1@sea.gmane.org>
References: <20051221095628.BE79.JCARLSON@uci.edu> <doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org>
	<17322.83.459862.800945@montanaro.dyndns.org>
	<dodrhe$4g0$1@sea.gmane.org>
Message-ID: <20051222142206.GB15340@rogue.amk.ca>

On Thu, Dec 22, 2005 at 09:27:06AM +0000, Steve Holden wrote:
> Could the PSF help here by offering annual prizes for the best 
> contributions to the documentation, or wouldn't that be an adequate 
> motivator?

I think the most effective thing would be to award a grant to someone
to build a real comment-on-the-docs system.  There were a few Summer
of Code proposals for this sort of thing; one was funded but the
developer decided to do a KDE project instead.  

I had lunch with Fred the other day, and he was worried about whether
anyone would garden the comments to remove spam.  That is indeed an
issue, but I think we can cope with that problem once a system is
built.

Another worry is versioning.  Once lots of people have made comments
on Python 2.4.0's documentation, what do you do when 2.4.1 is
released?  Do you move the comments to the new docs, or leave them
attached to 2.4.0 and start 2.4.1 with a clean slate?  (Perhaps the
system could work a little like a bug tracking system; comments could
be marked as 'applied', and applied comments don't get moved from
2.4.0 to 2.4.1 because their content is now in the docs.)

--amk

From mcherm at mcherm.com  Thu Dec 22 14:50:29 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu, 22 Dec 2005 05:50:29 -0800
Subject: [Python-Dev] status of development documentation
Message-ID: <20051222055029.vo5utol152os40gg@login.werra.lunarpages.com>

Steve Holden writes:
> Could the PSF help here by offering annual prizes for the best
> contributions to the documentation, or wouldn't that be an adequate
> motivator?

Money is not a very effective motivator for this sort of work. (Well,
in sufficient quantities it is, but the quantities required are
quite large.) Offering *credit* is more effective -- a mention within
a contributors list perhaps. Even more effective is offering the
chance to make a difference: immediate feedback (seeing your edit in
place). Thus, I'm a big fan of amk's suggestion:

> I think the most effective thing would be [...]
> to build a real comment-on-the-docs system.

But I agree strongly with Fred's concerns:
> he was worried about whether
> anyone would garden the comments to remove spam.

and as Michael Hudson put it:
> Writing good documentation is hard.
>
> And sometimes the problem is that the document isn't really structured
> right, or it has been hastily updated to cover too many changes that
> it's a dogs breakfast, or some other 'global' problem and these
> *really* take time to fix.

My own favorite idea is to create a comment-on-the-docs mechanism
allowing both COMMENTS, and PROPOSED EDITS. The proposed edits would
need to be reviewed by one of a small number of skilled and dedicated
editors (Fred Drake... you're a hero!) before being officially
incorporated. That's not all that different from the current system
(submit a patch to sourceforge), except that the format for entering
the change would be simpler.

Of course, the person who REALLY gets to decide how it works isn't me;
it's whoever decides to spend the time to BUILD this system.

-- Michael Chermside


From fdrake at acm.org  Thu Dec 22 15:16:30 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 22 Dec 2005 09:16:30 -0500
Subject: [Python-Dev] documentation comments
In-Reply-To: <20051222142206.GB15340@rogue.amk.ca>
References: <20051221095628.BE79.JCARLSON@uci.edu> <dodrhe$4g0$1@sea.gmane.org>
	<20051222142206.GB15340@rogue.amk.ca>
Message-ID: <200512220916.30516.fdrake@acm.org>

On Thursday 22 December 2005 09:22, A.M. Kuchling wrote:
 > I had lunch with Fred the other day, and he was worried about whether
 > anyone would garden the comments to remove spam.  That is indeed an
 > issue, but I think we can cope with that problem once a system is
 > built.
 >
 > Another worry is versioning.  Once lots of people have made comments
 > on Python 2.4.0's documentation, what do you do when 2.4.1 is
 > released?  Do you move the comments to the new docs, or leave them
 > attached to 2.4.0 and start 2.4.1 with a clean slate?  

This was actually a big part of my gardening concern: comments from the 
release X.Y.Z docs need to be handled before releasing X.Y.Z+1 or X.Y+1.*, or 
they aren't being used to improve the documentation at all.

 > (Perhaps the 
 > system could work a little like a bug tracking system; comments could
 > be marked as 'applied', and applied comments don't get moved from
 > 2.4.0 to 2.4.1 because their content is now in the docs.)

I'd be more inclined to see that comments are handled (even if handling them 
is a matter of determining that they aren't actually interesting), and just 
toss comments for a new release.  A patch release would be an occaission to 
turn off commenting on the previous releases for the same X.Y version (though 
comments would still exist in the older version).


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From fdrake at acm.org  Thu Dec 22 15:26:55 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 22 Dec 2005 09:26:55 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <20051222055029.vo5utol152os40gg@login.werra.lunarpages.com>
References: <20051222055029.vo5utol152os40gg@login.werra.lunarpages.com>
Message-ID: <200512220926.55425.fdrake@acm.org>

On Thursday 22 December 2005 08:50, Michael Chermside wrote:
 > Money is not a very effective motivator for this sort of work. (Well,
 > in sufficient quantities it is, but the quantities required are
 > quite large.) Offering *credit* is more effective -- a mention within
 > a contributors list perhaps. 

There is a credits list for the documentation, and it's included in the HTML 
version of the formatted result as well.  Extra credit if you know where it 
is without looking, though.

 > My own favorite idea is to create a comment-on-the-docs mechanism
 > allowing both COMMENTS, and PROPOSED EDITS. The proposed edits would
 > need to be reviewed by one of a small number of skilled and dedicated

I'm unclear on what you buy with having these two labels; are comments things 
that (presumably) get ignored by the documentation editor, or are the 
proposed edits simply more specific?  If the later, I'm not sure having the 
labels helps.

(I'm also concerned that the whole thing could end up being misused as a help 
desk, littering the docs with questions about application problems.)

 > Of course, the person who REALLY gets to decide how it works isn't me;
 > it's whoever decides to spend the time to BUILD this system.

The builder certainly determines what they build, but in the longer term, 
whoever is using it to incorporate changes into the documentation will likely 
have something to say about it, since that's who determines if it actually 
gets used to improve the documentation.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From facundobatista at gmail.com  Thu Dec 22 15:40:19 2005
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 22 Dec 2005 11:40:19 -0300
Subject: [Python-Dev] status of development documentation
In-Reply-To: <5.1.1.6.0.20051221152616.0209a2f8@mail.telecommunity.com>
References: <20051221095628.BE79.JCARLSON@uci.edu> <doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<5.1.1.6.0.20051221152616.0209a2f8@mail.telecommunity.com>
Message-ID: <e04bdf310512220640k775cb19fk@mail.gmail.com>

2005/12/21, Phillip J. Eby <pje at telecommunity.com>:

> 3. Fredrik believes that more people would participate in updating Python
> documentation if it didn't require a LaTeX toolchain or LaTeX-friendly editor.

I'm sure he's right. I'm not talking about any random user that finds
a doc bug and wants to generate a patch, here I'm talking of my own
experience:

I had to correct a few lines in the almost perfect documentation that
Raymond generated for Decimal. I fighted with my Linux (at that time,
FC1) to be able to compile the docs, and couldn't do it.

I ended touching the XML by hand. It worked, but

  a) Took some time.
  b) Wasn't really sure that it was well corrected.

So, I really think that a more human friendly format will help here.

What I do NOT know, if the effort of converting the whole docs to
another format is worth it, and that effort should be deviated to
something that will help more other users to help with docs (for
example, that the official docs could be annotatted, a la MySQL (AMK
did something like this, right?)).

Regards,

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From pje at telecommunity.com  Thu Dec 22 16:18:53 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Dec 2005 10:18:53 -0500
Subject: [Python-Dev] status of development documentation
Message-ID: <5.1.1.6.0.20051222101850.01b12df8@mail.telecommunity.com>

At 10:27 AM 12/22/2005 +0100, Walter D?rwald wrote:
>Phillip J. Eby wrote:
>
> > [...]
> >
> > If someone has examples of actual "Pythondoc" markup that don't translate
> > to reST, I'd be really interested in seeing them, just for my own
> > education.  Of course, I'd also be curious how common such constructs are.
>
>I'm using XML markup for our packages. Examples can be found at
>[snip]

By "Pythondoc", I mean the LaTeX-based markup system being used for the 
official Python documentation, not arbitrary methods of documentation for 
Python code.


>The source is definitely wordier than reST, but adding new markup is
>trivial. Take a look at
>http://www.livinglogic.de/Python/xist/Download.html and at the source at
>http://www.livinglogic.de/Python/xist/Download.htmlxsc. The download
>element automatically determines the size of the package. Source can be
>found here
>http://www.livinglogic.de/viewcvs/index.cgi/LivingLogic/WWW-Python/site/Python_xmlns.py?rev=1.43&content-type=text/vnd.viewcvs-markup
>(search for "class download"). Would something like this be possible
>with reST?

The docutils toolchain converts reST input into a DOM, and allows arbitrary 
transformation phases to be added to processing before conversion to 
output.  This includes processing of "directives", e.g. commands like:

.. include:: filename

And of interpreted text "roles",  e.g. `Foobar`:class:.

It is not, however, a general XML transformation toolkit, if that's what 
you're asking.  However, if you wanted to be able to use XML input as part 
of a docutils DOM, you could certainly do that.  For that matter, you could 
take a reST document and simply transform it to XML for use with the rest 
of your toolset.

But this isn't particularly relevant to the discussion about *Python's* 
documentation, and I'm not even advocating that Python switch, let alone 
arbitrary other projects.


From collinw at gmail.com  Thu Dec 22 16:11:02 2005
From: collinw at gmail.com (Collin Winter)
Date: Thu, 22 Dec 2005 16:11:02 +0100
Subject: [Python-Dev] Patch to make unittest.TestCase easier to subclass
Message-ID: <43aa6ff70512220711v6a344c73o1299b89308c2d11f@mail.gmail.com>

Hello all!

I just submitted Patch #1388073, designed to make unittest's TestCase
class easier to subclass, and I'd appreciate a review of/feedback on
the code there.

While recently working on a subclass of unittest.TestCase to support
TODO-tests, I found a large number of __-prefixed attributes in
TestCase. The presence of these attributes meant that I had to copy
several methods over to my new subclass in order for things to work.

The patch I've provided converts these __-prefixed attributes to
_-prefixed attributes, making it much simpler to subclass TestCase.
The patch is against unittest.py from SVN revision 41775.

Included with the patch are "before" and "after" versions of my
subclass showing the impact of the change to unittest.TestCase.

Thanks,
Collin Winter

From jim at zope.com  Thu Dec 22 16:34:46 2005
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Dec 2005 10:34:46 -0500
Subject: [Python-Dev] timeout options in high-level networking modules
Message-ID: <43AAC796.6070506@zope.com>


Yesterday, I needed to make a web request in a program (actually a test)
that could block indefinately, so I needed to set a socket timeout.
Unfortunately, AFAICT none of urllib, urllib2, httplib provide options to set
the timeout on the sockets they use.  I ended up having to roll my own
code to make the request.

It would be nice if high-level network modules, like the ones mentioned
above, had options to provide a timeout.  (For example, urlopen could
grow an optional timout argument.)

Thoughts?

If we think this is a good idea, then someone who has time could start chipping
away at it.  I'm happy to work on this *if* I can find time.  This would make
a nice easy sprint project at PyCon too.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From jeremy at alum.mit.edu  Thu Dec 22 16:48:01 2005
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 22 Dec 2005 10:48:01 -0500
Subject: [Python-Dev] timeout options in high-level networking modules
In-Reply-To: <43AAC796.6070506@zope.com>
References: <43AAC796.6070506@zope.com>
Message-ID: <e8bf7a530512220748t4124b65aqe42c5159a34748d3@mail.gmail.com>

Yup.  I just went through a similar exercise with urllib2.  It wasn't
too hard to plumb through a different HTTPHandler that set the
timeout, but it would be much nicer as a default option.  It seems
like a 30 minute project; might fit in an "odds and ends" sprint.

Jeremy

On 12/22/05, Jim Fulton <jim at zope.com> wrote:
>
> Yesterday, I needed to make a web request in a program (actually a test)
> that could block indefinately, so I needed to set a socket timeout.
> Unfortunately, AFAICT none of urllib, urllib2, httplib provide options to set
> the timeout on the sockets they use.  I ended up having to roll my own
> code to make the request.
>
> It would be nice if high-level network modules, like the ones mentioned
> above, had options to provide a timeout.  (For example, urlopen could
> grow an optional timout argument.)
>
> Thoughts?
>
> If we think this is a good idea, then someone who has time could start chipping
> away at it.  I'm happy to work on this *if* I can find time.  This would make
> a nice easy sprint project at PyCon too.
>
> Jim
>
> --
> Jim Fulton           mailto:jim at zope.com       Python Powered!
> CTO                  (540) 361-1714            http://www.python.org
> Zope Corporation     http://www.zope.com       http://www.zope.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>

From steve at holdenweb.com  Thu Dec 22 16:52:47 2005
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 22 Dec 2005 15:52:47 +0000
Subject: [Python-Dev] timeout options in high-level networking modules
In-Reply-To: <43AAC796.6070506@zope.com>
References: <43AAC796.6070506@zope.com>
Message-ID: <doei4f$cvl$1@sea.gmane.org>

Jim Fulton wrote:
> Yesterday, I needed to make a web request in a program (actually a test)
> that could block indefinately, so I needed to set a socket timeout.
> Unfortunately, AFAICT none of urllib, urllib2, httplib provide options to set
> the timeout on the sockets they use.  I ended up having to roll my own
> code to make the request.
> 
> It would be nice if high-level network modules, like the ones mentioned
> above, had options to provide a timeout.  (For example, urlopen could
> grow an optional timout argument.)
> 
> Thoughts?
> 
> If we think this is a good idea, then someone who has time could start chipping
> away at it.  I'm happy to work on this *if* I can find time.  This would make
> a nice easy sprint project at PyCon too.
> 
That's a very good idea. At present the only option one has is to set a 
global socket.defaulttimout() or somehow monkey-patch the modules you 
want to use, and neither of those options are entirely satisfactory.

Basically any method that can create a new TCP connection should acquire 
an optional timeout=None parameter, right?

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From lac at strakt.com  Wed Dec 21 21:26:19 2005
From: lac at strakt.com (Laura Creighton)
Date: Wed, 21 Dec 2005 21:26:19 +0100
Subject: [Python-Dev] [Doc-SIG]  status of development documentation
In-Reply-To: Message from Ian Bicking <ianb@colorstudy.com> 
	of "Wed, 21 Dec 2005 13:43:21 CST." <43A9B059.9060900@colorstudy.com> 
References: <dobupj$f9i$1@sea.gmane.org> <doc3n2$2ll$1@sea.gmane.org>
	<20051221095628.BE79.JCARLSON@uci.edu>
	<200512211335.09433.fdrake@acm.org> <doc8lu$lla$1@sea.gmane.org>
	<17321.44040.97252.894883@montanaro.dyndns.org>
	<43A9B059.9060900@colorstudy.com> 
Message-ID: <200512212026.jBLKQJWZ027961@theraft.strakt.com>


Whenever people have demanded that I write documentation in html
I have always done this:

<pre>
all my documentation, as output from a text editor.

All subsequent formatting to be done by somebody else who doesn't
find dealing with html as excruciatingly painful as I do.
</pre>

I suspect there are lots of people who have concluded that this
is all the html that you really need.  The question is, are you
willing to put up with documentation like this from people?

Laura

From python at discworld.dyndns.org  Thu Dec 22 17:35:41 2005
From: python at discworld.dyndns.org (Charles Cazabon)
Date: Thu, 22 Dec 2005 10:35:41 -0600
Subject: [Python-Dev] timeout options in high-level networking modules
In-Reply-To: <doei4f$cvl$1@sea.gmane.org>
References: <43AAC796.6070506@zope.com> <doei4f$cvl$1@sea.gmane.org>
Message-ID: <20051222163541.GC3543@discworld.dyndns.org>

Steve Holden <steve at holdenweb.com> wrote:
> Jim Fulton wrote:
> > Yesterday, I needed to make a web request in a program (actually a test)
> > that could block indefinately, so I needed to set a socket timeout.
> > Unfortunately, AFAICT none of urllib, urllib2, httplib provide options to set
> > the timeout on the sockets they use.  I ended up having to roll my own
> > code to make the request.
[...]
> That's a very good idea. At present the only option one has is to set a 
> global socket.defaulttimout() or somehow monkey-patch the modules you 
> want to use, and neither of those options are entirely satisfactory.
> 
> Basically any method that can create a new TCP connection should acquire 
> an optional timeout=None parameter, right?

Yes.  

It might also be nice if the modules that rely on blocking mode being set on
sockets (basically anything using socket.ssl()) actually explicitly set that
first.  Right now, if you do socket.setdefaulttimeout() to a non-None value
and then try to use anything that does SSL (poplib, imaplib), the connections
will quickly die.

Charles
-- 
-----------------------------------------------------------------------
Charles Cazabon                           <python at discworld.dyndns.org>
GPL'ed software available at:               http://pyropus.ca/software/
-----------------------------------------------------------------------

From fdrake at acm.org  Thu Dec 22 17:44:42 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 22 Dec 2005 11:44:42 -0500
Subject: [Python-Dev] LaTeX and Python doc contributions
Message-ID: <200512221144.42307.fdrake@acm.org>

Just a quick note based on some of the discussion on the Doc-SIG list:

Some people are getting asked to convert their documentation contributions to 
LaTeX themselves, and that *is* a barrier to contribution.  I've generally 
stated that I'm willing to perform conversion, making plain text / ReST 
completely acceptable for documentation contributions.  Others have commonly 
converted plain text to LaTeX as well.

I'd like to make sure that Python committers know that this is reasonable; if 
the only thing holding a contribution back is LaTeXification of 
documentation, feel free to assign it to me for conversion.  I do not want 
LaTeX itself to cause us to lose documentation contributions; the hard part 
for documentation really is getting good content.  Hard workers shouldn't be 
turned away.  :-)


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From fdrake at acm.org  Thu Dec 22 17:58:43 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 22 Dec 2005 11:58:43 -0500
Subject: [Python-Dev] LaTeX and Python doc contributions
In-Reply-To: <200512221144.42307.fdrake@acm.org>
References: <200512221144.42307.fdrake@acm.org>
Message-ID: <200512221158.43814.fdrake@acm.org>

On Thursday 22 December 2005 11:44, Fred L. Drake, Jr. wrote:
 > I've
 > generally stated that I'm willing to perform conversion, making plain text
 > / ReST completely acceptable for documentation contributions.  Others have
 > commonly converted plain text to LaTeX as well.

I've started a list of volunteer TeXnicians for the Python documentation:

    http://www.python.org/dev/doc/

If you'd like to be on the list, please add yourself if you have commit 
privileges to the website repository, or ask webmaster at python.org to add 
you.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From mcherm at mcherm.com  Thu Dec 22 18:33:39 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu, 22 Dec 2005 09:33:39 -0800
Subject: [Python-Dev] status of development documentation
Message-ID: <20051222093339.kk30p0m0cieosg8g@login.werra.lunarpages.com>

I wrote:
> My own favorite idea is to create a comment-on-the-docs mechanism
> allowing both COMMENTS, and PROPOSED EDITS.

Fred Drake replies:
> I'm unclear on what you buy with having these two labels; are comments things
> that (presumably) get ignored by the documentation editor, or are the
> proposed edits simply more specific?

Things that get ignored by the doc editors.

> (I'm also concerned that the whole thing could end up being misused as a help
> desk, littering the docs with questions about application problems.)

Me too. Specifically, I think if you make it really easy to write notes
on the docs you will get some helpful documentation content. You will
also get lots of things that are too lengthy or exhaustive, to specific
to one person's problem, helpdesk style questions, and probably spam. All
I meant was to allow the contributor to specify which category they think
this particular note belongs to so the doc editors can read only the ones
that people thought ought to be included in the docs.

-- Michael Chermside


From facundobatista at gmail.com  Thu Dec 22 18:37:05 2005
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 22 Dec 2005 14:37:05 -0300
Subject: [Python-Dev] status of development documentation
In-Reply-To: <20051222093339.kk30p0m0cieosg8g@login.werra.lunarpages.com>
References: <20051222093339.kk30p0m0cieosg8g@login.werra.lunarpages.com>
Message-ID: <e04bdf310512220937x65eb537am@mail.gmail.com>

2005/12/22, Michael Chermside <mcherm at mcherm.com>:

> > (I'm also concerned that the whole thing could end up being misused as a help
> > desk, littering the docs with questions about application problems.)
>
> Me too. Specifically, I think if you make it really easy to write notes
> on the docs you will get some helpful documentation content. You will
> also get lots of things that are too lengthy or exhaustive, to specific
> to one person's problem, helpdesk style questions, and probably spam. All

I sent a mail to MySQL folks, asking them some feedback about the
dynamics of their documentation annotation system (regarding this
issues, spam, etc.). Let's see if they answer.

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From skip at pobox.com  Thu Dec 22 19:23:03 2005
From: skip at pobox.com (skip@pobox.com)
Date: Thu, 22 Dec 2005 12:23:03 -0600
Subject: [Python-Dev] LaTeX and Python doc contributions
In-Reply-To: <200512221144.42307.fdrake@acm.org>
References: <200512221144.42307.fdrake@acm.org>
Message-ID: <17322.61191.851677.47037@montanaro.dyndns.org>


    Fred> Some people are getting asked to convert their documentation
    Fred> contributions to LaTeX themselves...

Who is asking this of potential contributors?  I know you, Aahz and I have
repeatedly told people on c.l.py that LaTeX knowledge is not necessary.
Plain text is okay.  What do we need to do to squash this meme?

Tony & other python-dev summarizers (and maybe Cameron Laird for the c.l.py
summaries): please make a note of this in your next summary.  The
I-can't-contribute-because-I-don't-know-LaTeX notion has to die, die, die.

Skip

From fredrik at pythonware.com  Thu Dec 22 19:20:09 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 22 Dec 2005 19:20:09 +0100
Subject: [Python-Dev] status of development documentation
References: <20051222093339.kk30p0m0cieosg8g@login.werra.lunarpages.com>
Message-ID: <doeqoq$c16$1@sea.gmane.org>

Michael Chermside wrote:¨

> Me too. Specifically, I think if you make it really easy to write notes
> on the docs you will get some helpful documentation content. You will
> also get lots of things that are too lengthy or exhaustive, to specific
> to one person's problem, helpdesk style questions, and probably spam.

fwiw, the effbot.org useredit mechanism results in nice patches, suggestions,
occasional questions, and, in periods, huge amounts of spam (from spammers
who treat it as an ordinary wiki).

</F>




From facundobatista at gmail.com  Thu Dec 22 19:39:01 2005
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 22 Dec 2005 15:39:01 -0300
Subject: [Python-Dev] LaTeX and Python doc contributions
In-Reply-To: <17322.61191.851677.47037@montanaro.dyndns.org>
References: <200512221144.42307.fdrake@acm.org>
	<17322.61191.851677.47037@montanaro.dyndns.org>
Message-ID: <e04bdf310512221039x5fe9a9fay@mail.gmail.com>

2005/12/22, skip at pobox.com <skip at pobox.com>:

> Tony & other python-dev summarizers (and maybe Cameron Laird for the c.l.py
> summaries): please make a note of this in your next summary.  The
> I-can't-contribute-because-I-don't-know-LaTeX notion has to die, die, die.

Very interesting. What I don't know here is how to submit patches...

I mean, if they were in LaTeX, a diff file would be enough. But in
plain text (or ReST), how should people specify the corrections, the
position of new paragraphs, etc?

I'm really interested in this, we've been discussing about docs in
Python Argentina and some people were willing to help (and scared
about LaTeX).

Thank you!

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From nnorwitz at gmail.com  Thu Dec 22 19:44:33 2005
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 22 Dec 2005 10:44:33 -0800
Subject: [Python-Dev] documentation comments
In-Reply-To: <20051222142206.GB15340@rogue.amk.ca>
References: <20051221095628.BE79.JCARLSON@uci.edu> <doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org>
	<17322.83.459862.800945@montanaro.dyndns.org>
	<dodrhe$4g0$1@sea.gmane.org> <20051222142206.GB15340@rogue.amk.ca>
Message-ID: <ee2a432c0512221044o71f01392yeb4c9b5adbac3318@mail.gmail.com>

On 12/22/05, A.M. Kuchling <amk at amk.ca> wrote:
>
> I had lunch with Fred the other day, and he was worried about whether
> anyone would garden the comments to remove spam.

I would help assuming this is easy--meaning a single click to remove a comment.

n

From jim at zope.com  Thu Dec 22 19:56:49 2005
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Dec 2005 13:56:49 -0500
Subject: [Python-Dev] timeout options in high-level networking modules
In-Reply-To: <doei4f$cvl$1@sea.gmane.org>
References: <43AAC796.6070506@zope.com> <doei4f$cvl$1@sea.gmane.org>
Message-ID: <43AAF6F1.7080206@zope.com>

Steve Holden wrote:
> Jim Fulton wrote:
> 
>>Yesterday, I needed to make a web request in a program (actually a test)
>>that could block indefinately, so I needed to set a socket timeout.
>>Unfortunately, AFAICT none of urllib, urllib2, httplib provide options to set
>>the timeout on the sockets they use.  I ended up having to roll my own
>>code to make the request.
>>
>>It would be nice if high-level network modules, like the ones mentioned
>>above, had options to provide a timeout.  (For example, urlopen could
>>grow an optional timout argument.)
>>
>>Thoughts?
>>
>>If we think this is a good idea, then someone who has time could start chipping
>>away at it.  I'm happy to work on this *if* I can find time.  This would make
>>a nice easy sprint project at PyCon too.
>>
> 
> That's a very good idea. At present the only option one has is to set a 
> global socket.defaulttimout() or somehow monkey-patch the modules you 
> want to use, and neither of those options are entirely satisfactory.

Dang, I missed that. I could have abused that yesterday. :)

> Basically any method that can create a new TCP connection should acquire 
> an optional timeout=None parameter, right?

Yup, except that None shouldn't be the "I didn't pass anything"
marker, since None is a valid settimeout parameter.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From amk at amk.ca  Thu Dec 22 21:54:48 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 22 Dec 2005 15:54:48 -0500
Subject: [Python-Dev] LaTeX and Python doc contributions
In-Reply-To: <17322.61191.851677.47037@montanaro.dyndns.org>
References: <200512221144.42307.fdrake@acm.org>
	<17322.61191.851677.47037@montanaro.dyndns.org>
Message-ID: <20051222205448.GA15763@rogue.amk.ca>

On Thu, Dec 22, 2005 at 12:23:03PM -0600, skip at pobox.com wrote:
> Who is asking this of potential contributors?  I know you, Aahz and I have
> repeatedly told people on c.l.py that LaTeX knowledge is not necessary.

One comment on a bug to this effect was found.  I don't think there's
a point in naming names -- the person in question doubtless just
wasn't aware of this policy.

--amk

From blais at furius.ca  Thu Dec 22 22:08:11 2005
From: blais at furius.ca (Martin Blais)
Date: Thu, 22 Dec 2005 16:08:11 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <1135196162.14495.15.camel@geddy.wooz.org>
References: <20051221095628.BE79.JCARLSON@uci.edu> <doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
Message-ID: <8393fff0512221308t1d635ef3p5b1bf1ecd324cf5a@mail.gmail.com>

On 12/21/05, Barry Warsaw <barry at python.org> wrote:
> On Wed, 2005-12-21 at 20:36 +0100, Fredrik Lundh wrote:
>
> > I'm not really interested in optimizing for you, I'm interested in optimizing
> > for everyone else.  They already know HTML.  They don't know ReST, and
> > I doubt they care about it (how many blogs accept ReST for comments?)
>
> Sorry, but HTML and (even more so) XML are not human-writable. :)  Yeah,
> we can all do the simple stuff, but I absolutely hate authoring in HTML,
> and it would be a nightmare if the documentation production system
> didn't handle lots and lots of magic for you (like weaving in the right
> footers, css, etc. -- oh wait, that's ht2html!).
>
> reST is a fine language but it seems more suitable to simpler linear
> documents like wiki pages and PEPs, rather than those with complicated
> nested structure.
>
> Maybe it's just because I came in late on this thread, but what exactly
> is broken about the current LaTeX documentation?
>
> -Barry

Good point.

Nothing is really "broken", but it's just not flexible because there
is no way to get a solid document model from LaTeX to do some
conversion and processing on.  i.e. you convert from LaTeX direct to
the output.  Having the intermediate representation would allow
generating nicer output, and in more formats, without necessarily
having to reparse the input everytime either.

What we need is not necessarily a change of syntax: the problem is not
the input, it's the conversion.  The input is fine--if someone can't
learn the super simple LaTeX macros for the Python docs, I don't want
to imagine what kind of prose they would come up with.  LaTeX is NOT
hard, at least if you limit yourself to the stuff you need to document
Python code.

About ReST:

Somehow there is a recurrent stream of people--include me at some
point-- who think that ReST could express any document structure for
any task, and that if we use that we will be happy ever after.  ReST
does an amazing job of inferring generic document structures from
text, but for documenting source code, you really want to be able to
say "This is a function", "this is an optional argument", etc.  ReST
does not provide this kind of functionality, and if you try to stretch
the interpreted roles to do this you get an equally ugly syntax as
LaTeX input (I would even argue that I prefer the LaTeX source).

Also, ReST has many gotchas: if you will infer structures from
invisible markup, it's very easy to make mistakes, and there are many
cases where it's not clear what the parsed document will be like, you
have to "learn" a lot of how it parses the documents, and the corner
cases, by checking with rst2pseudoxml.py.  I'm facing this problem
with some of my Nabu extractors, which attempts to extract
semantically meaningful chunks out of the docutils tree, for example,
contact information.

If there is a problem it is not the input, it's the toolchain and conversion.

From arekm at pld-linux.org  Thu Dec 22 22:34:44 2005
From: arekm at pld-linux.org (Akradiusz Miskiewicz)
Date: Thu, 22 Dec 2005 22:34:44 +0100
Subject: [Python-Dev] timeout options in high-level networking modules
References: <43AAC796.6070506@zope.com> <doei4f$cvl$1@sea.gmane.org>
	<20051222163541.GC3543@discworld.dyndns.org>
Message-ID: <dof65k$j14$1@sea.gmane.org>

Charles Cazabon wrote:

> It might also be nice if the modules that rely on blocking mode being set
> on sockets (basically anything using socket.ssl()) actually explicitly set
> that
> first.  Right now, if you do socket.setdefaulttimeout() to a non-None
> value and then try to use anything that does SSL (poplib, imaplib), the
> connections will quickly die.
There is a patch for that in python patch tracking system. Just someone
needs to recheck it and apply.

> 
> Charles



From fdrake at acm.org  Thu Dec 22 23:37:42 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 22 Dec 2005 17:37:42 -0500
Subject: [Python-Dev] LaTeX and Python doc contributions
In-Reply-To: <e04bdf310512221039x5fe9a9fay@mail.gmail.com>
References: <200512221144.42307.fdrake@acm.org>
	<17322.61191.851677.47037@montanaro.dyndns.org>
	<e04bdf310512221039x5fe9a9fay@mail.gmail.com>
Message-ID: <200512221737.42645.fdrake@acm.org>

On Thursday 22 December 2005 13:39, Facundo Batista wrote:
 > Very interesting. What I don't know here is how to submit patches...

"Patches" certainly isn't the right word for changes not described as source 
diffs.  I cleaned up some text about that on python.org earlier.

 > I mean, if they were in LaTeX, a diff file would be enough. But in
 > plain text (or ReST), how should people specify the corrections, the
 > position of new paragraphs, etc?

In English is fine.  I'd expect something like: in the section on imaplib, 
before the paragraph starting with "...".  I often get descriptions like this 
when people point out typos to the docs at python.org address; it works well, 
and has almost no barriers to entry at all.

 > I'm really interested in this, we've been discussing about docs in
 > Python Argentina and some people were willing to help (and scared
 > about LaTeX).

Hopefully we can make sure it's easy for everyone to contribute.  I'm 
certainly interested in suggestions, though I make all of them happen.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From fdrake at acm.org  Fri Dec 23 00:54:56 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 22 Dec 2005 18:54:56 -0500
Subject: [Python-Dev] LaTeX and Python doc contributions
In-Reply-To: <17322.61191.851677.47037@montanaro.dyndns.org>
References: <200512221144.42307.fdrake@acm.org>
	<17322.61191.851677.47037@montanaro.dyndns.org>
Message-ID: <200512221854.56900.fdrake@acm.org>

On Thursday 22 December 2005 13:23, skip at pobox.com wrote:
 > Who is asking this of potential contributors?  I know you, Aahz and I have
 > repeatedly told people on c.l.py that LaTeX knowledge is not necessary.
 > Plain text is okay.  What do we need to do to squash this meme?

As Andrew noted, it doesn't really matter who it was.  That person is now 
aware of what's going on, I think.  :-)

I've added a note to the "developer's intro," and there should probably be a 
note in the development FAQ as well.

 > Tony & other python-dev summarizers (and maybe Cameron Laird for the
 > c.l.py summaries): please make a note of this in your next summary.  The
 > I-can't-contribute-because-I-don't-know-LaTeX notion has to die, die, die.

An excellent idea!


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From fdrake at acm.org  Fri Dec 23 00:58:49 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 22 Dec 2005 18:58:49 -0500
Subject: [Python-Dev] documentation comments
In-Reply-To: <ee2a432c0512221044o71f01392yeb4c9b5adbac3318@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<20051222142206.GB15340@rogue.amk.ca>
	<ee2a432c0512221044o71f01392yeb4c9b5adbac3318@mail.gmail.com>
Message-ID: <200512221858.49521.fdrake@acm.org>

On Thursday 22 December 2005 13:44, Neal Norwitz wrote:
 > I would help assuming this is easy--meaning a single click to remove a
 > comment.

It looks like the system the MySQL folks are using makes it easy, but I've not 
tried polluting their documentation with tests, just in case.  :-)

In general, my worry is less with dealing with spam than with ensuring 
integration of content enhancements before release candidates go out.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From pje at telecommunity.com  Fri Dec 23 01:41:20 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Dec 2005 19:41:20 -0500
Subject: [Python-Dev] reST limitations? (was Re: status of development
 documentation)
In-Reply-To: <8393fff0512221308t1d635ef3p5b1bf1ecd324cf5a@mail.gmail.com
 >
References: <1135196162.14495.15.camel@geddy.wooz.org>
	<20051221095628.BE79.JCARLSON@uci.edu> <doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
Message-ID: <5.1.1.6.0.20051222192801.04fac608@mail.telecommunity.com>

At 04:08 PM 12/22/2005 -0500, Martin Blais wrote:
>ReST does an amazing job of inferring generic document structures from
>text, but for documenting source code, you really want to be able to
>say "This is a function", "this is an optional argument", etc.  ReST
>does not provide this kind of functionality, and if you try to stretch
>the interpreted roles to do this you get an equally ugly syntax as
>LaTeX input (I would even argue that I prefer the LaTeX source).

That sounds like a misuse of the tool to me; if you need structured, 
extractable information from a reST document, fields and directives are 
probably going to be the way to go.  Similarly, I'd suggest that 
interpreted roles to identify the context of a name probably isn't the best 
way to go about it either; a link to the definition of the referenced item 
will be more useful and more uniform.  A formatter or intermediate 
processor can then decide whether it should actually be rendered as a 
hyperlink, a fully-qualified name, or just the function/method/class name.

So, definitions of functions, classes, and other structured stuff would 
just use fields under a directive, and references to those definitions 
would just be reST links.


>Also, ReST has many gotchas: if you will infer structures from
>invisible markup, it's very easy to make mistakes, and there are many
>cases where it's not clear what the parsed document will be like, you
>have to "learn" a lot of how it parses the documents, and the corner
>cases, by checking with rst2pseudoxml.py.

Huh?  I've never used rst2pseudoxml.py, so I don't understand how it's a 
requirement.  Do you mean, if you're writing some kind of reST processor 
it's helpful to understand how stuff is parsed?

Can you list some of these "gotchas"?  I've on maybe two occasions had to 
add a backslash to something to prevent it being interpreted as markup, and 
that's about it, although I've written many hundreds of K of Python 
documentation in reST.  (Not the Python core documentation, but other open 
source projects written in Python.)


From walter at livinglogic.de  Fri Dec 23 02:11:55 2005
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Fri, 23 Dec 2005 02:11:55 +0100
Subject: [Python-Dev] status of development documentation
In-Reply-To: <5.1.1.6.0.20051222101850.01b12df8@mail.telecommunity.com>
References: <5.1.1.6.0.20051222101850.01b12df8@mail.telecommunity.com>
Message-ID: <43AB4EDB.20805@livinglogic.de>

Phillip J. Eby wrote:

> At 10:27 AM 12/22/2005 +0100, Walter D?rwald wrote:
>> Phillip J. Eby wrote:
>>
>> > [...]
>> >
>> > If someone has examples of actual "Pythondoc" markup that don't 
>> translate
>> > to reST, I'd be really interested in seeing them, just for my own
>> > education.  Of course, I'd also be curious how common such 
>> constructs are.
>>
>> I'm using XML markup for our packages. Examples can be found at
>> [snip]
> 
> By "Pythondoc", I mean the LaTeX-based markup system being used for the 
> official Python documentation, not arbitrary methods of documentation 
> for Python code.

OK, I didn't realize that.

I guess the only thing compatible with LaTeX is LaTeX.

I'd really like to see a version of Fred's XML converter that works for 
the current Python documentation.

>> The source is definitely wordier than reST, but adding new markup is
>> trivial. Take a look at
>> http://www.livinglogic.de/Python/xist/Download.html and at the source at
>> http://www.livinglogic.de/Python/xist/Download.htmlxsc. The download
>> element automatically determines the size of the package. Source can be
>> found here
>> http://www.livinglogic.de/viewcvs/index.cgi/LivingLogic/WWW-Python/site/Python_xmlns.py?rev=1.43&content-type=text/vnd.viewcvs-markup 
>>
>> (search for "class download"). Would something like this be possible
>> with reST?
> 
> The docutils toolchain converts reST input into a DOM, and allows 
> arbitrary transformation phases to be added to processing before 
> conversion to output.  This includes processing of "directives", e.g. 
> commands like:
> 
> .. include:: filename
> 
> And of interpreted text "roles",  e.g. `Foobar`:class:.

This sound like it should be possible.

> It is not, however, a general XML transformation toolkit, if that's what 
> you're asking.  However, if you wanted to be able to use XML input as 
> part of a docutils DOM, you could certainly do that.

More the other way around.

> For that matter, 
> you could take a reST document and simply transform it to XML for use 
> with the rest of your toolset.

That's the way I'd like to use docutils: Write docstring in reST and 
transform them via XML tools.

> But this isn't particularly relevant to the discussion about *Python's* 
> documentation, and I'm not even advocating that Python switch, let alone 
> arbitrary other projects.

If we had a way to losslessly convert Python-LaTeX to XML tools for both 
system could live side by side.

Bye,
    Walter D?rwald


From blais at furius.ca  Fri Dec 23 02:56:20 2005
From: blais at furius.ca (Martin Blais)
Date: Thu, 22 Dec 2005 20:56:20 -0500
Subject: [Python-Dev] reST limitations? (was Re: status of development
	documentation)
In-Reply-To: <5.1.1.6.0.20051222192801.04fac608@mail.telecommunity.com>
References: <20051221095628.BE79.JCARLSON@uci.edu> <doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<5.1.1.6.0.20051222192801.04fac608@mail.telecommunity.com>
Message-ID: <8393fff0512221756h199099a4h5b69c24aa490f19c@mail.gmail.com>

On 12/22/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 04:08 PM 12/22/2005 -0500, Martin Blais wrote:
> >ReST does an amazing job of inferring generic document structures from
> >text, but for documenting source code, you really want to be able to
> >say "This is a function", "this is an optional argument", etc.  ReST
> >does not provide this kind of functionality, and if you try to stretch
> >the interpreted roles to do this you get an equally ugly syntax as
> >LaTeX input (I would even argue that I prefer the LaTeX source).
>
> That sounds like a misuse of the tool to me; if you need structured,
> extractable information from a reST document, fields and directives are
> probably going to be the way to go.  Similarly, I'd suggest that
> interpreted roles to identify the context of a name probably isn't the best
> way to go about it either; a link to the definition of the referenced item
> will be more useful and more uniform.  A formatter or intermediate
> processor can then decide whether it should actually be rendered as a
> hyperlink, a fully-qualified name, or just the function/method/class name.
>
> So, definitions of functions, classes, and other structured stuff would
> just use fields under a directive, and references to those definitions
> would just be reST links.

So you end up with a document with a bunch of custom directives, like::

.. python-class::  MyClass
   :arg: comfobulator
   :arg optional: bliptor

   My Class description.

This does not look significantly better to me than the LaTeX code, and
the docutils directives are not as flexible as the commands provided
by tex/latex.



> >Also, ReST has many gotchas: if you will infer structures from
> >invisible markup, it's very easy to make mistakes, and there are many
> >cases where it's not clear what the parsed document will be like, you
> >have to "learn" a lot of how it parses the documents, and the corner
> >cases, by checking with rst2pseudoxml.py.
>
> Huh?  I've never used rst2pseudoxml.py, so I don't understand how it's a
> requirement.  Do you mean, if you're writing some kind of reST processor
> it's helpful to understand how stuff is parsed?

It is if you're relying on specific document structures to provide
information about your special constructs.  rst2pseudoxml.py just
helps you display that parsed structure.

For example, you could write some kind of processor on the docutils
document tree that looks for definition lists whose "term" starts with
"class" and then assumes some other things about what it will find in
the body of this definition, e.g.

class MyClass
  <Some indented material
  about the class MyClass>

(this gets parsed as a ReST definition term/body because of the
indented line right after the "class MyClass" line).

I'm doing this kind of processing, albeit in a limited way, to extract
book reviews, bookmarks, and contact info from a body of text files
that I maintain, using my nabu system (http://furius.ca/nabu/  come to
my talk at PyCon for mode details).  This might be a better way to
hijack ReST than to create a gazillion custom directives, thereby
creating more or less another markup language (with a smaller
userbase, less tested!).


> Can you list some of these "gotchas"?  I've on maybe two occasions had to
> add a backslash to something to prevent it being interpreted as markup, and
> that's about it, although I've written many hundreds of K of Python
> documentation in reST.  (Not the Python core documentation, but other open
> source projects written in Python.)

Lots. No time to go through the whole list now, but here is an example:

-----------------
..
  Some text
-----------------

and

-----------------
..

  Some text
-----------------

Generate the following document structures, respectively:

    <comment xml:space="preserve">
        Some text

and

    <comment xml:space="preserve">
    <block_quote>
        <paragraph>
            Some text

One is a comment, the other is an empty comment followed by a block
quote.  Not very obvious to me, unless you know "the rules".  Easy to
make mistakes.  There are *many* other issues just like this one.  
Pop quiz: what does this generate?

-----------------
..
  Some text

  Some other text
-----------------

From stephen at xemacs.org  Fri Dec 23 04:46:55 2005
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 23 Dec 2005 12:46:55 +0900
Subject: [Python-Dev] LaTeX and Python doc contributions
In-Reply-To: <200512221854.56900.fdrake@acm.org> (Fred L. Drake, Jr.'s
	message of "Thu, 22 Dec 2005 18:54:56 -0500")
References: <200512221144.42307.fdrake@acm.org>
	<17322.61191.851677.47037@montanaro.dyndns.org>
	<200512221854.56900.fdrake@acm.org>
Message-ID: <877j9wjx00.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Fred" == Fred L Drake, <fdrake at acm.org> writes:

    Fred> On Thursday 22 December 2005 13:23, skip at pobox.com wrote:

    >> Who is asking this of potential contributors?  I know you, Aahz
    >> and I have repeatedly told people on c.l.py that LaTeX
    >> knowledge is not necessary.  Plain text is okay.  What do we
    >> need to do to squash this meme?

    Fred> As Andrew noted, it doesn't really matter who it was.

I interpreted Skip's first question as 100% rhetorical.<wink>

I think one aspect of the meme is that projects generally strongly
emphasize standard-format patches to source for code.  But this is
typically less important for documentation, where good and consistent
natural language style probably means that the editor applies the
patch, and then revises in place rather than requesting a revision
from the contributor.

I don't know whether that distinction helps with creating a vaccine,
though.  I don't see an obvious application beyond the suggestion of
saying "patches aren't necessary" more frequently and prominently.

-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From ncoghlan at gmail.com  Fri Dec 23 06:41:05 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 23 Dec 2005 15:41:05 +1000
Subject: [Python-Dev] status of development documentation
In-Reply-To: <e04bdf310512220640k775cb19fk@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<doc6h2$dsi$1@sea.gmane.org>	<20051221105132.BE7C.JCARLSON@uci.edu>
	<docare$tjs$1@sea.gmane.org>	<1135196162.14495.15.camel@geddy.wooz.org>	<5.1.1.6.0.20051221152616.0209a2f8@mail.telecommunity.com>
	<e04bdf310512220640k775cb19fk@mail.gmail.com>
Message-ID: <43AB8DF1.5030000@gmail.com>

Facundo Batista wrote:
> 2005/12/21, Phillip J. Eby <pje at telecommunity.com>:
> 
>> 3. Fredrik believes that more people would participate in updating Python
>> documentation if it didn't require a LaTeX toolchain or LaTeX-friendly editor.
> 
> I'm sure he's right. I'm not talking about any random user that finds
> a doc bug and wants to generate a patch, here I'm talking of my own
> experience:
> 
> I had to correct a few lines in the almost perfect documentation that
> Raymond generated for Decimal. I fighted with my Linux (at that time,
> FC1) to be able to compile the docs, and couldn't do it.
> 
> I ended touching the XML by hand. It worked, but
> 
>   a) Took some time.
>   b) Wasn't really sure that it was well corrected.
> 
> So, I really think that a more human friendly format will help here.
> 
> What I do NOT know, if the effort of converting the whole docs to
> another format is worth it, and that effort should be deviated to
> something that will help more other users to help with docs (for
> example, that the official docs could be annotatted, a la MySQL (AMK
> did something like this, right?)).

If I remember rightly, the biggest problem I had in the whole exercise was 
getting latex2html to run - I actually had to modify the Perl source to get it 
to work (fortunately, I didn't have to try to *understand* said source first - 
other people had already figured out the necessary incantations, so I was able 
to find out how to fix it via a Google search).

latex/tex weren't a big problem, because they were in the distro archives - 
but latex2html was a definite pain.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From robey at lag.net  Fri Dec 23 06:58:55 2005
From: robey at lag.net (Robey Pointer)
Date: Thu, 22 Dec 2005 21:58:55 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <2m4q512vub.fsf@starship.python.net>
References: <20051221095628.BE79.JCARLSON@uci.edu> <doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org> <2m4q512vub.fsf@starship.python.net>
Message-ID: <1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>


On 22 Dec 2005, at 3:51, Michael Hudson wrote:

> "Fredrik Lundh" <fredrik at pythonware.com> writes:
>
>> Checked the python-list archives lately?  If you google c.l.python  
>> for the
>> word "documentation", you'll find recent megathreads with subjects  
>> like
>> "bitching about the documentation", "opensource documentation  
>> problems"
>> and "python documentation should be better" among the top hits.   
>> But if
>> you check the bug and patch trackers, you don't find many  
>> contributions.
>> Something's definitely broken.
>
> Hmm, it's this discussion again!  Let me make my point again!
>
> Writing good documentation is hard.

I can only speak for my own experience, but maybe it will help.  I  
once tried to help fix a piece of the python docs.  The description  
of Py_UNICODE on <http://docs.python.org/api/unicodeObjects.html> was  
-- and still is -- incorrect.

Looking through my mail archives, I sent a patch on 10 October, which  
was apparently taken, but never showed up on the web site.  I emailed  
a few reminders, but was eventually told that I should email a third  
person -- who didn't have an email address.  At that point I passed  
the level of effort I was willing to put in. :)  I think I probably  
put more effort into it than an average person would, so I think the  
barriers of entry are much higher than they should be.

Perhaps something with fast feedback would work a lot better.  It  
seems to work well for Wikipedia.

robey


From fredrik at pythonware.com  Fri Dec 23 09:14:26 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri, 23 Dec 2005 09:14:26 +0100
Subject: [Python-Dev] reST limitations? (was Re: status of development
	documentation)
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<doc6h2$dsi$1@sea.gmane.org><20051221105132.BE7C.JCARLSON@uci.edu>
	<docare$tjs$1@sea.gmane.org><1135196162.14495.15.camel@geddy.wooz.org><5.1.1.6.0.20051222192801.04fac608@mail.telecommunity.com>
	<8393fff0512221756h199099a4h5b69c24aa490f19c@mail.gmail.com>
Message-ID: <dogbl4$j0k$1@sea.gmane.org>

Martin Blais wrote:

> > So, definitions of functions, classes, and other structured stuff would
> > just use fields under a directive, and references to those definitions
> > would just be reST links.
>
> So you end up with a document with a bunch of custom directives, like::
>
> .. python-class::  MyClass
>    :arg: comfobulator
>    :arg optional: bliptor
>
>    My Class description.
>
> This does not look significantly better to me than the LaTeX code, and
> the docutils directives are not as flexible as the commands provided
> by tex/latex.

Except that tex/latex don't give you the same structure.  Using the real-
life example from the documentation list again:

- LaTeX with Python extensions:

   \begin{funcdesc}{dumps}{params\optional{, methodname\optional{,
                            methodresponse\optional{, encoding}}}}
    Convert \var{params} into an XML-RPC request.
    or into a response if \var{methodresponse} is true.
    \var{params} can be either a tuple of arguments or an instance of the
    \exception{Fault} exception class.  If \var{methodresponse} is true,
    only a single value can be returned, meaning that \var{params} must be of length 1.
    \var{encoding}, if supplied, is the encoding to use in the generated
    XML; the default is UTF-8.
    \end{funcdesc}

- hypothetical ReST (based on martin's example and the above LaTeX
  markup):

    .. python-function:: dumps
    :arg: params
    :arg optional: methodname
    :arg optional: methodresponse
    :arg optional: encoding

    Convert _`params` into an XML-RPC request, or into a response if
    _`methodresponse` is true.
    _`params` can be either a tuple of arguments or an instance of the
    `:exception: _`Fault` exception class.  If _`methodresponse` is true,
    only a single value can be returned, meaning that _`params` must be
    of length 1.
    _`encoding`, if supplied, is the encoding to use in the generated
    XML; the default is UTF-8.
    .. python-function-end

Informationwise, this is mostly identical to the latex example, except that
you can use existing tools to get an XML structure from this markup.

- JavaDoc/PythonDoc markup:

    Converts a Python tuple or a Fault instance to an XML-RPC request.

    @def dumps(params, **options)
    @param params A tuple or {@link Fault} instance.
    @keyparam methodname If given, create a call request for
        this method name.
    @keyparam methodresponse If given, create a response request.
        If used with a tuple, the tuple must be a singleton (that is,
        it must contain exactly one element).
    @keyparam encoding The encoding to use for this request. Defaults to
        UTF-8.
    @return A string containing marshalled data.

The LaTeX solution is one line shorter, but the JavaDoc/PythonDoc solution
squeezes a lot more structural information into those 11 lines.

Also note that the JavaDoc/PythonDoc version is the only one that reflects
the designer's intent: all arguments but the first are keyword options, not
optional positional arguments (I don't know how to express that efficiently
in today's LaTeX markup).

It's also the only one here where existing tools can be used to get a clean
information model

   <function name="xmlrpclib.dumps">
        <info>
            <name>dumps</name>
            <summary>Convert a Python tuple or a Fault instance to an
                XML-RPC request.</summary>
            <description>Convert a Python tuple or a Fault instance to an
                XML-RPC request.</description>
            <def>dumps(params, **options)</def>
            <param name="params">A tuple or Fault instance.</param>
            <keyparam name="methodname">If given, create a methodCall
                request for this method name.</keyparam>
            <keyparam name="methodresponse">If given, create a
                methodResponse request. If used with a tuple, the tuple must
                be a singleton (that is, it must contain exactly one element).
                </keyparam>
            <keyparam name="encoding">The request encoding. Defaults to
                UTF-8.</keyparam>
            <return>A string containing marshalled data.</return>
        </info>
    </function>

for further processing (using XSLT to turn this into nice HTML is trivial, for
example).  It's impossible to extract this level of information from the given
LaTeX and ReST examples.

</F>




From reinhold-birkenfeld-nospam at wolke7.net  Fri Dec 23 09:53:50 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Fri, 23 Dec 2005 09:53:50 +0100
Subject: [Python-Dev] status of development documentation
In-Reply-To: <1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<doc6h2$dsi$1@sea.gmane.org>	<20051221105132.BE7C.JCARLSON@uci.edu>
	<docare$tjs$1@sea.gmane.org>	<1135196162.14495.15.camel@geddy.wooz.org>	<doci50$nom$1@sea.gmane.org>
	<2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
Message-ID: <dogduu$o1m$1@sea.gmane.org>

Robey Pointer wrote:
> On 22 Dec 2005, at 3:51, Michael Hudson wrote:
> 
>> "Fredrik Lundh" <fredrik at pythonware.com> writes:
>>
>>> Checked the python-list archives lately?  If you google c.l.python  
>>> for the
>>> word "documentation", you'll find recent megathreads with subjects  
>>> like
>>> "bitching about the documentation", "opensource documentation  
>>> problems"
>>> and "python documentation should be better" among the top hits.   
>>> But if
>>> you check the bug and patch trackers, you don't find many  
>>> contributions.
>>> Something's definitely broken.
>>
>> Hmm, it's this discussion again!  Let me make my point again!
>>
>> Writing good documentation is hard.
> 
> I can only speak for my own experience, but maybe it will help.  I  
> once tried to help fix a piece of the python docs.  The description  
> of Py_UNICODE on <http://docs.python.org/api/unicodeObjects.html> was  
> -- and still is -- incorrect.

The current docs were released on September 28. They are not updated until
the next Python release, so that's probably why your patch doesn't show up there.

That may not be a good thing. Documentation fixes should go online much
quicker than with every Python release, or am I mistaken?

Reinhold

-- 
Mail address is perfectly valid!


From fredrik at pythonware.com  Fri Dec 23 10:06:47 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri, 23 Dec 2005 10:06:47 +0100
Subject: [Python-Dev] status of development documentation
References: <20051221095628.BE79.JCARLSON@uci.edu><doc6h2$dsi$1@sea.gmane.org>	<20051221105132.BE7C.JCARLSON@uci.edu><docare$tjs$1@sea.gmane.org>	<1135196162.14495.15.camel@geddy.wooz.org>	<doci50$nom$1@sea.gmane.org><2m4q512vub.fsf@starship.python.net><1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
Message-ID: <dogen9$qj5$1@sea.gmane.org>

Reinhold Birkenfeld wrote:

> > I can only speak for my own experience, but maybe it will help.  I
> > once tried to help fix a piece of the python docs.  The description
> > of Py_UNICODE on <http://docs.python.org/api/unicodeObjects.html> was
> > -- and still is -- incorrect.
>
> The current docs were released on September 28. They are not updated until
> the next Python release, so that's probably why your patch doesn't show up there.
>
> That may not be a good thing. Documentation fixes should go online much
> quicker than with every Python release, or am I mistaken?

as was noted at the top of this thread, the "trunk" documentation on python.org
is even older.  a bit further down the thread, Trent Mick came to the rescue:

        http://trentm.com/python/

</F>




From skip at pobox.com  Fri Dec 23 19:07:40 2005
From: skip at pobox.com (skip@pobox.com)
Date: Fri, 23 Dec 2005 12:07:40 -0600
Subject: [Python-Dev] status of development documentation
In-Reply-To: <1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
References: <20051221095628.BE79.JCARLSON@uci.edu> <doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org> <2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
Message-ID: <17324.15596.688668.838228@montanaro.dyndns.org>


    Robey> I can only speak for my own experience, but maybe it will help.
    Robey> I once tried to help fix a piece of the python docs.  The
    Robey> description of Py_UNICODE on
    Robey> <http://docs.python.org/api/unicodeObjects.html> was
    Robey> -- and still is -- incorrect.

Check here:

    http://www.trentm.com/python/dailyhtml/api/unicodeObjects.html

Is it up-to-date?  If so, your changes have made it into the documentation
source.

Skip

From robey at lag.net  Fri Dec 23 21:58:15 2005
From: robey at lag.net (Robey Pointer)
Date: Fri, 23 Dec 2005 12:58:15 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <dogduu$o1m$1@sea.gmane.org>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<doc6h2$dsi$1@sea.gmane.org>	<20051221105132.BE7C.JCARLSON@uci.edu>
	<docare$tjs$1@sea.gmane.org>	<1135196162.14495.15.camel@geddy.wooz.org>	<doci50$nom$1@sea.gmane.org>
	<2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
Message-ID: <5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>


On 23 Dec 2005, at 0:53, Reinhold Birkenfeld wrote:

> Robey Pointer wrote:
>> On 22 Dec 2005, at 3:51, Michael Hudson wrote:
>>
>>> "Fredrik Lundh" <fredrik at pythonware.com> writes:
>>>
>>>> Checked the python-list archives lately?  If you google c.l.python
>>>> for the
>>>> word "documentation", you'll find recent megathreads with subjects
>>>> like
>>>> "bitching about the documentation", "opensource documentation
>>>> problems"
>>>> and "python documentation should be better" among the top hits.
>>>> But if
>>>> you check the bug and patch trackers, you don't find many
>>>> contributions.
>>>> Something's definitely broken.
>>>
>>> Hmm, it's this discussion again!  Let me make my point again!
>>>
>>> Writing good documentation is hard.
>>
>> I can only speak for my own experience, but maybe it will help.  I
>> once tried to help fix a piece of the python docs.  The description
>> of Py_UNICODE on <http://docs.python.org/api/unicodeObjects.html> was
>> -- and still is -- incorrect.
>
> The current docs were released on September 28. They are not  
> updated until
> the next Python release, so that's probably why your patch doesn't  
> show up there.
>
> That may not be a good thing. Documentation fixes should go online  
> much
> quicker than with every Python release, or am I mistaken?

Yes, I think that's obviously ridiculous on the face of it, since  
fixes to the python 2.4 docs may be useless by the time 2.5 comes  
out, and may be too late to help anyone anyway. :)

I'm glad I'm not the only one who found the process completely  
broken, at least.

robey


From nnorwitz at gmail.com  Fri Dec 23 22:47:35 2005
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Fri, 23 Dec 2005 13:47:35 -0800
Subject: [Python-Dev] Patch reviews & request for patch review
In-Reply-To: <Pine.LNX.4.58.0512042309580.6221@alice>
References: <Pine.LNX.4.58.0512042309580.6221@alice>
Message-ID: <ee2a432c0512231347r54a4b3b7kd594a7835176b2be@mail.gmail.com>

On 12/5/05, John J Lee <jjl at pobox.com> wrote:
>
> I'm hoping one of those nice people who offered 'review 5 get 1 free'
> might look at a patch of mine.

Oooh, ooh, do I count?!?  (Well, you can drop the nice part. :-)

> http://python.org/sf/1157027

Checked in to 2.5.

> http://python.org/sf/1117398

Checked in to 2.4 and 2.5.

> Thanks in advance to anybody who has time to look at these,

Sorry it took so long.

n

From skip at pobox.com  Fri Dec 23 23:39:53 2005
From: skip at pobox.com (skip@pobox.com)
Date: Fri, 23 Dec 2005 16:39:53 -0600
Subject: [Python-Dev] status of development documentation
In-Reply-To: <5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
References: <20051221095628.BE79.JCARLSON@uci.edu> <doc6h2$dsi$1@sea.gmane.org>
	<20051221105132.BE7C.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org> <2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
Message-ID: <17324.31929.979592.958222@montanaro.dyndns.org>


    >> That may not be a good thing. Documentation fixes should go online
    >> much quicker than with every Python release, or am I mistaken?

    Robey> Yes, I think that's obviously ridiculous on the face of it, since
    Robey> fixes to the python 2.4 docs may be useless by the time 2.5 comes
    Robey> out, and may be too late to help anyone anyway. :)

So for at least the time being they go up nightly
(http://www.trentm.com/python).  I don't know what Trent did to make that
happen, but he did it fairly quickly.  I doubt it would be hard to replicate
on the docs server.

Skip


From nnorwitz at gmail.com  Sat Dec 24 03:28:54 2005
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Fri, 23 Dec 2005 18:28:54 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <17324.31929.979592.958222@montanaro.dyndns.org>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<20051221105132.BE7C.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org> <2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
Message-ID: <ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>

On 12/23/05, skip at pobox.com <skip at pobox.com> wrote:
>
> So for at least the time being they go up nightly
> (http://www.trentm.com/python).  I don't know what Trent did to make that
> happen, but he did it fairly quickly.  I doubt it would be hard to replicate
> on the docs server.

I couldn't let Trent have all the fun.

    http://docs.python.org/dev/

And hopefully of interest to many here:

    http://docs.python.org/dev/results/

These are the results of svn update, configure, build, test, install
and the doc run.
Run on the PSFs box and updated every 12 hours.  I currently have it
send mail to me if there are any test failures.  I will probably
update that to python-checkins or maybe even python-dev depending on
what people think.  I'm not likely to be around much for the rest of
the year, so I don't want to turn it on just yet.

The script should be updated to handle more types of failures. 
configure should probably specify CFLAGS of --std=c89 and maybe others
too.

I should check this script in somewhere, but I'm not sure where.

n

From bcannon at gmail.com  Sat Dec 24 03:48:55 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Fri, 23 Dec 2005 18:48:55 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org> <2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
Message-ID: <bbaeab100512231848u3264e4dbseedc0d54d6a41e56@mail.gmail.com>

On 12/23/05, Neal Norwitz <nnorwitz at gmail.com> wrote:
> On 12/23/05, skip at pobox.com <skip at pobox.com> wrote:
> >
> > So for at least the time being they go up nightly
> > (http://www.trentm.com/python).  I don't know what Trent did to make that
> > happen, but he did it fairly quickly.  I doubt it would be hard to replicate
> > on the docs server.
>
> I couldn't let Trent have all the fun.
>
>     http://docs.python.org/dev/
>

Cool!  Thanks to Trent for sparking Neal, and thanks to Neal for
feeling the fire under our arses for getting this done.

> And hopefully of interest to many here:
>
>     http://docs.python.org/dev/results/
>
> These are the results of svn update, configure, build, test, install
> and the doc run.
> Run on the PSFs box and updated every 12 hours.  I currently have it
> send mail to me if there are any test failures.  I will probably
> update that to python-checkins or maybe even python-dev depending on
> what people think.  I'm not likely to be around much for the rest of
> the year, so I don't want to turn it on just yet.
>

python-checkins seems the most reasonable.  But I would have no
problem with it going to python-dev.

> The script should be updated to handle more types of failures.
> configure should probably specify CFLAGS of --std=c89 and maybe others
> too.
>

With the recent issues in the compiler not being C89 compatible that
would probably be good.

-Brett

From kbk at shore.net  Sat Dec 24 04:58:36 2005
From: kbk at shore.net (Kurt B. Kaiser)
Date: Fri, 23 Dec 2005 22:58:36 -0500 (EST)
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200512240358.jBO3waMh004567@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  379 open ( -4) /  3002 closed (+12) /  3381 total ( +8)
Bugs    :  914 open (-13) /  5452 closed (+37) /  6366 total (+24)
RFE     :  204 open ( +0) /   193 closed ( +1) /   397 total ( +1)

New / Reopened Patches
______________________

Some fixes for the binary distribution builder  (2005-12-14)
       http://python.org/sf/1380777  opened by  Ronald Oussoren

Fix bug read() would hang on ssl socket if settimeout() used  (2005-12-15)
       http://python.org/sf/1380952  opened by  Arkadiusz Miskiewicz

bind() for netlink sockets  (2005-12-15)
       http://python.org/sf/1381398  opened by  Timo Mets?l?

list.count() patch for feature request 1370948  (2005-12-15)
CLOSED http://python.org/sf/1382087  opened by  Mike Fondo

Expose Subversion revision number  (2005-12-16)
CLOSED http://python.org/sf/1382163  opened by  Barry A. Warsaw

Specify new reference return value for PyObject_Call  (2005-12-17)
CLOSED http://python.org/sf/1383115  opened by  Farshid Lashkari

Make unittest.TestCase easier to subclass  (2005-12-22)
       http://python.org/sf/1388073  opened by  Collin Winter

add more readline support  (2005-12-22)
       http://python.org/sf/1388440  opened by  Sebastien Boving

Patches Closed
______________

PyXxx_Check() speed-up  (2005-02-27)
       http://python.org/sf/1153056  closed by  arigo

[PATCH] mmap fails on AMD64  (2005-11-24)
       http://python.org/sf/1365916  closed by  nnorwitz

list.count() patch for feature request 1370948  (2005-12-15)
       http://python.org/sf/1382087  closed by  gvanrossum

Expose Subversion revision number  (2005-12-16)
       http://python.org/sf/1382163  closed by  bwarsaw

fix description of format_exc in traceback doc  (2005-12-09)
       http://python.org/sf/1376914  closed by  birkenfeld

xml.parsers.expat documentation fix  (2005-12-10)
       http://python.org/sf/1377848  closed by  birkenfeld

correct display of pathnames in SimpleHTTPServer  (2005-11-18)
       http://python.org/sf/1360443  closed by  birkenfeld

PEP 341 - Unification of try/except and try/finally  (2005-11-13)
       http://python.org/sf/1355913  closed by  nnorwitz

Specify new reference return value for PyObject_Call  (2005-12-17)
       http://python.org/sf/1383115  closed by  effbot

LibRef: reworked chapter organization  (2005-12-07)
       http://python.org/sf/1375417  closed by  akuchling

cookielib LWPCookieJar and MozillaCookieJar exceptions  (2005-02-06)
       http://python.org/sf/1117398  closed by  nnorwitz

cookielib mis-handles RFC 2109 cookies in Netscape mode  (2005-03-04)
       http://python.org/sf/1157027  closed by  nnorwitz

New / Reopened Bugs
___________________

split() description not fully accurate  (2005-12-14)
       http://python.org/sf/1380970  opened by  K.C.

csv.reader endless loop   (2005-12-15)
       http://python.org/sf/1381476  opened by  Christian Harms

mode 't' not documented as posssible mode for file built-in  (2005-12-15)
       http://python.org/sf/1381717  opened by  Simo Salminen

cElementTree only supports a few encodings  (2005-12-15)
CLOSED http://python.org/sf/1381939  opened by  Fredrik Lundh

MacRoman Encoding Bug (OHM vs. OMEGA)  (2005-12-16)
CLOSED http://python.org/sf/1382096  opened by  Sean B. Palmer

Tutorial section 9.5.1 ignores MRO for new-style classes  (2005-12-16)
       http://python.org/sf/1382213  opened by  GaryD

--install-base not honored on win32  (2005-12-16)
       http://python.org/sf/1382562  opened by  John Ehresman

len() on class broken  (2005-12-16)
       http://python.org/sf/1382740  reopened by  kquick

len() on class broken  (2005-12-16)
       http://python.org/sf/1382740  opened by  Kevin Quick

len() on class broken  (2005-12-16)
CLOSED http://python.org/sf/1382815  opened by  Kevin Quick

MacOS.WMAvailable() doesn't launch Python.app properly  (2005-12-17)
CLOSED http://python.org/sf/1383644  opened by  has

random module - Provider DLL failed to initialize correctly  (2005-12-18)
       http://python.org/sf/1384175  opened by  Greg Hazel

exec statement link in index broken  (2005-12-19)
       http://python.org/sf/1385004  opened by  Harri Pasanen

compiler module does not detect a syntax error  (2005-12-19)
       http://python.org/sf/1385040  opened by  Harri Pasanen

execfile anomaly with "from __future__ import division"  (2005-12-19)
CLOSED http://python.org/sf/1385055  opened by  Harri Pasanen

_winreg specifies EnvironmentError instead of WindowsError  (2005-12-21)
       http://python.org/sf/1386675  opened by  Tony Meyer

sys.path[0] when executed thru a symbolic link  (2005-12-21)
CLOSED http://python.org/sf/1387483  opened by  Tomasz Kowaltowski

weird behavior when assigning locals() to a variable  (2005-12-22)
CLOSED http://python.org/sf/1387650  opened by  Samuel Bayer

Minor error in md5 docs  (2005-12-22)
CLOSED http://python.org/sf/1388141  opened by  Kent Johnson

bug in rstrip & lstrip  (2005-12-23)
CLOSED http://python.org/sf/1388489  opened by  Jason Whitlark

Polymorphic getters / setters  (2005-12-23)
CLOSED http://python.org/sf/1388804  opened by  Adde

xmlrpc howto link incorrect  (2005-12-23)
CLOSED http://python.org/sf/1388910  opened by  Jonathan Marshall

Decimal sqrt() ignores rounding  (2005-12-23)
       http://python.org/sf/1388949  opened by  Adam Olsen

imaplib causes excessive fragmentation for large documents  (2005-12-23)
       http://python.org/sf/1389051  opened by  Fredrik Lundh

test_tarfile fails with readonly source dir for Python 2.4.2  (2005-12-23)
       http://python.org/sf/1389157  opened by  Langly

Bugs Closed
___________

freeze idle-python2.3 on my debia n sarge  (2005-11-29)
       http://python.org/sf/1369116  closed by  kbk

Memory leak in the email package  (2005-12-09)
       http://python.org/sf/1376775  closed by  effbot

logging : fileConfig does not check existance of the file  (2005-12-12)
       http://python.org/sf/1378755  closed by  vsajip

source utf8  (2005-12-10)
       http://python.org/sf/1378022  closed by  nnorwitz

loogger module locks  (2005-11-27)
       http://python.org/sf/1367814  closed by  vsajip

cElementTree only supports a few encodings  (2005-12-15)
       http://python.org/sf/1381939  closed by  effbot

"unicode_escape" and "raw_unicode_escape" encoding is broken  (2005-12-14)
       http://python.org/sf/1379994  closed by  perky

a problem of urllib using open_local_file  (2005-12-12)
       http://python.org/sf/1378455  closed by  birkenfeld

uncaught AttributeError deep in urllib  (2005-03-15)
       http://python.org/sf/1163401  closed by  birkenfeld

MacRoman Encoding Bug (OHM vs. OMEGA)  (2005-12-16)
       http://python.org/sf/1382096  closed by  lemburg

len() on class broken  (2005-12-16)
       http://python.org/sf/1382740  closed by  birkenfeld

len() on class broken  (2005-12-16)
       http://python.org/sf/1382815  closed by  birkenfeld

MacOS.WMAvailable() doesn't launch Python.app properly  (2005-12-17)
       http://python.org/sf/1383644  closed by  jackjansen

os.makedirs() ignores mode parameter  (2005-01-21)
       http://python.org/sf/1106572  closed by  birkenfeld

Broken docs for os.removedirs  (2005-10-31)
       http://python.org/sf/1343671  closed by  birkenfeld

os.makedirs fail if path contains os.pardir  (2005-12-05)
       http://python.org/sf/1373197  closed by  birkenfeld

mmap does not accept length as 0  (2005-10-28)
       http://python.org/sf/1341031  closed by  birkenfeld

test_re failure on 64bit targets (alpha, ia64)  (2003-08-06)
       http://python.org/sf/783990  closed by  nnorwitz

apparently leaking snippet  (2003-09-18)
       http://python.org/sf/808596  closed by  nnorwitz

webbrowser.open hangs under certain conditions  (2003-10-02)
       http://python.org/sf/816810  closed by  nnorwitz

python 2.3.2 make test segfault  (2003-10-26)
       http://python.org/sf/830573  closed by  nnorwitz

Erroneous code objects created with PyCode_New  (2003-12-08)
       http://python.org/sf/856623  closed by  nnorwitz

2.4a0 build fails in Modules/signalmodule.c  (2004-01-21)
       http://python.org/sf/881812  closed by  nnorwitz

Line 0 SyntaxWarning with duplicate global declarations  (2004-02-02)
       http://python.org/sf/889500  closed by  nnorwitz

File read of Chinese utf-16-le treats upper byte 1A as EOF  (2004-02-25)
       http://python.org/sf/904474  closed by  nnorwitz

test_timeout failure on trunk  (2004-06-11)
       http://python.org/sf/971238  closed by  nnorwitz

Problems with os.system and ulimit -f  (2004-10-12)
       http://python.org/sf/1045509  closed by  nnorwitz

import.c bug when getting mod time fails  (2004-10-26)
       http://python.org/sf/1054615  closed by  nnorwitz

pwent objects from the pwd module ar e not pickle-able  (2004-11-08)
       http://python.org/sf/1062708  closed by  nnorwitz

bad arg type to isspace in struct module  (2004-11-23)
       http://python.org/sf/1072182  closed by  nnorwitz

sys.stdin segfaults on invalid stdin  (2004-12-13)
       http://python.org/sf/1084766  closed by  nnorwitz

execfile anomaly with "from __future__ import division"  (2005-12-19)
       http://python.org/sf/1385055  closed by  nnorwitz

sys.path[0] when executed thru a symbolic link  (2005-12-21)
       http://python.org/sf/1387483  closed by  jackjansen

weird behavior when assigning locals() to a variable  (2005-12-21)
       http://python.org/sf/1387650  closed by  nnorwitz

Minor error in md5 docs  (2005-12-22)
       http://python.org/sf/1388141  closed by  birkenfeld

bug in rstrip & lstrip  (2005-12-23)
       http://python.org/sf/1388489  closed by  birkenfeld

Polymorphic getters / setters  (2005-12-23)
       http://python.org/sf/1388804  closed by  birkenfeld

xmlrpc howto link incorrect  (2005-12-23)
       http://python.org/sf/1388910  closed by  effbot

New / Reopened RFE
__________________

Polymorphic getters / setters  (2005-12-23)
       http://python.org/sf/1388872  opened by  Adde

RFE Closed
__________

Start and end parameters for list.count()  (2005-12-01)
       http://python.org/sf/1370948  closed by  rhettinger


From trentm at ActiveState.com  Sat Dec 24 06:36:50 2005
From: trentm at ActiveState.com (Trent Mick)
Date: Fri, 23 Dec 2005 21:36:50 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
References: <20051221105132.BE7C.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org> <2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
Message-ID: <20051224053650.GA1853@ActiveState.com>

[Neal Norwitz wrote]
> I couldn't let Trent have all the fun.
> 
>     http://docs.python.org/dev/

Yah, I'd had a great time. <wink> Back to Xmas drinking.

Cheers,
Trent

-- 
Trent Mick
TrentM at ActiveState.com

From tim.peters at gmail.com  Sat Dec 24 07:44:00 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 24 Dec 2005 01:44:00 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu> <docare$tjs$1@sea.gmane.org>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org> <2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
Message-ID: <1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>

[Neal Norwitz]
> ...
> I couldn't let Trent have all the fun.
>
>     http://docs.python.org/dev/
>
> And hopefully of interest to many here:
>
>     http://docs.python.org/dev/results/

Wow!  You get no test failures!  I guess nobody tests on Windows
anymore.  I've been getting test failures for months, and just
_assumed_ this was known damage everywhere so was waiting for someone
else to fix it ;-)  (A parenthentical question:  is there a reason you
don't pass -uall to regrtest.py?)

On WinXP Pro SP2 today, passing -uall, and after fixing all the MS
compiler warnings that have crept in:

251 tests OK.
12 tests failed:
    test_builtin test_coding test_compiler test_pep263
    test_univnewlines test_urllib test_urllib2 test_urllibnet
    test_userlist test_wave test_whichdb test_zipfile
39 tests skipped:
    test__locale test_aepack test_al test_applesingle test_bsddb185
    test_cd test_cl test_commands test_crypt test_curses test_dbm
    test_dl test_fcntl test_fork1 test_gdbm test_gl test_grp
    test_hashlib_speed test_imgfile test_ioctl test_linuxaudiodev
    test_macfs test_macostools test_mhlib test_nis test_openpty
    test_ossaudiodev test_plistlib test_poll test_posix test_pty
    test_pwd test_resource test_scriptpackages test_signal
    test_sunaudiodev test_threadsignals test_timing test_xml_etree_c
1 skip unexpected on win32:
    test_xml_etree_c

Where to begin?  Let's start with the first:

C:\Code\python\PCbuild>rt -q test_builtin

C:\Code\python\PCbuild>python  -E -tt ../lib/test/regrtest.py test_builtin
test_builtin
test test_builtin failed -- errors occurred; run in verbose mode for details
1 test failed:
    test_builtin

OK, try again:

C:\Code\python\PCbuild>rt -q -v test_builtin

C:\Code\python\PCbuild>python  -E -tt ../lib/test/regrtest.py -v test_builtin
test_builtin
test_abs (test.test_builtin.BuiltinTest) ... ok
...
======================================================================
ERROR: test_compile (test.test_builtin.BuiltinTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Code\python\lib\test\test_builtin.py", line 237, in test_compile
    compile(bom + 'print 1\n', '', 'exec')
  File "<string>", line 1
    ¡É¨[©´print 1
    ^
SyntaxError: invalid syntax

======================================================================
ERROR: test_eval (test.test_builtin.BuiltinTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Code\python\lib\test\test_builtin.py", line 306, in test_eval
    self.assertEqual(eval(bom + 'a', globals, locals), 1)
  File "<string>", line 1
    ¡É¨[©´a
    ^
SyntaxError: invalid syntax

I have no idea what those are trying to test, and remember guessing
the first time I saw this that it was fallout from the AST-branch
merge.  Apparently it wasn't :-(.  Anyone have a clue on this one? 
The code up to the first failure is short:

        bom = '\xef\xbb\xbf'
        compile(bom + 'print 1\n', '', 'exec')

Curiously, that sequence doesn't blow up under the released Windows
Python 2.4.2, so somebody broke something here since then ...

From nnorwitz at gmail.com  Sat Dec 24 08:14:48 2005
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Fri, 23 Dec 2005 23:14:48 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org> <2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
Message-ID: <ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>

On 12/23/05, Tim Peters <tim.peters at gmail.com> wrote:
> >
> >     http://docs.python.org/dev/results/
>
> Wow!  You get no test failures!  I guess nobody tests on Windows
> anymore.  I've been getting test failures for months, and just

Hmmm, I thought others were running the tests on Windows too.  There
was one report on Nov 22 about running Purify on Windows 2k (subject:
ast status, memory leaks, etc).  He had problems with a stack overflow
in test_compile.  He was going to disable the test and re-run.  I
never heard back though.  Based on that info, I would guess that
test_builtin was working on Win 2k on Nov 22.

> _assumed_ this was known damage everywhere so was waiting for someone
> else to fix it ;-)  (A parenthentical question:  is there a reason you
> don't pass -uall to regrtest.py?)

It's calling make test.  I thought about calling regrtest.py instead
and doing as you suggest.  Is there a benefit to running make test?  I
know it runs with and without -O.  I guess it's only machine time, I
could run make test and regrtest.py -uall.

> On WinXP Pro SP2 today, passing -uall, and after fixing all the MS
> compiler warnings that have crept in:
>
> 251 tests OK.
> 12 tests failed:
>     test_builtin test_coding test_compiler test_pep263
>     test_univnewlines test_urllib test_urllib2 test_urllibnet
>     test_userlist test_wave test_whichdb test_zipfile
> 1 skip unexpected on win32:
>     test_xml_etree_c

Ouch!  I'm might be to blame for at least some of those. :-(

> ERROR: test_compile (test.test_builtin.BuiltinTest)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "C:\Code\python\lib\test\test_builtin.py", line 237, in test_compile
>     compile(bom + 'print 1\n', '', 'exec')
>   File "<string>", line 1
>     ¡É¨[©´print 1
>     ^
> SyntaxError: invalid syntax

> I have no idea what those are trying to test, and remember guessing
> the first time I saw this that it was fallout from the AST-branch
> merge.  Apparently it wasn't :-(.  Anyone have a clue on this one?

This test code was added a while ago by Just.  So the test code isn't new.
I changed some compile code wrt unicode that was a memory leak (r41553).
I just ran valgrind and it didn't report any problems.  So I don't
think that change broke Windows.

Do you know if the tests were broken before the AST merge (about Oct
22 I think)?

> The code up to the first failure is short:
>
>         bom = '\xef\xbb\xbf'
>         compile(bom + 'print 1\n', '', 'exec')
>
> Curiously, that sequence doesn't blow up under the released Windows
> Python 2.4.2, so somebody broke something here since then ...

There were a bunch of changes to Parser/tokenizer.c to handle error
conditions.  Those go back to Oct 1.  I don't *think* those would
cause these, but I'm not sure.

Sorry, I don't know any more.  I guess you might have to binary search
by date to try and find the problem.

n

From steve at holdenweb.com  Sat Dec 24 12:03:58 2005
From: steve at holdenweb.com (Steve Holden)
Date: Sat, 24 Dec 2005 11:03:58 +0000
Subject: [Python-Dev] status of development documentation
In-Reply-To: <ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>	<1135196162.14495.15.camel@geddy.wooz.org>	<doci50$nom$1@sea.gmane.org>
	<2m4q512vub.fsf@starship.python.net>	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>	<dogduu$o1m$1@sea.gmane.org>	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>	<17324.31929.979592.958222@montanaro.dyndns.org>	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
Message-ID: <doj9us$226$1@sea.gmane.org>

Neal Norwitz wrote:
> On 12/23/05, Tim Peters <tim.peters at gmail.com> wrote:
> 
>>>    http://docs.python.org/dev/results/
>>
>>Wow!  You get no test failures!  I guess nobody tests on Windows
>>anymore.  I've been getting test failures for months, and just
> 
> 
> Hmmm, I thought others were running the tests on Windows too.  There
> was one report on Nov 22 about running Purify on Windows 2k (subject:
> ast status, memory leaks, etc).  He had problems with a stack overflow
> in test_compile.  He was going to disable the test and re-run.  I
> never heard back though.  Based on that info, I would guess that
> test_builtin was working on Win 2k on Nov 22.
> 
> 
>>_assumed_ this was known damage everywhere so was waiting for someone
>>else to fix it ;-)  (A parenthentical question:  is there a reason you
>>don't pass -uall to regrtest.py?)
> 
[...]
I suppose this might be evidence in support of the arguments for trying 
to make Python compile on Windows under an open-source toolset.

The reason *I* don't test under Windows is because I can't build under 
Windows, so I only run Python installed from packaged installers.

Alternatively, is there any mileage in trying to either get Sourceforge 
to provide Windows machines in the compile farm, or get Microsoft to 
provide more software fee to Windows testers?

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From steve at holdenweb.com  Sat Dec 24 12:16:38 2005
From: steve at holdenweb.com (Steve Holden)
Date: Sat, 24 Dec 2005 11:16:38 +0000
Subject: [Python-Dev] status of development documentation
In-Reply-To: <doj9us$226$1@sea.gmane.org>
References: <20051221095628.BE79.JCARLSON@uci.edu>	<1135196162.14495.15.camel@geddy.wooz.org>	<doci50$nom$1@sea.gmane.org>	<2m4q512vub.fsf@starship.python.net>	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>	<dogduu$o1m$1@sea.gmane.org>	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>	<17324.31929.979592.958222@montanaro.dyndns.org>	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	<doj9us$226$1@sea.gmane.org>
Message-ID: <dojamk$3bn$1@sea.gmane.org>

Steve Holden wrote:
[...]
> Alternatively, is there any mileage in trying to either get Sourceforge 
> to provide Windows machines in the compile farm, or get Microsoft to 
> provide more software fee to Windows testers?
^fee^free^
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From skip at pobox.com  Sat Dec 24 14:12:35 2005
From: skip at pobox.com (skip@pobox.com)
Date: Sat, 24 Dec 2005 07:12:35 -0600
Subject: [Python-Dev] status of development documentation
In-Reply-To: <ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org> <2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
Message-ID: <17325.18755.863396.71450@montanaro.dyndns.org>

    Neal> I guess you might have to binary search by date to try and find
    Neal> the problem.

Probably needs to be a binary search by revision.  I believe Martin 
indicated a side effect of the conversion to subversion was that date-based
updates don't work.

Skip

From skip at pobox.com  Sat Dec 24 14:13:52 2005
From: skip at pobox.com (skip@pobox.com)
Date: Sat, 24 Dec 2005 07:13:52 -0600
Subject: [Python-Dev] status of development documentation
In-Reply-To: <doj9us$226$1@sea.gmane.org>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org> <2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	<doj9us$226$1@sea.gmane.org>
Message-ID: <17325.18832.329887.580689@montanaro.dyndns.org>


    Steve> Alternatively, is there any mileage in trying to either get
    Steve> Sourceforge to provide Windows machines in the compile farm, or
    Steve> get Microsoft to provide more software fee to Windows testers?

How about seeing if Microsoft has or will create a compile farm?

Skip

From steve at holdenweb.com  Sat Dec 24 15:12:04 2005
From: steve at holdenweb.com (Steve Holden)
Date: Sat, 24 Dec 2005 14:12:04 +0000
Subject: [Python-Dev] status of development documentation
In-Reply-To: <17325.18832.329887.580689@montanaro.dyndns.org>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org> <2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	<doj9us$226$1@sea.gmane.org>
	<17325.18832.329887.580689@montanaro.dyndns.org>
Message-ID: <43AD5734.1020903@holdenweb.com>

skip at pobox.com wrote:
>     Steve> Alternatively, is there any mileage in trying to either get
>     Steve> Sourceforge to provide Windows machines in the compile farm, or
>     Steve> get Microsoft to provide more software fee to Windows testers?
> 
> How about seeing if Microsoft has or will create a compile farm?
> 
I'll see what I can do ...

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/

From tim.peters at gmail.com  Sat Dec 24 16:19:57 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 24 Dec 2005 10:19:57 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu> <doci50$nom$1@sea.gmane.org>
	<2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
Message-ID: <1f7befae0512240719l969a61ep2eafaa616c4b840a@mail.gmail.com>

[Neal]
> Hmmm, I thought others were running the tests on Windows too.  There
> was one report on Nov 22 about running Purify on Windows 2k (subject:
> ast status, memory leaks, etc).  He had problems with a stack overflow
> in test_compile.  He was going to disable the test and re-run.  I
> never heard back though.  Based on that info, I would guess that
> test_builtin was working on Win 2k on Nov 22.

I wouldn't assume that.  My "nobody" was wrt the universe of Python
developers, not users; folks like Trent and MarkH and you and me. 
Without "normal" baseline test experience, users don't understand what
they're seeing, and so their reports can be highly misleading.  You
can trust that while I don't understand what I'm seeing here either,
at least I told the absolute truth about it and didn't hold back
critical details ;-)

That said, I was hoping to do some Python work over Thanksgiving week,
but was mortally discouraged on the Nov 19-20 weekend by all the test
failures I saw.  So I'm pretty sure (but not certain) that
test_builtin was failing then.

>> (A parenthentical question:  is there a reason you don't pass -uall to
>> regrtest.py?)

> It's calling make test.  I thought about calling regrtest.py instead
> and doing as you suggest.  Is there a benefit to running make test?

You're asking a Windows guy about make:  bad career move ;-)

> I know it runs with and without -O.  I guess it's only machine time, I
> could run make test and regrtest.py -uall.

-uall is helpful in finding bugs.  One thing in particular here is
that test_compiler runs only a tiny subset of its full test unless an
appropriate -u flag is given.

>> On WinXP Pro SP2 today, passing -uall, and after fixing all the MS
>> compiler warnings that have crept in:
>>
>> 251 tests OK.
>> 12 tests failed:
>>     test_builtin test_coding test_compiler test_pep263
>>     test_univnewlines test_urllib test_urllib2 test_urllibnet
>>     test_userlist test_wave test_whichdb test_zipfile
>> 1 skip unexpected on win32:
>>     test_xml_etree_c

> Ouch!  I'm might be to blame for at least some of those. :-(

I'm sure it's not as bad as it looks.  For example, test_coding and
(the -uall) test_compiler fail for exactly the same reason.  For
another, when a test fails on Windows, it sometimes leaves a "@test"
file or directory behind, which causes a cascade of bogus failures
later.  For example, test_userlist, test_wave, test_whichdb, and
test_zipfile all pass when run alone here.  Others probably do too.

...
>
> Do you know if the tests were broken before the AST merge (about Oct
> 22 I think)?

I don't know.  I'm getting more evidence that most (if not all) of the
failures are due to compile-time parsing screwups, so the AST merge is
a prime suspect.

Is it possible that generated parse tables (whatever) are out-of-date
on a Windows box?  There are no makefiles here, and the Windows build
has always relied on Unix-heads to check in correct generated parser
files.

>> The code up to the first failure is short:
>>
>>         bom = '\xef\xbb\xbf'
>>         compile(bom + 'print 1\n', '', 'exec')
>>
>> Curiously, that sequence doesn't blow up under the released Windows
>> Python 2.4.2, so somebody broke something here since then ...

> There were a bunch of changes to Parser/tokenizer.c to handle error
> conditions.  Those go back to Oct 1.  I don't *think* those would
> cause these, but I'm not sure.
>
> Sorry, I don't know any more.  I guess you might have to binary search
> by date to try and find the problem.

That's just silly ;-)  What I need is someone who understands what
this code is _supposed_ to do, so we can fix it; finding the checkin
that caused it isn't nearly as interesting.  Windows has an excellent
debug-build debugger and I can easily step through the code.  But I
have no idea why compiling a string starting with  '\xef\xbb\xbf'
should _not_ be a syntax error -- it looks like a syntax error to me.

And when I step thru the code, it looks like a syntax error to the
parser too.  It peels off the first character (\xef), and says "syntax
error" at that point:

Py_CompileStringFlags ->
PyParser_ASTFromString ->
PyParser_ParseStringFlagsFilename ->
parsetok ->
PyTokenizer_Get

That sets `a` to point at the start of the string, `b` to point at the
second character, and returns type==51.  Then `len` is set to 1, 
`str` is malloc'ed to hold 2 bytes, and `str` is filled in with
"\xef\x00" (the first byte of the input, as a NUL-terminated C
string).

PyParser_AddToken then calls classify(), which falls off the end of
its last loop and returns -1:  syntax error.

So how it gets there is really quite straightfoward.  The problem for
me is that I have no idea why it _should_ do something other than
that, let alone what that may be.  This needs someone who knows
something :-)

From pedronis at strakt.com  Sat Dec 24 16:34:41 2005
From: pedronis at strakt.com (Samuele Pedroni)
Date: Sat, 24 Dec 2005 16:34:41 +0100
Subject: [Python-Dev] status of development documentation
In-Reply-To: <1f7befae0512240719l969a61ep2eafaa616c4b840a@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<doci50$nom$1@sea.gmane.org>	<2m4q512vub.fsf@starship.python.net>	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>	<dogduu$o1m$1@sea.gmane.org>	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>	<17324.31929.979592.958222@montanaro.dyndns.org>	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	<1f7befae0512240719l969a61ep2eafaa616c4b840a@mail.gmail.com>
Message-ID: <43AD6A91.3050608@strakt.com>

Tim Peters wrote:
> [Neal]
> 
>>Hmmm, I thought others were running the tests on Windows too.  There
>>was one report on Nov 22 about running Purify on Windows 2k (subject:
>>ast status, memory leaks, etc).  He had problems with a stack overflow
>>in test_compile.  He was going to disable the test and re-run.  I
>>never heard back though.  Based on that info, I would guess that
>>test_builtin was working on Win 2k on Nov 22.
> 
> 
> I wouldn't assume that.  My "nobody" was wrt the universe of Python
> developers, not users; folks like Trent and MarkH and you and me. 
> Without "normal" baseline test experience, users don't understand what
> they're seeing, and so their reports can be highly misleading.  You
> can trust that while I don't understand what I'm seeing here either,
> at least I told the absolute truth about it and didn't hold back
> critical details ;-)
> 
> That said, I was hoping to do some Python work over Thanksgiving week,
> but was mortally discouraged on the Nov 19-20 weekend by all the test
> failures I saw.  So I'm pretty sure (but not certain) that
> test_builtin was failing then.
> 
> 
>>>(A parenthentical question:  is there a reason you don't pass -uall to
>>>regrtest.py?)
> 
> 
>>It's calling make test.  I thought about calling regrtest.py instead
>>and doing as you suggest.  Is there a benefit to running make test?
> 
> 
> You're asking a Windows guy about make:  bad career move ;-)
> 
> 
>>I know it runs with and without -O.  I guess it's only machine time, I
>>could run make test and regrtest.py -uall.
> 
> 
> -uall is helpful in finding bugs.  One thing in particular here is
> that test_compiler runs only a tiny subset of its full test unless an
> appropriate -u flag is given.
> 
> 
>>>On WinXP Pro SP2 today, passing -uall, and after fixing all the MS
>>>compiler warnings that have crept in:
>>>
>>>251 tests OK.
>>>12 tests failed:
>>>    test_builtin test_coding test_compiler test_pep263
>>>    test_univnewlines test_urllib test_urllib2 test_urllibnet
>>>    test_userlist test_wave test_whichdb test_zipfile
>>>1 skip unexpected on win32:
>>>    test_xml_etree_c
> 
> 
>>Ouch!  I'm might be to blame for at least some of those. :-(
> 
> 
> I'm sure it's not as bad as it looks.  For example, test_coding and
> (the -uall) test_compiler fail for exactly the same reason.  For
> another, when a test fails on Windows, it sometimes leaves a "@test"
> file or directory behind, which causes a cascade of bogus failures
> later.  For example, test_userlist, test_wave, test_whichdb, and
> test_zipfile all pass when run alone here.  Others probably do too.
> 
> ...
> 
>>Do you know if the tests were broken before the AST merge (about Oct
>>22 I think)?
> 
> 
> I don't know.  I'm getting more evidence that most (if not all) of the
> failures are due to compile-time parsing screwups, so the AST merge is
> a prime suspect.
> 
> Is it possible that generated parse tables (whatever) are out-of-date
> on a Windows box?  There are no makefiles here, and the Windows build
> has always relied on Unix-heads to check in correct generated parser
> files.
> 
> 
>>>The code up to the first failure is short:
>>>
>>>        bom = '\xef\xbb\xbf'
>>>        compile(bom + 'print 1\n', '', 'exec')
>>>
>>>Curiously, that sequence doesn't blow up under the released Windows
>>>Python 2.4.2, so somebody broke something here since then ...
> 
> 
>>There were a bunch of changes to Parser/tokenizer.c to handle error
>>conditions.  Those go back to Oct 1.  I don't *think* those would
>>cause these, but I'm not sure.
>>
>>Sorry, I don't know any more.  I guess you might have to binary search
>>by date to try and find the problem.
> 
> 
> That's just silly ;-)  What I need is someone who understands what
> this code is _supposed_ to do, so we can fix it; finding the checkin
> that caused it isn't nearly as interesting.  Windows has an excellent
> debug-build debugger and I can easily step through the code.  But I
> have no idea why compiling a string starting with  '\xef\xbb\xbf'
> should _not_ be a syntax error -- it looks like a syntax error to me.
> 
> And when I step thru the code, it looks like a syntax error to the
> parser too.  It peels off the first character (\xef), and says "syntax
> error" at that point:
> 
> Py_CompileStringFlags ->
> PyParser_ASTFromString ->
> PyParser_ParseStringFlagsFilename ->
> parsetok ->
> PyTokenizer_Get
> 
> That sets `a` to point at the start of the string, `b` to point at the
> second character, and returns type==51.  Then `len` is set to 1, 
> `str` is malloc'ed to hold 2 bytes, and `str` is filled in with
> "\xef\x00" (the first byte of the input, as a NUL-terminated C
> string).
> 
> PyParser_AddToken then calls classify(), which falls off the end of
> its last loop and returns -1:  syntax error.
> 
> So how it gets there is really quite straightfoward.  The problem for
> me is that I have no idea why it _should_ do something other than
> that, let alone what that may be.

PEP263:

"""
     To aid with platforms such as Windows, which add Unicode BOM marks
     to the beginning of Unicode files, the UTF-8 signature
     '\xef\xbb\xbf' will be interpreted as 'utf-8' encoding as well
     (even if no magic encoding comment is given).
"""







> This needs someone who knows
> something :-)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/pedronis%40strakt.com


From facundobatista at gmail.com  Sat Dec 24 16:33:10 2005
From: facundobatista at gmail.com (Facundo Batista)
Date: Sat, 24 Dec 2005 12:33:10 -0300
Subject: [Python-Dev] documentation comments
In-Reply-To: <200512221858.49521.fdrake@acm.org>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<20051222142206.GB15340@rogue.amk.ca>
	<ee2a432c0512221044o71f01392yeb4c9b5adbac3318@mail.gmail.com>
	<200512221858.49521.fdrake@acm.org>
Message-ID: <e04bdf310512240733u5413a228q@mail.gmail.com>

2005/12/22, Fred L. Drake, Jr. <fdrake at acm.org>:

> In general, my worry is less with dealing with spam than with ensuring
> integration of content enhancements before release candidates go out.

Well, I think that the most important part of annotable documentation
is just cuantitative feedback, not cualitative.

I mean, I don't care if a doc page has a comment and it doesn't get
included in the next release (if you want it included, submit it in a
more formal way).

But, if there's a page with 25 notes, that implies that there's a
problem with that doc page, and something (integrating some of the
notes or not) must be done with that text before the next release.

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From tim.peters at gmail.com  Sat Dec 24 17:08:50 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 24 Dec 2005 11:08:50 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <43AD6A91.3050608@strakt.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	<1f7befae0512240719l969a61ep2eafaa616c4b840a@mail.gmail.com>
	<43AD6A91.3050608@strakt.com>
Message-ID: <1f7befae0512240808l5e598d6fvcf16888f767ec776@mail.gmail.com>

[Samuele Pedroni]
> PEP263:
>
> """
>      To aid with platforms such as Windows, which add Unicode BOM marks
>      to the beginning of Unicode files, the UTF-8 signature
>      '\xef\xbb\xbf' will be interpreted as 'utf-8' encoding as well
>      (even if no magic encoding comment is given).
> """

So test_pep263 is almost certainly failing for a related reason (and
I'll skip noting the irony that this was for the benefit of Windows
;-)).  I didn't step through any code under the debugger that looked
like it was _trying_ to special-case '\xef\xbb\xbf', so this still
needs someone who knows which path(s) the code is supposed to take. 
I'll help, but I don't have time to drive this.

test_pep263
test test_pep263 failed -- Traceback (most recent call last):
  File "C:\Code\python\lib\test\test_pep263.py", line 12, in test_pep263
    '\xd0\x9f\xd0\xb8\xd1\x82\xd0\xbe\xd0\xbd'
AssertionError: '\xc3\xb0\xc3\x89\xc3\x94\xc3\x8f\xc3\x8e' !=
'\xd0\x9f\xd0\xb8\xd1\x82\xd0\xbe\xd0\xbd'

From nnorwitz at gmail.com  Sat Dec 24 17:40:10 2005
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sat, 24 Dec 2005 08:40:10 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <1f7befae0512240719l969a61ep2eafaa616c4b840a@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	<1f7befae0512240719l969a61ep2eafaa616c4b840a@mail.gmail.com>
Message-ID: <ee2a432c0512240840p1e37a4d9n7c2379096514ccc8@mail.gmail.com>

On 12/24/05, Tim Peters <tim.peters at gmail.com> wrote:

> You're asking a Windows guy about make:  bad career move ;-)

:-)

> -uall is helpful in finding bugs.  One thing in particular here is
> that test_compiler runs only a tiny subset of its full test unless an
> appropriate -u flag is given.

Not to mention networking tests.

> >> The code up to the first failure is short:
> >>
> >>         bom = '\xef\xbb\xbf'
> >>         compile(bom + 'print 1\n', '', 'exec')

> That sets `a` to point at the start of the string, `b` to point at the
> second character, and returns type==51.  Then `len` is set to 1,
> `str` is malloc'ed to hold 2 bytes, and `str` is filled in with
> "\xef\x00" (the first byte of the input, as a NUL-terminated C
> string).

This gives me an idea (ie, wild ass guess).  r39680 checked in on
2005-10-06 to speed up unicode charmap decoding.  Dunno if it's likely
or not.  Gotta run, I'm headed east.  Good luck.

Merry Christmas and Happy New Year everyone!

n

From walter at livinglogic.de  Sat Dec 24 17:59:04 2005
From: walter at livinglogic.de (=?iso-8859-1?Q?Walter_D=F6rwald?=)
Date: Sat, 24 Dec 2005 17:59:04 +0100 (CET)
Subject: [Python-Dev] status of development documentation
In-Reply-To: <ee2a432c0512240840p1e37a4d9n7c2379096514ccc8@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	<1f7befae0512240719l969a61ep2eafaa616c4b840a@mail.gmail.com>
	<ee2a432c0512240840p1e37a4d9n7c2379096514ccc8@mail.gmail.com>
Message-ID: <61099.85.74.61.45.1135443544.squirrel@isar.livinglogic.de>

Neal Norwitz wrote:

> On 12/24/05, Tim Peters <tim.peters at gmail.com> wrote:
>
> [...]
>> >> The code up to the first failure is short:
>> >>
>> >>         bom = '\xef\xbb\xbf'
>> >>         compile(bom + 'print 1\n', '', 'exec')
>
>> That sets `a` to point at the start of the string, `b` to point at the second character, and returns type==51.  Then `len`
>> is set to 1, `str` is malloc'ed to hold 2 bytes, and `str` is filled in with
>> "\xef\x00" (the first byte of the input, as a NUL-terminated C
>> string).
>
> This gives me an idea (ie, wild ass guess).  r39680 checked in on
> 2005-10-06 to speed up unicode charmap decoding.  Dunno if it's likely or not.  Gotta run, I'm headed east.  Good luck.

Other candidates might be the patches to Parser/tokenizer.c. (Unfortunately I don't have a machine to test this right now).

Bye,
   Walter D?rwald




From facundobatista at gmail.com  Sat Dec 24 18:31:19 2005
From: facundobatista at gmail.com (Facundo Batista)
Date: Sat, 24 Dec 2005 14:31:19 -0300
Subject: [Python-Dev] NotImplemented reaching top-level
Message-ID: <e04bdf310512240931t1049346n@mail.gmail.com>

Folks,

There's a bug about number coercion about Decimal
(http://www.python.org/sf/1355842).

The bug appeared after some changes Raymond and I did a few months
ago, solving something else (started to return NotImplemented instead
of raising TypeError, to better work with custom objects that
implements type coercion from Decimal).

The point is that I'm really astonished about the following behaviour,
and don't know where to start searching:

>>> d
Decimal("1")   # using decimal.py rev. 39328 from svn
>>> d + 1.2
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: unsupported operand type(s) for +:
'Decimal' and
'float'
>>> d += 1.2
>>> d
NotImplemented
>>>

Thanks for any tip.

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From tim.peters at gmail.com  Sat Dec 24 22:54:05 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 24 Dec 2005 16:54:05 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <ee2a432c0512240840p1e37a4d9n7c2379096514ccc8@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	<1f7befae0512240719l969a61ep2eafaa616c4b840a@mail.gmail.com>
	<ee2a432c0512240840p1e37a4d9n7c2379096514ccc8@mail.gmail.com>
Message-ID: <1f7befae0512241354o1e5a13fen5ea6bec7a17a17ab@mail.gmail.com>

[Neal Norwitz]
> This gives me an idea (ie, wild ass guess).  r39680 checked in on
> 2005-10-06 to speed up unicode charmap decoding.  Dunno if it's likely
> or not.  Gotta run, I'm headed east.  Good luck.

Nope, it's not calling any decoding functions at all on Windows, let
alone optimized ones ;-)

Who knows what the code is supposed to do?  (Sorry, but I don't want
to chase random patch guesses -- it's a wasteful approach to this
one.)

I'm getting a strong suspicion that I'm the only developer to _try_
building the trunk on WinXP since the AST merge was done, and that
something obscure is fundamentally broken with it on this box.  For
example, in tokenizer.c, these functions don't even exist on Windows
today (because an enclosing #ifdef says not to compile them):

error_ret
new_string
get_normal_name
get_coding_spec
check_coding_spec
check_bom
fp_readl
fp_setreadl
fp_getc
fp_ungetc
decoding_fgets
decoding_feof
buf_getc
buf_ungetc
buf_setreadl
translate_into_utf8
decode_str

OK, that's not quite true.  "Degenerate" forms of three of those
functions exist on Windows:

static char *
decoding_fgets(char *s, int size, struct tok_state *tok)
{
	return fgets(s, size, tok->fp);
}

static int
decoding_feof(struct tok_state *tok)
{
	return feof(tok->fp);
}

static const char *
decode_str(const char *str, struct tok_state *tok)
{
	return str;
}

In the simple failing test, that degenerate decode_str() is getting
called.  If the "fancy" decode_str() were being used instead, that one
_does_ call check_bom().  Why do we have two versions of these
functions?  Which set is supposed to be in use now?  What's the
meaning of "#ifdef PGEN" today?  Should it be true or false?

> Merry Christmas and Happy New Year everyone!

Same to you, Neal!

From tim.peters at gmail.com  Sun Dec 25 02:43:04 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 24 Dec 2005 20:43:04 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <1f7befae0512241354o1e5a13fen5ea6bec7a17a17ab@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu> <dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	<1f7befae0512240719l969a61ep2eafaa616c4b840a@mail.gmail.com>
	<ee2a432c0512240840p1e37a4d9n7c2379096514ccc8@mail.gmail.com>
	<1f7befae0512241354o1e5a13fen5ea6bec7a17a17ab@mail.gmail.com>
Message-ID: <1f7befae0512241743n6952a2fbncb5d72b5f7ffa809@mail.gmail.com>

FWIW, test_builtin and test_pep263 both passed on WinXP in rev 39757. 
That's the last revision before the AST branch was merged.

I can't build rev 39758 on WinXP (VC complains that pythoncore.vcproj
can't be loaded -- looks like it got checked in with unresolved SVN
conflict markers -- which isn't easy to do under SVN ;-( ), so don't
know about that.

The first revision at which Python built again was 39791 (23 Oct), and
test_builtin and test_pep263 both fail under that the same way they
fail today.

I'm darned near certain that we're not using the _intended_ parsing
code on Windows now -- PGEN is still #define'd when the "final"
parsing code is compiled into python25.dll.  Don't know how to fix
that (I don't understand it).

From bcannon at gmail.com  Sun Dec 25 03:35:38 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Sat, 24 Dec 2005 18:35:38 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <1f7befae0512241743n6952a2fbncb5d72b5f7ffa809@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	<1f7befae0512240719l969a61ep2eafaa616c4b840a@mail.gmail.com>
	<ee2a432c0512240840p1e37a4d9n7c2379096514ccc8@mail.gmail.com>
	<1f7befae0512241354o1e5a13fen5ea6bec7a17a17ab@mail.gmail.com>
	<1f7befae0512241743n6952a2fbncb5d72b5f7ffa809@mail.gmail.com>
Message-ID: <bbaeab100512241835t32ba6a0boae45f92709edf960@mail.gmail.com>

On 12/24/05, Tim Peters <tim.peters at gmail.com> wrote:
> FWIW, test_builtin and test_pep263 both passed on WinXP in rev 39757.
> That's the last revision before the AST branch was merged.
>
> I can't build rev 39758 on WinXP (VC complains that pythoncore.vcproj
> can't be loaded -- looks like it got checked in with unresolved SVN
> conflict markers -- which isn't easy to do under SVN ;-( ), so don't
> know about that.
>
> The first revision at which Python built again was 39791 (23 Oct), and
> test_builtin and test_pep263 both fail under that the same way they
> fail today.
>

Both syntax errors, right?  My mind is partially gone thanks to being
on vacation so following this thread has been abnormally hard.  =)

Since it is a syntax error there won't be any bytecode to compare against.

> I'm darned near certain that we're not using the _intended_ parsing
> code on Windows now -- PGEN is still #define'd when the "final"
> parsing code is compiled into python25.dll.  Don't know how to fix
> that (I don't understand it).

But the AST branch didn't touch the parser (unless you are grouping
ast.c and compile.c under the "parser" umbrella just to throw me off 
=).

What can I do to help?  Do you need me to step through something?  Do
you need to know how gcc is preprocessing some file?

-Brett

From tim.peters at gmail.com  Sun Dec 25 05:43:08 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 24 Dec 2005 23:43:08 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <bbaeab100512241835t32ba6a0boae45f92709edf960@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	<1f7befae0512240719l969a61ep2eafaa616c4b840a@mail.gmail.com>
	<ee2a432c0512240840p1e37a4d9n7c2379096514ccc8@mail.gmail.com>
	<1f7befae0512241354o1e5a13fen5ea6bec7a17a17ab@mail.gmail.com>
	<1f7befae0512241743n6952a2fbncb5d72b5f7ffa809@mail.gmail.com>
	<bbaeab100512241835t32ba6a0boae45f92709edf960@mail.gmail.com>
Message-ID: <1f7befae0512242043j5644b80fgede35d239b203489@mail.gmail.com>

[Tim]
>> FWIW, test_builtin and test_pep263 both passed on WinXP in rev 39757.
>> That's the last revision before the AST branch was merged.
>>
>> I can't build rev 39758 on WinXP (VC complains that pythoncore.vcproj
>> can't be loaded -- looks like it got checked in with unresolved SVN
>> conflict markers -- which isn't easy to do under SVN ;-( ), so don't
>> know about that.
>>
>> The first revision at which Python built again was 39791 (23 Oct), and
>> test_builtin and test_pep263 both fail under that the same way they
>> fail today.

[Brett]
> Both syntax errors, right?

In test_builtin, yes, two syntax errors.  test_pep263 is different:

test test_pep263 failed -- Traceback (most recent call last):
  File "C:\Code\python\lib\test\test_pep263.py", line 12, in test_pep263
    '\xd0\x9f\xd0\xb8\xd1\x82\xd0\xbe\xd0\xbd'
AssertionError:
'\xc3\xb0\xc3\x89\xc3\x94\xc3\x8f\xc3\x8e' !=
'\xd0\x9f\xd0\xb8\xd1\x82\xd0\xbe\xd0\xbd'

That's not a syntax error, it's a wrong result.  There are other
parsing-related test failures, but those are the only two I've written
up so far (partly because I expect they all have the same underlying
cause, and partly because nobody so far seems to understand the code
well enough to explain why the first one works on any platform ;-)).

> My mind is partially gone thanks to being on vacation so following this thread
> has been abnormally hard.  =)
>
> Since it is a syntax error there won't be any bytecode to compare against.

Shouldn't be needed.  The snippet:

    bom = '\xef\xbb\xbf'
    compile(bom + 'print 1\n', '', 'exec')

treats the `bom` prefix like any other sequence of illegal characters.
 That's why it raises SyntaxError:

    It peels off the first character (\xef), and says "syntax
    error" at that point:

    Py_CompileStringFlags ->
    PyParser_ASTFromString ->
    PyParser_ParseStringFlagsFilename ->
    parsetok ->
    PyTokenizer_Get

    That sets `a` to point at the start of the string, `b` to point at the
    second character, and returns type==51.  Then `len` is set to 1,
    `str` is malloc'ed to hold 2 bytes, and `str` is filled in with
    "\xef\x00" (the first byte of the input, as a NUL-terminated C
    string).

    PyParser_AddToken then calls classify(), which falls off the end of
    its last loop and returns -1:  syntax error.

and later:

    I'm getting a strong suspicion that I'm the only developer to _try_
    building the trunk on WinXP since the AST merge was done, and that
    something obscure is fundamentally broken with it on this box.  For
    example, in tokenizer.c, these functions don't even exist on Windows
    today (because an enclosing #ifdef says not to compile them):

    error_ret
    new_string
    get_normal_name
    get_coding_spec
    check_coding_spec
    check_bom
    fp_readl
    fp_setreadl
    fp_getc
    fp_ungetc
    decoding_fgets
    decoding_feof
    buf_getc
    buf_ungetc
    buf_setreadl
    translate_into_utf8
    decode_str

    OK, that's not quite true.  "Degenerate" forms of three of those
    functions exist on Windows:

    static char *
    decoding_fgets(char *s, int size, struct tok_state *tok)
    {
           return fgets(s, size, tok->fp);
    }

    static int
    decoding_feof(struct tok_state *tok)
    {
           return feof(tok->fp);
    }

    static const char *
    decode_str(const char *str, struct tok_state *tok)
    {
          return str;
    }

    In the simple failing test, that degenerate decode_str() is getting
    called.  If the "fancy" decode_str() were being used instead, that one
    _does_ call check_bom().  Why do we have two versions of these
    functions?  Which set is supposed to be in use now?  What's the
    meaning of "#ifdef PGEN" today?  Should it be true or false?

>> I'm darned near certain that we're not using the _intended_ parsing
>> code on Windows now -- PGEN is still #define'd when the "final"
>> parsing code is compiled into python25.dll.  Don't know how to fix
>> that (I don't understand it).

> But the AST branch didn't touch the parser (unless you are grouping
> ast.c and compile.c under the "parser" umbrella just to throw me off
> =).

Possibly.  See above for unanswered questions about tokenizer.c, which
appears to be the whole problem wrt test_builtin.  Python couldn't be
built under VC7.1 on Windows after the AST merge.  However that got
repaired left parsing/tokenizing broken on Windows wrt (at least) some
encoding gimmicks.  Since the tests passed immediately before the AST
merge, and failed the first time Python could be built again after
that merge, it's the only natural candidate for finger-wagging.

> What can I do to help?

I don't know.  Enjoying Christmas couldn't hurt :-)  What this needs
is someone who understands how

    bom = '\xef\xbb\xbf'
    compile(bom + 'print 1\n', '', 'exec')

is supposed to work at the front-end level.

>  Do you need me to step through something?

Why doesn't the little code snippet above fail anywhere else? 
"Should" the degenerate decode_str() be getting called during it -- or
should the other decode_str() be getting called?  If the latter, what
got broke on Windows during the merge so that the wrong one is getting
called now?

> Do you need to know how gcc is preprocessing some file?

No, I just need to know how to fix Python on Windows ;-)

From bcannon at gmail.com  Sun Dec 25 07:29:36 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Sat, 24 Dec 2005 22:29:36 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <1f7befae0512242043j5644b80fgede35d239b203489@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	<1f7befae0512240719l969a61ep2eafaa616c4b840a@mail.gmail.com>
	<ee2a432c0512240840p1e37a4d9n7c2379096514ccc8@mail.gmail.com>
	<1f7befae0512241354o1e5a13fen5ea6bec7a17a17ab@mail.gmail.com>
	<1f7befae0512241743n6952a2fbncb5d72b5f7ffa809@mail.gmail.com>
	<bbaeab100512241835t32ba6a0boae45f92709edf960@mail.gmail.com>
	<1f7befae0512242043j5644b80fgede35d239b203489@mail.gmail.com>
Message-ID: <bbaeab100512242229j6ff54d42i7854389c40115a3f@mail.gmail.com>

On 12/24/05, Tim Peters <tim.peters at gmail.com> wrote:
> [Tim]
> >> FWIW, test_builtin and test_pep263 both passed on WinXP in rev 39757.
> >> That's the last revision before the AST branch was merged.
> >>
> >> I can't build rev 39758 on WinXP (VC complains that pythoncore.vcproj
> >> can't be loaded -- looks like it got checked in with unresolved SVN
> >> conflict markers -- which isn't easy to do under SVN ;-( ), so don't
> >> know about that.
> >>
> >> The first revision at which Python built again was 39791 (23 Oct), and
> >> test_builtin and test_pep263 both fail under that the same way they
> >> fail today.
>
> [Brett]
> > Both syntax errors, right?
>
> In test_builtin, yes, two syntax errors.  test_pep263 is different:
>
> test test_pep263 failed -- Traceback (most recent call last):
>   File "C:\Code\python\lib\test\test_pep263.py", line 12, in test_pep263
>     '\xd0\x9f\xd0\xb8\xd1\x82\xd0\xbe\xd0\xbd'
> AssertionError:
> '\xc3\xb0\xc3\x89\xc3\x94\xc3\x8f\xc3\x8e' !=
> '\xd0\x9f\xd0\xb8\xd1\x82\xd0\xbe\xd0\xbd'
>
> That's not a syntax error, it's a wrong result.  There are other
> parsing-related test failures, but those are the only two I've written
> up so far (partly because I expect they all have the same underlying
> cause, and partly because nobody so far seems to understand the code
> well enough to explain why the first one works on any platform ;-)).
>
> > My mind is partially gone thanks to being on vacation so following this thread
> > has been abnormally hard.  =)
> >
> > Since it is a syntax error there won't be any bytecode to compare against.
>
> Shouldn't be needed.  The snippet:
>
>     bom = '\xef\xbb\xbf'
>     compile(bom + 'print 1\n', '', 'exec')
>
> treats the `bom` prefix like any other sequence of illegal characters.
>  That's why it raises SyntaxError:
>
>     It peels off the first character (\xef), and says "syntax
>     error" at that point:
>
>     Py_CompileStringFlags ->
>     PyParser_ASTFromString ->
>     PyParser_ParseStringFlagsFilename ->
>     parsetok ->
>     PyTokenizer_Get
>
>     That sets `a` to point at the start of the string, `b` to point at the
>     second character, and returns type==51.  Then `len` is set to 1,
>     `str` is malloc'ed to hold 2 bytes, and `str` is filled in with
>     "\xef\x00" (the first byte of the input, as a NUL-terminated C
>     string).
>
>     PyParser_AddToken then calls classify(), which falls off the end of
>     its last loop and returns -1:  syntax error.
>
> and later:
>
>     I'm getting a strong suspicion that I'm the only developer to _try_
>     building the trunk on WinXP since the AST merge was done, and that
>     something obscure is fundamentally broken with it on this box.  For
>     example, in tokenizer.c, these functions don't even exist on Windows
>     today (because an enclosing #ifdef says not to compile them):
>
>     error_ret
>     new_string
>     get_normal_name
>     get_coding_spec
>     check_coding_spec
>     check_bom
>     fp_readl
>     fp_setreadl
>     fp_getc
>     fp_ungetc
>     decoding_fgets
>     decoding_feof
>     buf_getc
>     buf_ungetc
>     buf_setreadl
>     translate_into_utf8
>     decode_str
>
>     OK, that's not quite true.  "Degenerate" forms of three of those
>     functions exist on Windows:
>
>     static char *
>     decoding_fgets(char *s, int size, struct tok_state *tok)
>     {
>            return fgets(s, size, tok->fp);
>     }
>
>     static int
>     decoding_feof(struct tok_state *tok)
>     {
>            return feof(tok->fp);
>     }
>
>     static const char *
>     decode_str(const char *str, struct tok_state *tok)
>     {
>           return str;
>     }
>
>     In the simple failing test, that degenerate decode_str() is getting
>     called.  If the "fancy" decode_str() were being used instead, that one
>     _does_ call check_bom().  Why do we have two versions of these
>     functions?  Which set is supposed to be in use now?  What's the
>     meaning of "#ifdef PGEN" today?  Should it be true or false?
>

Looking at the logs for tokenizer.c, tokenizer.h, and
tokenizer_pgen.c, it looks like this stuff has not been heavily
touched since Martin did stuff for PEP 263.

> >> I'm darned near certain that we're not using the _intended_ parsing
> >> code on Windows now -- PGEN is still #define'd when the "final"
> >> parsing code is compiled into python25.dll.  Don't know how to fix
> >> that (I don't understand it).
>
> > But the AST branch didn't touch the parser (unless you are grouping
> > ast.c and compile.c under the "parser" umbrella just to throw me off
> > =).
>
> Possibly.  See above for unanswered questions about tokenizer.c, which
> appears to be the whole problem wrt test_builtin.  Python couldn't be
> built under VC7.1 on Windows after the AST merge.  However that got
> repaired left parsing/tokenizing broken on Windows wrt (at least) some
> encoding gimmicks.  Since the tests passed immediately before the AST
> merge, and failed the first time Python could be built again after
> that merge, it's the only natural candidate for finger-wagging.
>

Did it lead to tokenizer_pgen.c to suddenly be used for the build
instead of tokenizer.c?  The former seems to be the only place where
PGEN is defined.

> > What can I do to help?
>
> I don't know.  Enjoying Christmas couldn't hurt :-)  What this needs
> is someone who understands how
>
>     bom = '\xef\xbb\xbf'
>     compile(bom + 'print 1\n', '', 'exec')
>
> is supposed to work at the front-end level.
>

Hopefully Martin will have some inkling since he committed the phase 1
stuff for PEP 263.

> >  Do you need me to step through something?
>
> Why doesn't the little code snippet above fail anywhere else?
> "Should" the degenerate decode_str() be getting called during it -- or
> should the other decode_str() be getting called?  If the latter, what
> got broke on Windows during the merge so that the wrong one is getting
> called now?
>



> > Do you need to know how gcc is preprocessing some file?
>
> No, I just need to know how to fix Python on Windows ;-)

=)

-Brett

From arigo at tunes.org  Sun Dec 25 12:35:50 2005
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 25 Dec 2005 12:35:50 +0100
Subject: [Python-Dev] NotImplemented reaching top-level
In-Reply-To: <e04bdf310512240931t1049346n@mail.gmail.com>
References: <e04bdf310512240931t1049346n@mail.gmail.com>
Message-ID: <20051225113550.GA28497@code1.codespeak.net>

Hi Facundo,

On Sat, Dec 24, 2005 at 02:31:19PM -0300, Facundo Batista wrote:
> >>> d += 1.2
> >>> d
> NotImplemented

The situation appears to be a mess.  Some combinations of specific
operators fail to convert NotImplemented to a TypeError, depending on
old- or new-style-class-ness, although this is clearly a bug (e.g. in an
example like yours but using -= instead of +=, we get the correct
TypeError.)

Obviously, we need to write some comprehensive tests about this.  But
now I just found out that the old, still-pending SF bug #847024 about
A()*5 in new-style classes hasn't been given any attention; my theory is
that nobody fully understands the convoluted code paths of abstract.c
any more :-(


A bientot,

Armin

From reinhold-birkenfeld-nospam at wolke7.net  Sun Dec 25 12:37:53 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Sun, 25 Dec 2005 12:37:53 +0100
Subject: [Python-Dev] NotImplemented reaching top-level
In-Reply-To: <20051225113550.GA28497@code1.codespeak.net>
References: <e04bdf310512240931t1049346n@mail.gmail.com>
	<20051225113550.GA28497@code1.codespeak.net>
Message-ID: <dom0ah$f29$1@sea.gmane.org>

Armin Rigo wrote:
> Hi Facundo,
> 
> On Sat, Dec 24, 2005 at 02:31:19PM -0300, Facundo Batista wrote:
>> >>> d += 1.2
>> >>> d
>> NotImplemented
> 
> The situation appears to be a mess.  Some combinations of specific
> operators fail to convert NotImplemented to a TypeError, depending on
> old- or new-style-class-ness, although this is clearly a bug (e.g. in an
> example like yours but using -= instead of +=, we get the correct
> TypeError.)
> 
> Obviously, we need to write some comprehensive tests about this.  But
> now I just found out that the old, still-pending SF bug #847024 about
> A()*5 in new-style classes hasn't been given any attention; my theory is
> that nobody fully understands the convoluted code paths of abstract.c
> any more :-(

Time for a rewrite?

Reinhold

-- 
Mail address is perfectly valid!


From tismer at stackless.com  Sun Dec 25 13:56:31 2005
From: tismer at stackless.com (Christian Tismer)
Date: Sun, 25 Dec 2005 12:56:31 +0000
Subject: [Python-Dev] deque alternative (was: Linked lists)
In-Reply-To: <ca471dc20512141237i4f09098cudb951fbe70623085@mail.gmail.com>
References: <8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>
	<ca471dc20512141237i4f09098cudb951fbe70623085@mail.gmail.com>
Message-ID: <43AE96FF.2010303@stackless.com>

Guido van Rossum wrote:

> Python's philosophy about (built-in) data types, inherited from ABC,
> is to offer a few powerful clearly distinct choices rather than lots
> of alternatives with overlapping usages. This reduces the time it
> takes to choose a data type and reduces the risk of picking the wrong
> type. (You seem to be indicating that this is indeed what's happening
> to you in Lisp. :-)

Speaking about the new dequeue type, I have the impression that
this reasoning applies there a bit, too?
Dequeues of course have applications, and I saw them used,
especially by newcomers to the language. So I like their
functionality.

But actually, I'm not so convinced if we need to introduce
a new datatype. How about an extension to normal lists
that does a little tracking of use-cases and decides about
changing its implementation upon the presence of inserts/deletes
at the lower end of the list?

I'm btw. not sure if this raises an implicit/explicitiness issue.
My question is if it is necessary to put the burden about explicitly
choosing a specialized two-ended list on the programmer, or if it
is simple enough to make lists just do the right thing, if lists
are used in a dequeue-style manner. Or is this maybe too much magic
happening?

merry christmas -- chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From ncoghlan at gmail.com  Sun Dec 25 17:22:50 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 26 Dec 2005 02:22:50 +1000
Subject: [Python-Dev] status of development documentation
In-Reply-To: <bbaeab100512231848u3264e4dbseedc0d54d6a41e56@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<docare$tjs$1@sea.gmane.org>	<1135196162.14495.15.camel@geddy.wooz.org>	<doci50$nom$1@sea.gmane.org>
	<2m4q512vub.fsf@starship.python.net>	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>	<dogduu$o1m$1@sea.gmane.org>	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>	<17324.31929.979592.958222@montanaro.dyndns.org>	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<bbaeab100512231848u3264e4dbseedc0d54d6a41e56@mail.gmail.com>
Message-ID: <43AEC75A.8010201@gmail.com>

Brett Cannon wrote:
> On 12/23/05, Neal Norwitz <nnorwitz at gmail.com> wrote:
>> On 12/23/05, skip at pobox.com <skip at pobox.com> wrote:
>>> So for at least the time being they go up nightly
>>> (http://www.trentm.com/python).  I don't know what Trent did to make that
>>> happen, but he did it fairly quickly.  I doubt it would be hard to replicate
>>> on the docs server.
>> I couldn't let Trent have all the fun.
>>
>>     http://docs.python.org/dev/
>>
> 
> Cool!  Thanks to Trent for sparking Neal, and thanks to Neal for
> feeling the fire under our arses for getting this done.

Indeed - thanks folks.

>> And hopefully of interest to many here:
>>
>>     http://docs.python.org/dev/results/
>>
>> These are the results of svn update, configure, build, test, install
>> and the doc run.
>> Run on the PSFs box and updated every 12 hours.  I currently have it
>> send mail to me if there are any test failures.  I will probably
>> update that to python-checkins or maybe even python-dev depending on
>> what people think.  I'm not likely to be around much for the rest of
>> the year, so I don't want to turn it on just yet.
>>
> 
> python-checkins seems the most reasonable.  But I would have no
> problem with it going to python-dev.

If the message gets sent to python-checkins, then it should be fairly easy to 
identify candidate culprits (especially if the script is designed to send the 
email anytime the update actually retrieves new files from the repository, 
even if the tests all pass).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From arigo at tunes.org  Sun Dec 25 17:23:25 2005
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 25 Dec 2005 17:23:25 +0100
Subject: [Python-Dev] NotImplemented reaching top-level
In-Reply-To: <dom0ah$f29$1@sea.gmane.org>
References: <e04bdf310512240931t1049346n@mail.gmail.com>
	<20051225113550.GA28497@code1.codespeak.net>
	<dom0ah$f29$1@sea.gmane.org>
Message-ID: <20051225162325.GA32552@code1.codespeak.net>

Hi Reinhold,

On Sun, Dec 25, 2005 at 12:37:53PM +0100, Reinhold Birkenfeld wrote:
> > that nobody fully understands the convoluted code paths of abstract.c
> > any more :-(
> 
> Time for a rewrite?

Of course, speaking of a rewrite, PyPy does the "right thing" in these
two areas.  Won't happen to CPython, though.  There are too much
backward-compatibility issues with the PyTypeObject structure; I think
we're doomed with patching the bugs as they show up.

Looking up in the language reference, I see no mention of NotImplemented
in the page about __add__, __radd__, etc.  I guess it's a documentation
bug as well, isn't it?  The current code base tries to implement the
following behavior: Returning NotImplemented from any of the binary
special methods (__xxx__, __rxxx__, __ixxx__) makes Python proceed as if
the method was not defined in the first place.

If we agree on this, I could propose a doc fix, a test, and appropriate
bug fixes.


A bientot,

Armin

From tim.peters at gmail.com  Sun Dec 25 17:29:13 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 25 Dec 2005 11:29:13 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <bbaeab100512242229j6ff54d42i7854389c40115a3f@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	<1f7befae0512240719l969a61ep2eafaa616c4b840a@mail.gmail.com>
	<ee2a432c0512240840p1e37a4d9n7c2379096514ccc8@mail.gmail.com>
	<1f7befae0512241354o1e5a13fen5ea6bec7a17a17ab@mail.gmail.com>
	<1f7befae0512241743n6952a2fbncb5d72b5f7ffa809@mail.gmail.com>
	<bbaeab100512241835t32ba6a0boae45f92709edf960@mail.gmail.com>
	<1f7befae0512242043j5644b80fgede35d239b203489@mail.gmail.com>
	<bbaeab100512242229j6ff54d42i7854389c40115a3f@mail.gmail.com>
Message-ID: <1f7befae0512250829i277f50e1j6d10cc65465b6a9b@mail.gmail.com>

Santa sent me a bad-mood elf overnight, apparently just to motivate me ;-)

Since it's 2+ months after the fact, I doubt we'll ever know exactly
what went wrong here.  In outline:

Rev 39758 (the AST merge) left pythoncore.vcproj in an unusable state.
 That's the VC 7.1 project file that defines what goes into the core
Python DLL on Windows, and it was checked in with conflict markers
embedded.  VC couldn't load the resulting mess, so the Python DLL
couldn't be built anymore.

In rev 39791, MarkH removed the conflict markers and added
parsermodule.c (which had somehow gotten lost during the merge). 
Python then compiled again under VC7.1.  But looks like he didn't run
the tests -- or, like me, just assumed that all the test failures he
saw were universally-known breakage from the then-still-early days of
the AST merge (since most were obviously failures to compile
correctly, that was a tempting assumption).

It sat there then for two months.

As things turn out, rev 39758 also damaged pythoncore.vcproj in other,
non-syntactic ways:

- It removed tokenizer.c from the core DLL, but that's the correct
runtime tokenizer
  code.

- It added pgen.c and tokenizer_pgen.c to the core DLL.  The former is useless
  in the core DLL (I think), and including the latter was just wrong.

RaymondH would not have noticed anything wrong because he still runs
with VC6, and the AST merge didn't muck with _those_ project files.

Anyway, after removing pgen.c and tokenizer_pgen.c, and adding
tokenizer.c, test_builtin and test_pep263 pass again.  In fact, all
the -uall tests pass again (yippee!):

264 tests OK.
38 tests skipped:
    test__locale test_aepack test_al test_applesingle test_bsddb185
    test_cd test_cl test_commands test_crypt test_curses test_dbm
    test_dl test_fcntl test_fork1 test_gdbm test_gl test_grp
    test_hashlib_speed test_imgfile test_ioctl test_linuxaudiodev
    test_macfs test_macostools test_mhlib test_nis test_openpty
    test_ossaudiodev test_plistlib test_poll test_posix test_pty
    test_pwd test_resource test_scriptpackages test_signal
    test_sunaudiodev test_threadsignals test_timing
Those skips are all expected on win32.

So, Merry Christmas to all, and there's no longer any reason to
deprive yourself of the joy of upgrading to Windows ;-)

From skip at pobox.com  Sun Dec 25 17:37:25 2005
From: skip at pobox.com (skip@pobox.com)
Date: Sun, 25 Dec 2005 10:37:25 -0600
Subject: [Python-Dev] status of development documentation
In-Reply-To: <1f7befae0512250829i277f50e1j6d10cc65465b6a9b@mail.gmail.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	<1f7befae0512240719l969a61ep2eafaa616c4b840a@mail.gmail.com>
	<ee2a432c0512240840p1e37a4d9n7c2379096514ccc8@mail.gmail.com>
	<1f7befae0512241354o1e5a13fen5ea6bec7a17a17ab@mail.gmail.com>
	<1f7befae0512241743n6952a2fbncb5d72b5f7ffa809@mail.gmail.com>
	<bbaeab100512241835t32ba6a0boae45f92709edf960@mail.gmail.com>
	<1f7befae0512242043j5644b80fgede35d239b203489@mail.gmail.com>
	<bbaeab100512242229j6ff54d42i7854389c40115a3f@mail.gmail.com>
	<1f7befae0512250829i277f50e1j6d10cc65465b6a9b@mail.gmail.com>
Message-ID: <17326.51909.576619.398894@montanaro.dyndns.org>


    Tim> So, Merry Christmas to all, and there's no longer any reason to
    Tim> deprive yourself of the joy of upgrading to Windows ;-)

Merry Christmas to you as well Tim.  Hopefully the bad-mood elf left after
seeing how happy you were to have figured out the build problems...  Oh, and
I'll get right on that Windows thing.  Where can I download that again?
<wink>

Skip

From ncoghlan at gmail.com  Sun Dec 25 17:40:38 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 26 Dec 2005 02:40:38 +1000
Subject: [Python-Dev] NotImplemented reaching top-level
In-Reply-To: <20051225162325.GA32552@code1.codespeak.net>
References: <e04bdf310512240931t1049346n@mail.gmail.com>	<20051225113550.GA28497@code1.codespeak.net>	<dom0ah$f29$1@sea.gmane.org>
	<20051225162325.GA32552@code1.codespeak.net>
Message-ID: <43AECB86.50206@gmail.com>

Armin Rigo wrote:
> Hi Reinhold,
> 
> On Sun, Dec 25, 2005 at 12:37:53PM +0100, Reinhold Birkenfeld wrote:
>>> that nobody fully understands the convoluted code paths of abstract.c
>>> any more :-(
>> Time for a rewrite?
> 
> Of course, speaking of a rewrite, PyPy does the "right thing" in these
> two areas.  Won't happen to CPython, though.  There are too much
> backward-compatibility issues with the PyTypeObject structure; I think
> we're doomed with patching the bugs as they show up.
> 
> Looking up in the language reference, I see no mention of NotImplemented
> in the page about __add__, __radd__, etc.  I guess it's a documentation
> bug as well, isn't it?  The current code base tries to implement the
> following behavior: Returning NotImplemented from any of the binary
> special methods (__xxx__, __rxxx__, __ixxx__) makes Python proceed as if
> the method was not defined in the first place.
> 
> If we agree on this, I could propose a doc fix, a test, and appropriate
> bug fixes.

That sounds like the right definition to me (I believe this behaviour is what 
Raymond and Facundo were aiming for with the last round of updates to Decimal).

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From tim.peters at gmail.com  Sun Dec 25 18:54:32 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 25 Dec 2005 12:54:32 -0500
Subject: [Python-Dev] Automated Python testing (was Re: status of
	development documentation)
Message-ID: <1f7befae0512250954x4dee672w54efca5ebe0bd80d@mail.gmail.com>

Take a look at:

    http://buildbot.zope.org/

That runs code from:

    http://buildbot.sourceforge.net/

Someone sets up a "buildbot master" (that's what the Zope URL points
at), and then any number of people can volunteer to set up their boxes
as "buildbot slaves".  From time to time the buildbot master asks the
slaves to do the checkout/build/test dance (or any other code dance
you like), the slaves report results back to the master, and the
master displays the slaves' results.

If you look at the 2nd-leftmost column, you can see that the master
knows when checkins have been done.  Checkins can trigger asking the
slaves to run tests, and if the tests fail on some slave the master
can send email saying so, including the list of checkins ("the
blamelist") done since the last time that slave ran tests:

    The guilty developer can be identified and harassed without human
intervention.

:-)

This really helps at Zope Corp.  One downside is that we seem unable
to get an in-house Windows buildbot slave to work reliably, and so far
don't even know whether that's because of Windows, the buildbot code,
or flakiness in our internal network.  It seems quite reliable on
Linux, though.

From bcannon at gmail.com  Sun Dec 25 20:55:11 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Sun, 25 Dec 2005 11:55:11 -0800
Subject: [Python-Dev] NotImplemented reaching top-level
In-Reply-To: <20051225162325.GA32552@code1.codespeak.net>
References: <e04bdf310512240931t1049346n@mail.gmail.com>
	<20051225113550.GA28497@code1.codespeak.net>
	<dom0ah$f29$1@sea.gmane.org>
	<20051225162325.GA32552@code1.codespeak.net>
Message-ID: <bbaeab100512251155t1d056750y842bd1adf5916e30@mail.gmail.com>

On 12/25/05, Armin Rigo <arigo at tunes.org> wrote:
> Hi Reinhold,
>
> On Sun, Dec 25, 2005 at 12:37:53PM +0100, Reinhold Birkenfeld wrote:
> > > that nobody fully understands the convoluted code paths of abstract.c
> > > any more :-(
> >
> > Time for a rewrite?
>

Maybe.  Also realize we will have a chance to clean it up when Python
3 comes around since the classic class stuff will be ripped out.  That
way we might have a chance to streamline the code.

> Of course, speaking of a rewrite, PyPy does the "right thing" in these
> two areas.  Won't happen to CPython, though.  There are too much
> backward-compatibility issues with the PyTypeObject structure; I think
> we're doomed with patching the bugs as they show up.
>
> Looking up in the language reference, I see no mention of NotImplemented
> in the page about __add__, __radd__, etc.  I guess it's a documentation
> bug as well, isn't it?  The current code base tries to implement the
> following behavior: Returning NotImplemented from any of the binary
> special methods (__xxx__, __rxxx__, __ixxx__) makes Python proceed as if
> the method was not defined in the first place.
>

This is what I always assumed the behaviour was supposed to be, so I
am quite happy to go with that.

-Brett

From bcannon at gmail.com  Sun Dec 25 21:23:59 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Sun, 25 Dec 2005 12:23:59 -0800
Subject: [Python-Dev] Automated Python testing (was Re: status of
	development documentation)
In-Reply-To: <1f7befae0512250954x4dee672w54efca5ebe0bd80d@mail.gmail.com>
References: <1f7befae0512250954x4dee672w54efca5ebe0bd80d@mail.gmail.com>
Message-ID: <bbaeab100512251223j33b2fb96k513e6726b04cd233@mail.gmail.com>

On 12/25/05, Tim Peters <tim.peters at gmail.com> wrote:
> Take a look at:
>
>     http://buildbot.zope.org/
>
> That runs code from:
>
>     http://buildbot.sourceforge.net/
>
> Someone sets up a "buildbot master" (that's what the Zope URL points
> at), and then any number of people can volunteer to set up their boxes
> as "buildbot slaves".

As in some machine I might personally have left on?  That would
require a static IP which I don't know how common that will be.  But
then again I am willing to bet that the Python community is big enough
that people who do have machines that are idle enough that we should
be able to get good coverage.  Wonder if we would have to worry about
result pollution from someone who thought it was funny to send in
false negatives?

>  From time to time the buildbot master asks the
> slaves to do the checkout/build/test dance (or any other code dance
> you like), the slaves report results back to the master, and the
> master displays the slaves' results.
>
> If you look at the 2nd-leftmost column, you can see that the master
> knows when checkins have been done.  Checkins can trigger asking the
> slaves to run tests, and if the tests fail on some slave the master
> can send email saying so, including the list of checkins ("the
> blamelist") done since the last time that slave ran tests:
>
>     The guilty developer can be identified and harassed without human
> intervention.
>
> :-)
>
> This really helps at Zope Corp.  One downside is that we seem unable
> to get an in-house Windows buildbot slave to work reliably, and so far
> don't even know whether that's because of Windows, the buildbot code,
> or flakiness in our internal network.  It seems quite reliable on
> Linux, though.

Well, it is written in Python so *someone* here should either be able
to fix it or properly blame it on Windows.  =)

The idea of the PSF paying to have some machines set up to run
consistent tests has come up multiple times.  I know Neal has said he
would be willing to host the machines at his house before (but I think
this may have been before his relocation to California).  This whole
situation of going two months without knowing that a major platform is
broken shows that this is a real problem and ignoring it is probably
not a good thing.  =)

If we ask for volunteer machines we could offer to put up company or
personal names on the buildbot page of those who have volunteered CPU
cycles.  I am sure that will help motivate companies and people to
install the software on a spare machine.  Heck, I would have no
problem giving a specific company sole sponsorship kudos if they gave
us boxes that covered enough core operating systems.

Maybe this is something to bring up at the PSF meeting and to hash out
at the sprints?

-Brett

From fdrake at acm.org  Sun Dec 25 23:09:43 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sun, 25 Dec 2005 17:09:43 -0500
Subject: [Python-Dev] Automated Python testing (was Re: status of
	development documentation)
In-Reply-To: <bbaeab100512251223j33b2fb96k513e6726b04cd233@mail.gmail.com>
References: <1f7befae0512250954x4dee672w54efca5ebe0bd80d@mail.gmail.com>
	<bbaeab100512251223j33b2fb96k513e6726b04cd233@mail.gmail.com>
Message-ID: <200512251709.43980.fdrake@acm.org>

On Sunday 25 December 2005 15:23, Brett Cannon wrote:
 > As in some machine I might personally have left on?  That would
 > require a static IP which I don't know how common that will be.  But

Only buildbot masters are required to have resolvable names (not necessarily 
static, though it helps; dynamic DNS is fine if the name propogation to the 
slaves is fast).  The slaves do not need to be reachable from the buildbot 
master, since the slave is responsible for contacting the master.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From tim.peters at gmail.com  Sun Dec 25 23:35:43 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 25 Dec 2005 17:35:43 -0500
Subject: [Python-Dev] Automated Python testing (was Re: status of
	development documentation)
In-Reply-To: <bbaeab100512251223j33b2fb96k513e6726b04cd233@mail.gmail.com>
References: <1f7befae0512250954x4dee672w54efca5ebe0bd80d@mail.gmail.com>
	<bbaeab100512251223j33b2fb96k513e6726b04cd233@mail.gmail.com>
Message-ID: <1f7befae0512251435x56a1157ew9fc3ecdb2bcb3480@mail.gmail.com>

[Tim]
>> Take a look at:
>>
>>     http://buildbot.zope.org/
>>
>> That runs code from:
>>
>>     http://buildbot.sourceforge.net/
>>
>> Someone sets up a "buildbot master" (that's what the Zope URL points
>> at), and then any number of people can volunteer to set up their boxes
>> as "buildbot slaves".

[Brett]
> As in some machine I might personally have left on?

Slaves have to be powered on to work, yes ;-)

> That would require a static IP which I don't know how common that
> will be.

Spend a few minutes skimming the buildbot docs -- I'm not an expert,
but it's a real system in real use, and they have solutions for the
obvious problems.  In this case, while the master passes out commands
to run and collects status, a slave _initiates_ communication with the
master.  A static IP for the master is good for that, but I figure the
slave can keep participating happily then for just as long as it can
keep a socket connection open with the master.

If a slave goes away (network problem; powered off; whatever), that's
fine too.  The master can't talk to it then, and the slave's column in
the master's display will say the slave is offline.  Or, if it's been
so long that all info about the slave has "scrolled off" the display,
the column will just say "none" above it.  You can see a couple
examples of that in the

    http://buildbot.zope.org

display today, for some Windows slaves that have gone missing in action.

> then again I am willing to bet that the Python community is big enough
> that people who do have machines that are idle enough that we should
> be able to get good coverage.  Wonder if we would have to worry about
> result pollution from someone who thought it was funny to send in
> false negatives?

I wouldn't worry about it.  For one thing, while anyone can volunteer
to participate, the buildbot master's admin has to set up config info
for each specific slave they want to _allow_ to participate.  It's
more like a moderated mailing list that way ;-)  A slave also needs to
know a password (which the master's admin emails (for example) to the
slave's admin if the former wants the latter to participate).

...

>> One downside is that we seem unable to get an in-house Windows
>> buildbot slave to work reliably, and so far don't even know whether that's
>> because of Windows, the buildbot code, or flakiness in our internal
>> network.  It seems quite reliable on Linux, though.

> Well, it is written in Python so *someone* here should either be able
> to fix it or properly blame it on Windows.  =)

Yup!

> The idea of the PSF paying to have some machines set up to run
> consistent tests has come up multiple times.

A brilliant part of the buildbot approach is that a sub-community
claiming to care about a  platform (major or not) can put a bit of
resource where their mouth is by offering part-time use of a box to
run the checkout/build/test dance.  That way platform experts who
presumably care about their platform are in charge of all aspects of
setting their platform up.  No centralized "compile farm" can work as
well, unless it has enough money to buy machines-- and expert
caretakers --for umpteen off-the-wall OS variations.

> I know Neal has said he would be willing to host the machines at his
> house before (but I think this may have been before his relocation to
> California).

Looks like he's already running automated tests:

    http://docs.python.org/dev/results/

The various steps there could be defined as buildbot actions, and then
run on any number of boxes "whenever something changes".

> This whole situation of going two months without knowing that a major
> platform is broken shows that this is a real problem and ignoring it is
> probably not a good thing.  =)

It's generally true that the sooner you find out something has broken,
the easier it is to repair it.  For "minor" platforms, I'd say we have
no idea whether the trunk has regressed wrt 2.4.2.  There's simply no
way to know without trying it.

> If we ask for volunteer machines we could offer to put up company or
> personal names on the buildbot page of those who have volunteered CPU
> cycles.  I am sure that will help motivate companies and people to
> install the software on a spare machine.

Finding volunteers has been surprisingly (to me) difficult.  Most (but
not all) of the machines you see on the Zope page are ZC-internal
boxes, and, e.g., a Mac OS box is still missing.

> Heck, I would have no problem giving a specific company sole sponsorship
> kudos if they gave us boxes that covered enough core operating systems.

Cool too.

> Maybe this is something to bring up at the PSF meeting and to hash out
> at the sprints?

It primarly takes someone with access to "a python.org machine" to
volunteer to install and play admin for a buildbot master.  A
committee wouldn't actually help with that ;-)

From aahz at pythoncraft.com  Sun Dec 25 23:51:28 2005
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 25 Dec 2005 14:51:28 -0800
Subject: [Python-Dev] status of development documentation
In-Reply-To: <1f7befae0512250829i277f50e1j6d10cc65465b6a9b@mail.gmail.com>
References: <1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	<1f7befae0512240719l969a61ep2eafaa616c4b840a@mail.gmail.com>
	<ee2a432c0512240840p1e37a4d9n7c2379096514ccc8@mail.gmail.com>
	<1f7befae0512241354o1e5a13fen5ea6bec7a17a17ab@mail.gmail.com>
	<1f7befae0512241743n6952a2fbncb5d72b5f7ffa809@mail.gmail.com>
	<bbaeab100512241835t32ba6a0boae45f92709edf960@mail.gmail.com>
	<1f7befae0512242043j5644b80fgede35d239b203489@mail.gmail.com>
	<bbaeab100512242229j6ff54d42i7854389c40115a3f@mail.gmail.com>
	<1f7befae0512250829i277f50e1j6d10cc65465b6a9b@mail.gmail.com>
Message-ID: <20051225225128.GA23480@panix.com>

On Sun, Dec 25, 2005, Tim Peters wrote:
>
> So, Merry Christmas to all, and there's no longer any reason to
> deprive yourself of the joy of upgrading to Windows ;-)

Much Grass!  I already upgraded to Windows, but it's turned off in favor
of my Linux box and iBook.

Yesterday I decided to try doing an svn checkout for the first time.  Has
anyone ever tried doing that from the top?  There's more than 18GB of
stuff, and it never actually completes.  (I'm mostly mentioning this as
FYI/FYA -- I'm doing it to exercise my new computer.)
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Don't listen to schmucks on USENET when making legal decisions.  Hire
yourself a competent schmuck."  --USENET schmuck (aka Robert Kern)

From bcannon at gmail.com  Mon Dec 26 01:54:44 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Sun, 25 Dec 2005 16:54:44 -0800
Subject: [Python-Dev] Automated Python testing (was Re: status of
	development documentation)
In-Reply-To: <1f7befae0512251435x56a1157ew9fc3ecdb2bcb3480@mail.gmail.com>
References: <1f7befae0512250954x4dee672w54efca5ebe0bd80d@mail.gmail.com>
	<bbaeab100512251223j33b2fb96k513e6726b04cd233@mail.gmail.com>
	<1f7befae0512251435x56a1157ew9fc3ecdb2bcb3480@mail.gmail.com>
Message-ID: <bbaeab100512251654l18f29e34rfd13a3becaa704ec@mail.gmail.com>

On 12/25/05, Tim Peters <tim.peters at gmail.com> wrote:
> [Tim]
> >> Take a look at:
> >>
> >>     http://buildbot.zope.org/
> >>
> >> That runs code from:
> >>
> >>     http://buildbot.sourceforge.net/
> >>
> >> Someone sets up a "buildbot master" (that's what the Zope URL points
> >> at), and then any number of people can volunteer to set up their boxes
> >> as "buildbot slaves".
>
> [Brett]
> > As in some machine I might personally have left on?
>
> Slaves have to be powered on to work, yes ;-)
>
> > That would require a static IP which I don't know how common that
> > will be.
>
> Spend a few minutes skimming the buildbot docs -- I'm not an expert,
> but it's a real system in real use, and they have solutions for the
> obvious problems.  In this case, while the master passes out commands
> to run and collects status, a slave _initiates_ communication with the
> master.  A static IP for the master is good for that, but I figure the
> slave can keep participating happily then for just as long as it can
> keep a socket connection open with the master.
>

OK, so it is a pull from the slave side, not a push on the master side.

> If a slave goes away (network problem; powered off; whatever), that's
> fine too.  The master can't talk to it then, and the slave's column in
> the master's display will say the slave is offline.  Or, if it's been
> so long that all info about the slave has "scrolled off" the display,
> the column will just say "none" above it.  You can see a couple
> examples of that in the
>
>     http://buildbot.zope.org
>
> display today, for some Windows slaves that have gone missing in action.
>
> > then again I am willing to bet that the Python community is big enough
> > that people who do have machines that are idle enough that we should
> > be able to get good coverage.  Wonder if we would have to worry about
> > result pollution from someone who thought it was funny to send in
> > false negatives?
>
> I wouldn't worry about it.  For one thing, while anyone can volunteer
> to participate, the buildbot master's admin has to set up config info
> for each specific slave they want to _allow_ to participate.  It's
> more like a moderated mailing list that way ;-)  A slave also needs to
> know a password (which the master's admin emails (for example) to the
> slave's admin if the former wants the latter to participate).
>
> ...
>
> >> One downside is that we seem unable to get an in-house Windows
> >> buildbot slave to work reliably, and so far don't even know whether that's
> >> because of Windows, the buildbot code, or flakiness in our internal
> >> network.  It seems quite reliable on Linux, though.
>
> > Well, it is written in Python so *someone* here should either be able
> > to fix it or properly blame it on Windows.  =)
>
> Yup!
>
> > The idea of the PSF paying to have some machines set up to run
> > consistent tests has come up multiple times.
>
> A brilliant part of the buildbot approach is that a sub-community
> claiming to care about a  platform (major or not) can put a bit of
> resource where their mouth is by offering part-time use of a box to
> run the checkout/build/test dance.  That way platform experts who
> presumably care about their platform are in charge of all aspects of
> setting their platform up.  No centralized "compile farm" can work as
> well, unless it has enough money to buy machines-- and expert
> caretakers --for umpteen off-the-wall OS variations.
>

I guess if someone complains about wanting better support for platform
X we could then say that we don't have a buildbot slave for it and if
they want to help the situation they can get one set up.  =)

> > I know Neal has said he would be willing to host the machines at his
> > house before (but I think this may have been before his relocation to
> > California).
>
> Looks like he's already running automated tests:
>
>     http://docs.python.org/dev/results/
>
> The various steps there could be defined as buildbot actions, and then
> run on any number of boxes "whenever something changes".
>

See, that is what threw me; thinking that when the master knows a
change happens it pushes out to the slaves.  I guess the master notes
that there is a reason to do a new run and that is what the slaves are
constantly checking with the master about.

> > This whole situation of going two months without knowing that a major
> > platform is broken shows that this is a real problem and ignoring it is
> > probably not a good thing.  =)
>
> It's generally true that the sooner you find out something has broken,
> the easier it is to repair it.  For "minor" platforms, I'd say we have
> no idea whether the trunk has regressed wrt 2.4.2.  There's simply no
> way to know without trying it.
>

Right.  Part of the reason AIX, I am sure, keeps breaking.

> > If we ask for volunteer machines we could offer to put up company or
> > personal names on the buildbot page of those who have volunteered CPU
> > cycles.  I am sure that will help motivate companies and people to
> > install the software on a spare machine.
>
> Finding volunteers has been surprisingly (to me) difficult.  Most (but
> not all) of the machines you see on the Zope page are ZC-internal
> boxes, and, e.g., a Mac OS box is still missing.
>

If the install process is really simple and we give people an easy way
to specify how often/when they poll the master then I think more
people would be willing to do it.  If you can have your box at work do
it after work hours or have  your box at home do it while you are at
work during the week then I think more people will step up.  Lowering
the barrier and helping people minimize the impact on their machines
to only when they want it to occur should help.

Maybe this is all in the docs, I don't know (about to leave for Xmas
dinner so don't have the time right now).

> > Heck, I would have no problem giving a specific company sole sponsorship
> > kudos if they gave us boxes that covered enough core operating systems.
>
> Cool too.
>
> > Maybe this is something to bring up at the PSF meeting and to hash out
> > at the sprints?
>
> It primarly takes someone with access to "a python.org machine" to
> volunteer to install and play admin for a buildbot master.  A
> committee wouldn't actually help with that ;-)

Well, maybe Neal will be up for this on top of the auto test he has
set up.  I would say I would do it but I don't have the proper server
access on pydotorg and I don't have much experience administering on
Linux or else I would be willing to do it with someone.

The other testing option I have seen tossed around is having
regrtest.py have an option of emailing the test results of a test run
somewhere.  So if tests failed run them directly and then append that
output in an email with the system information and an optional contact
email address if the person is willing to help debug the problem. 
Would be great for alpha and beta releases since it doesn't require a
dedicated system but just allowing an email to be sent with some
system info.

-Brett

From blais at furius.ca  Mon Dec 26 03:10:42 2005
From: blais at furius.ca (Martin Blais)
Date: Sun, 25 Dec 2005 21:10:42 -0500
Subject: [Python-Dev] deque alternative (was: Linked lists)
In-Reply-To: <43AE96FF.2010303@stackless.com>
References: <8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>
	<ca471dc20512141237i4f09098cudb951fbe70623085@mail.gmail.com>
	<43AE96FF.2010303@stackless.com>
Message-ID: <8393fff0512251810q6aa5d583s2733e265b4bcd648@mail.gmail.com>

On 12/25/05, Christian Tismer <tismer at stackless.com> wrote:
> Guido van Rossum wrote:
>
> > Python's philosophy about (built-in) data types, inherited from ABC,
> > is to offer a few powerful clearly distinct choices rather than lots
> > of alternatives with overlapping usages. This reduces the time it
> > takes to choose a data type and reduces the risk of picking the wrong
> > type. (You seem to be indicating that this is indeed what's happening
> > to you in Lisp. :-)
>
> Speaking about the new dequeue type, I have the impression that
> this reasoning applies there a bit, too?
> Dequeues of course have applications, and I saw them used,
> especially by newcomers to the language. So I like their
> functionality.
>
> But actually, I'm not so convinced if we need to introduce
> a new datatype. How about an extension to normal lists
> that does a little tracking of use-cases and decides about
> changing its implementation upon the presence of inserts/deletes
> at the lower end of the list?
>
> I'm btw. not sure if this raises an implicit/explicitiness issue.
> My question is if it is necessary to put the burden about explicitly
> choosing a specialized two-ended list on the programmer, or if it
> is simple enough to make lists just do the right thing, if lists
> are used in a dequeue-style manner. Or is this maybe too much magic
> happening?

IMO it's a little bit too much magic.  Plus, if you pass these
instances around e.g. between libraries, how could you determine with
certainty the usage patterns without analysing the entire codebase?  I
think the user has to be involved.

A problem I see now is that there is no way for me to
indicate--whether there currently exists or not an appropriate data
type with the desired characteristic-- whether I want an array (Python
list) or a real list, both of which are really, really basic
fundamental kinds of data structures with very different complexity
characteristics.  The source code I write does not carry this
information and therefore if in the future it would become available
there won't be an easy way to convert code to take advantage of this. 
This choice, is being made in the programmer's head, but the
distinction blurred in the source code because lists are just not
there.

I still haven't had time to cook a proper reply to Guido, but one
thing I see is that many ppl on the list seem to think that because
there aren't many use cases (that they can see), therefore there isn't
much use for a list collection type.  I would instead argue that
because the list type has never been available, people have gotten
used not to use them, and therefore settle with arrays and do not see
a need for them.  When all you have is a hammer, everything seems to
be a nail.  When you don't have lists, you make do with arrays and you
think it's ok (and it pretty much is, in the sense that Python, at
least compared to C/LISP, is very slow, so it for the kinds of things
you use it for, it does not matter all that much).  When you're used
to having lists available, you miss being able to express that the
data structure you want should have the characteristics of a list,
with the intended performance characteristics.  Being able to select
between a list and an array is like having a bit more vocabulary in
natural languages.

I would agree with the use case condition if we were talking about a
very domain-specific collection type, but we're talking about lists
here... it's really fundamental stuff...

Also, there is something incredibly elegant and simple and compact
about the cons cell, maybe all we need is a good simple cons cell type
and a nice interface on it, so you get both single-linked lists and
trees at the same time...

(more after the xmas thing has passed)

From pje at telecommunity.com  Mon Dec 26 04:08:36 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 25 Dec 2005 22:08:36 -0500
Subject: [Python-Dev] deque alternative (was: Linked lists)
In-Reply-To: <8393fff0512251810q6aa5d583s2733e265b4bcd648@mail.gmail.com
 >
References: <43AE96FF.2010303@stackless.com>
	<8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>
	<ca471dc20512141237i4f09098cudb951fbe70623085@mail.gmail.com>
	<43AE96FF.2010303@stackless.com>
Message-ID: <5.1.1.6.0.20051225215827.02233c30@mail.telecommunity.com>

At 09:10 PM 12/25/2005 -0500, Martin Blais wrote:
>I still haven't had time to cook a proper reply to Guido, but one
>thing I see is that many ppl on the list seem to think that because
>there aren't many use cases (that they can see), therefore there isn't
>much use for a list collection type.  I would instead argue that
>because the list type has never been available, people have gotten
>used not to use them, and therefore settle with arrays and do not see
>a need for them.

Since I routinely use 2-item tuples (twoples?) as cons cells when I 
actually need a linked list, I think I'm in a good position to say that 
this isn't true.  Certainly it's not true for me.

Since Python effectively has single-character operations for both consing 
and car/cdr (a ',' on the right or left sides of an assignment statement 
respectively), as well as trivial truth testing for () as a "nil", suggests 
to me that anybody who wants a lispy list can easily have one.  For any 
application where such a structure excels (like shared sublists/trees and 
recursive traversals), it's so darn easy to use them in Python that I don't 
think anything special is needed.

Honestly, ISTM that the One Obvious Way to do lispy lists in Python is to 
just use two-tuples, with no special library functions.  Sequence packing 
and unpacking is extremely fast in Python, too, probably faster than any 
method calls to a more heavyweight builtin type could be.

Thus, I'm -1 on creating another cons type.  Why do this:

     foo = cons(bar, None)
     car_foo = foo.car
     cdr_foo = foo.cdr

when you can just do this:

     foo = bar, ()

     car_foo, cdr_foo = foo

How much simpler can you get?


From aahz at pythoncraft.com  Mon Dec 26 04:28:58 2005
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 25 Dec 2005 19:28:58 -0800
Subject: [Python-Dev] deque alternative (was: Linked lists)
In-Reply-To: <8393fff0512251810q6aa5d583s2733e265b4bcd648@mail.gmail.com>
References: <8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>
	<ca471dc20512141237i4f09098cudb951fbe70623085@mail.gmail.com>
	<43AE96FF.2010303@stackless.com>
	<8393fff0512251810q6aa5d583s2733e265b4bcd648@mail.gmail.com>
Message-ID: <20051226032858.GA22089@panix.com>

On Sun, Dec 25, 2005, Martin Blais wrote:
>
> I still haven't had time to cook a proper reply to Guido, but one
> thing I see is that many ppl on the list seem to think that because
> there aren't many use cases (that they can see), therefore there isn't
> much use for a list collection type.  

Please take this to comp.lang.python

Side note: nobody AFAICT has suggested that you drop this -- only that
getting a built-in is extremely unlikely.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Don't listen to schmucks on USENET when making legal decisions.  Hire
yourself a competent schmuck."  --USENET schmuck (aka Robert Kern)

From tim.peters at gmail.com  Mon Dec 26 04:46:45 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 25 Dec 2005 22:46:45 -0500
Subject: [Python-Dev] deque alternative (was: Linked lists)
In-Reply-To: <8393fff0512251810q6aa5d583s2733e265b4bcd648@mail.gmail.com>
References: <8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>
	<ca471dc20512141237i4f09098cudb951fbe70623085@mail.gmail.com>
	<43AE96FF.2010303@stackless.com>
	<8393fff0512251810q6aa5d583s2733e265b4bcd648@mail.gmail.com>
Message-ID: <1f7befae0512251946y31552befx468d1f7dda23b6e1@mail.gmail.com>

[Martin Blais]
> ...
> Also, there is something incredibly elegant and simple and compact
> about the cons cell, maybe all we need is a good simple cons cell type
> and a nice interface on it, so you get both single-linked lists and
> trees at the same time...

The first "cons cell" C extension for Python I know about was written in 1994:

    http://www.python.org/search/hypermail/python-1994q2/0110.html

There have been others, but the audience for them appears so small
that none catch on.

Like Phillip Eby, I use 2-tuples for this when I feel the need
(usually during a backtracking graph search, to keep track of paths
back to the root in a space-efficient way), and am happy with that.

From tim.peters at gmail.com  Mon Dec 26 04:51:38 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 25 Dec 2005 22:51:38 -0500
Subject: [Python-Dev] Build failure and problem on Windows
In-Reply-To: <3bkms96x.fsf@python.net>
References: <hd93u0lr.fsf@python.net> <8xufu01l.fsf@python.net>
	<3bkms96x.fsf@python.net>
Message-ID: <1f7befae0512251951u238fdf23hb141c884336c34ce@mail.gmail.com>

[Thomas Heller]
>>> Building the svn trunk on Windows fails because Python\pyarena.c is
>>> missing in the pythoncore.vcproj file (I'm not yet up to speed with svn,
>>> otherwise I would have checked in a fix for this myself).
>>>
>>> Worse, when running the built exe it segfaults in Py_GetBuildInfo(),
>>> because it is picking up somehow a definition of #define BUILD 'b' (from
>>> cPickle.c? Could that be?)

[also Thomas Heller]
>> I should have known better, but BUILD is defined in the MSVC project
>> file as BUILD=60.

[Thomas Heller again]
> I've committed a fix for both (Hope these comments aren't off-topic
> nowadays for python-dev).

Not at all, according to me.  Nobody appears to be paying attention to
Python under VC 7.1 these days, but what you did here certainly helped
me when I got some time for it this weekend!  Thank you.

From exarkun at divmod.com  Mon Dec 26 05:25:54 2005
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Sun, 25 Dec 2005 23:25:54 -0500
Subject: [Python-Dev] Automated Python testing (was Re: status of
 development documentation)
In-Reply-To: <bbaeab100512251654l18f29e34rfd13a3becaa704ec@mail.gmail.com>
Message-ID: <20051226042554.1217.597721330.divmod.quotient.8225@ohm>

On Sun, 25 Dec 2005 16:54:44 -0800, Brett Cannon <bcannon at gmail.com> wrote:
>On 12/25/05, Tim Peters <tim.peters at gmail.com> wrote:
>> [Tim]
>> >> Take a look at:
>> >>
>> >>     http://buildbot.zope.org/
>> >>
>> >> That runs code from:
>> >>
>> >>     http://buildbot.sourceforge.net/
>> >>
>> >> Someone sets up a "buildbot master" (that's what the Zope URL points
>> >> at), and then any number of people can volunteer to set up their boxes
>> >> as "buildbot slaves".
>>
>> [Brett]
>> > As in some machine I might personally have left on?
>>
>> Slaves have to be powered on to work, yes ;-)
>>
>> > That would require a static IP which I don't know how common that
>> > will be.
>>
>> Spend a few minutes skimming the buildbot docs -- I'm not an expert,
>> but it's a real system in real use, and they have solutions for the
>> obvious problems.  In this case, while the master passes out commands
>> to run and collects status, a slave _initiates_ communication with the
>> master.  A static IP for the master is good for that, but I figure the
>> slave can keep participating happily then for just as long as it can
>> keep a socket connection open with the master.
>>
>
>OK, so it is a pull from the slave side, not a push on the master side.
>
> [snip]
>
>See, that is what threw me; thinking that when the master knows a
>change happens it pushes out to the slaves.  I guess the master notes
>that there is a reason to do a new run and that is what the slaves are
>constantly checking with the master about.

The whole world isn't HTTP.  There is not necessarily any correlation 
between the party which initiates the TCP connection and the party 
which instigates a run.  Slaves connect to the master when they come 
online.  The master uses that connection to push commands to the slaves 
at the appropriate time.

Buildbot also supports having multiple slaves perform runs (or "builds", 
in buildbot terminology) for a particular configuration.  This means 
that several people can cooperate to give full coverage for a particular 
platform, even if none of them can leave a machine on 24/7.  You really 
do want 24/7 coverage for a supported platform, because utility declines 
rather rapidly as you fall short of this.  Developers tend to have an 
amazing knack for committing broken changes when the slave which would 
have noticed them is offline :)

Jean-Paul

From aahz at pythoncraft.com  Mon Dec 26 05:38:31 2005
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 25 Dec 2005 20:38:31 -0800
Subject: [Python-Dev] file() vs open(), round 7
Message-ID: <20051226043831.GA24197@panix.com>

Guido sez in 
http://mail.python.org/pipermail/python-dev/2004-July/045921.html
that it is not correct to recommend using ``file()`` instead of
``open()``.  However, because ``open()`` currently *is* an alias to
``file()``, we end up with the following problem (verified in current
HEAD) where doing ``help(open)`` brings up the docs for ``file()``:

>>> help(open)
Help on class file in module __builtin__:

class file(object)
 |  file(name[, mode[, buffering]]) -> file object
 |
 |  Open a file.  The mode can be 'r', 'w' or 'a' for reading (default),
[...]
 |  Note:  open() is an alias for file().

This is confusing.  I suggest that we make ``open()`` a factory function
right now.  (I'll submit a bug report (and possibly a patch) after I get
agreement.)
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Don't listen to schmucks on USENET when making legal decisions.  Hire
yourself a competent schmuck."  --USENET schmuck (aka Robert Kern)

From jcarlson at uci.edu  Mon Dec 26 05:54:42 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sun, 25 Dec 2005 20:54:42 -0800
Subject: [Python-Dev] deque alternative (was: Linked lists)
In-Reply-To: <1f7befae0512251946y31552befx468d1f7dda23b6e1@mail.gmail.com>
References: <8393fff0512251810q6aa5d583s2733e265b4bcd648@mail.gmail.com>
	<1f7befae0512251946y31552befx468d1f7dda23b6e1@mail.gmail.com>
Message-ID: <20051225204328.BECE.JCARLSON@uci.edu>


Tim Peters <tim.peters at gmail.com> wrote:
> Like Phillip Eby, I use 2-tuples for this when I feel the need
> (usually during a backtracking graph search, to keep track of paths
> back to the root in a space-efficient way), and am happy with that.

Then there's the whole Python list with append() and pop().  It takes a
method resolution and function call, but at least in Python 2.3, it is a
hair faster...

 - Josiah

>>> if 1:
...     t = time.time()
...     a = tuple()
...     for i in xrange(1000000):
...             a = (i, a)
...     for i in xrange(1000000):
...             i,a = a
...     print time.time()-t
...
3.28500008583
>>> if 1:
...     t = time.time()
...     a = []
...     for i in xrange(1000000):
...             a.append(i)
...     for i in xrange(1000000):
...             i = a.pop()
...     print time.time()-t
...
3.02399992943


From skip at pobox.com  Mon Dec 26 06:38:12 2005
From: skip at pobox.com (skip@pobox.com)
Date: Sun, 25 Dec 2005 23:38:12 -0600
Subject: [Python-Dev] Automated Python testing (was Re: status of
 development documentation)
In-Reply-To: <bbaeab100512251223j33b2fb96k513e6726b04cd233@mail.gmail.com>
References: <1f7befae0512250954x4dee672w54efca5ebe0bd80d@mail.gmail.com>
	<bbaeab100512251223j33b2fb96k513e6726b04cd233@mail.gmail.com>
Message-ID: <17327.33220.505496.27645@montanaro.dyndns.org>


    Brett> As in some machine I might personally have left on?  That would
    Brett> require a static IP which I don't know how common that will be.

Nah, just use dyndns.org.

Skip

From tismer at stackless.com  Mon Dec 26 13:38:37 2005
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 26 Dec 2005 13:38:37 +0100
Subject: [Python-Dev] deque alternative
In-Reply-To: <8393fff0512251810q6aa5d583s2733e265b4bcd648@mail.gmail.com>
References: <8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>	<ca471dc20512141237i4f09098cudb951fbe70623085@mail.gmail.com>	<43AE96FF.2010303@stackless.com>
	<8393fff0512251810q6aa5d583s2733e265b4bcd648@mail.gmail.com>
Message-ID: <43AFE44D.6040306@stackless.com>

Martin Blais wrote:
> On 12/25/05, Christian Tismer <tismer at stackless.com> wrote:

[is auto-dequeue too much magic?]

> IMO it's a little bit too much magic.  Plus, if you pass these
> instances around e.g. between libraries, how could you determine with
> certainty the usage patterns without analysing the entire codebase?  I
> think the user has to be involved.

Well, I would treat a list as a special case of dequeue.
Dequeues are made of a chain of list pieces. ATM they are
all equally sized. But turning a list into a dequeue like
structure at some point in the future can be done in many
ways. One way would be to embrace the list into the dequeue
structure without any copying involved, if a smooth transition
is crucial.
Passing the object between libraries gives me no problem.
We just track usage a little and even convert back to a list
if it seems appropriate. I think this can be cheap and elegant.
The question is less about implementation, but if we want this
to be explicitly more versatile, documenting it like
"""lists can be efficiently extended at both ends. Dependent on
the usage pattern, the implementation is free to choose a more
compact representation""".

> A problem I see now is that there is no way for me to
> indicate--whether there currently exists or not an appropriate data
> type with the desired characteristic-- whether I want an array (Python
> list) or a real list, both of which are really, really basic
> fundamental kinds of data structures with very different complexity
> characteristics.  The source code I write does not carry this
> information and therefore if in the future it would become available
> there won't be an easy way to convert code to take advantage of this. 
> This choice, is being made in the programmer's head, but the
> distinction blurred in the source code because lists are just not
> there.

I don't think your code has to decide about this. The power lies
in the fact that you don't specify that, but just use the list
in a different way. We do this in the PyPy implementation;
right now it is true that we have a static analysis, but a JIT
is to come, and I'm pretty sure it will try to use an array
until something gets used like a list.
No concrete layout available, yet.

...

> Also, there is something incredibly elegant and simple and compact
> about the cons cell, maybe all we need is a good simple cons cell type
> and a nice interface on it, so you get both single-linked lists and
> trees at the same time...

Tuples are fine for cons cells, and if your usage pattern
indicates that we can optimize, why should you use an extra
data type? You already said what you want, by the way you use
it. Extra-specification would be redundant, wouldn't it? :-)

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From nas at arctrix.com  Mon Dec 26 14:52:41 2005
From: nas at arctrix.com (Neil Schemenauer)
Date: Mon, 26 Dec 2005 13:52:41 +0000 (UTC)
Subject: [Python-Dev] NotImplemented reaching top-level
References: <e04bdf310512240931t1049346n@mail.gmail.com>
	<20051225113550.GA28497@code1.codespeak.net>
	<dom0ah$f29$1@sea.gmane.org>
	<20051225162325.GA32552@code1.codespeak.net>
Message-ID: <doosj9$t1f$1@sea.gmane.org>

Armin Rigo <arigo at tunes.org> wrote:
> Of course, speaking of a rewrite, PyPy does the "right thing" in
> these two areas.  Won't happen to CPython, though.  There are too
> much backward-compatibility issues with the PyTypeObject
> structure; I think we're doomed with patching the bugs as they
> show up.

This is definitely something that should be cleaned up for Python
3k.

> Looking up in the language reference, I see no mention of NotImplemented
> in the page about __add__, __radd__, etc.  I guess it's a documentation
> bug as well, isn't it?  The current code base tries to implement the
> following behavior: Returning NotImplemented from any of the binary
> special methods (__xxx__, __rxxx__, __ixxx__) makes Python proceed as if
> the method was not defined in the first place.
>
> If we agree on this, I could propose a doc fix, a test, and appropriate
> bug fixes.

I believe that's correct.

  Neil


From arigo at tunes.org  Mon Dec 26 15:47:41 2005
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 26 Dec 2005 15:47:41 +0100
Subject: [Python-Dev] NotImplemented reaching top-level
In-Reply-To: <bbaeab100512251155t1d056750y842bd1adf5916e30@mail.gmail.com>
References: <e04bdf310512240931t1049346n@mail.gmail.com>
	<20051225113550.GA28497@code1.codespeak.net>
	<dom0ah$f29$1@sea.gmane.org>
	<20051225162325.GA32552@code1.codespeak.net>
	<bbaeab100512251155t1d056750y842bd1adf5916e30@mail.gmail.com>
Message-ID: <20051226144741.GB32552@code1.codespeak.net>

Hi Brett,

On Sun, Dec 25, 2005 at 11:55:11AM -0800, Brett Cannon wrote:
> Maybe.  Also realize we will have a chance to clean it up when Python
> 3 comes around since the classic class stuff will be ripped out.  That
> way we might have a chance to streamline the code.

For once, old-style classes are not to blame here: it's only about the
oldest aspects of the PyTypeObject structure and substructures.


I-said-that-no-one-knows-this-code-any-more'ly yours,

Armin

From blais at furius.ca  Mon Dec 26 17:07:03 2005
From: blais at furius.ca (Martin Blais)
Date: Mon, 26 Dec 2005 11:07:03 -0500
Subject: [Python-Dev] deque alternative (was: Linked lists)
In-Reply-To: <20051225204328.BECE.JCARLSON@uci.edu>
References: <8393fff0512251810q6aa5d583s2733e265b4bcd648@mail.gmail.com>
	<1f7befae0512251946y31552befx468d1f7dda23b6e1@mail.gmail.com>
	<20051225204328.BECE.JCARLSON@uci.edu>
Message-ID: <8393fff0512260807y53a73baar197e07cf8b8eaa8c@mail.gmail.com>

On 12/25/05, Josiah Carlson <jcarlson at uci.edu> wrote:
>
> Tim Peters <tim.peters at gmail.com> wrote:
> > Like Phillip Eby, I use 2-tuples for this when I feel the need
> > (usually during a backtracking graph search, to keep track of paths
> > back to the root in a space-efficient way), and am happy with that.
>
> Then there's the whole Python list with append() and pop().  It takes a
> method resolution and function call, but at least in Python 2.3, it is a
> hair faster...

Josiah, that's beside the point, because it is not space-efficient,
the list is actually an dynamic array/vector, and I would expect an
efficient array implementation to grow exponentially.  One of the
reasons we're talking about lists is to avoid exactly this problem...

From steve at holdenweb.com  Mon Dec 26 17:07:40 2005
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 26 Dec 2005 16:07:40 +0000
Subject: [Python-Dev] status of development documentation
In-Reply-To: <43AD5734.1020903@holdenweb.com>
References: <20051221095628.BE79.JCARLSON@uci.edu>	<1135196162.14495.15.camel@geddy.wooz.org>	<doci50$nom$1@sea.gmane.org>
	<2m4q512vub.fsf@starship.python.net>	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>	<dogduu$o1m$1@sea.gmane.org>	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>	<17324.31929.979592.958222@montanaro.dyndns.org>	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>	<doj9us$226$1@sea.gmane.org>	<17325.18832.329887.580689@montanaro.dyndns.org>
	<43AD5734.1020903@holdenweb.com>
Message-ID: <dop4g8$es6$2@sea.gmane.org>

Steve Holden wrote:
> skip at pobox.com wrote:
> 
>>    Steve> Alternatively, is there any mileage in trying to either get
>>    Steve> Sourceforge to provide Windows machines in the compile farm, or
>>    Steve> get Microsoft to provide more software fee to Windows testers?
>>
>>How about seeing if Microsoft has or will create a compile farm?
>>
> 
> I'll see what I can do ...
> 
Interestingly, David Murmann has developed a mechanism using NAnt 
(nant.sf.net) for building Python using the Microsoft free tools. He has 
a patch that he announced on c.l.py, so I encouraged him to post about 
it on python-dev if he's prepared to maintain it.

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From arigo at tunes.org  Mon Dec 26 17:27:10 2005
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 26 Dec 2005 17:27:10 +0100
Subject: [Python-Dev] NotImplemented reaching top-level
In-Reply-To: <43AECB86.50206@gmail.com>
References: <e04bdf310512240931t1049346n@mail.gmail.com>
	<20051225113550.GA28497@code1.codespeak.net>
	<dom0ah$f29$1@sea.gmane.org>
	<20051225162325.GA32552@code1.codespeak.net>
	<43AECB86.50206@gmail.com>
Message-ID: <20051226162710.GA19675@code1.codespeak.net>

Hi,

On Mon, Dec 26, 2005 at 02:40:38AM +1000, Nick Coghlan wrote:
> That sounds like the right definition to me (I believe this behaviour is what 
> Raymond and Facundo were aiming for with the last round of updates to Decimal).

Done in patch #1390657.

Although this patch passes all existing tests plus the ones it adds,
there is a corner and untested case where it could potentially break
code.  Indeed, the only sane patch I could come up with makes
user-defined types fail to work with PySequence_Concat() and
PySequence_Repeat() -- details in the patch.  So I propose that we
clarify what these two functions really mean in term of the Python
language spec, instead of just in term of the CPython-specific sq_concat
and sq_repeat slots.  (BTW that's needed for PyPy/Jython/etc., too, to
give a reasonable meaning to the operator.concat() and operator.repeat()
built-ins.)

I propose the following definitions (which are mostly what the
docstrings already explain anyway):

* PySequence_Concat(a, b) and operator.concat(a, b) mean "a + b", with
  the difference that we check that both arguments appear to be
  sequences (as checked with operator.isSequenceType()).

* PySequence_Repeat(a, b) and operator.repeat(a, b) mean "a * b", where
  "a" is checked to be a sequence and "b" is an integer.  Some bounds
  can be enforced on "b" -- for CPython, it means that it must fit in a
  C int.

The idea is to extend PySequence_Concat() and PySequence_Repeat() to
match the above definitions precisely, which means that for objects not
defining sq_repeat or sq_concat but still appearing to be sequences,
nb_add and nb_multiply should be tried.  I don't think that this would
break existing C or Python code, but it should probably only go in 2.5,
together with the patch #1390657 that relies on it.


A bientot,

Armin

From foom at fuhm.net  Mon Dec 26 17:58:30 2005
From: foom at fuhm.net (James Y Knight)
Date: Mon, 26 Dec 2005 11:58:30 -0500
Subject: [Python-Dev] deque alternative (was: Linked lists)
In-Reply-To: <8393fff0512260807y53a73baar197e07cf8b8eaa8c@mail.gmail.com>
References: <8393fff0512251810q6aa5d583s2733e265b4bcd648@mail.gmail.com>
	<1f7befae0512251946y31552befx468d1f7dda23b6e1@mail.gmail.com>
	<20051225204328.BECE.JCARLSON@uci.edu>
	<8393fff0512260807y53a73baar197e07cf8b8eaa8c@mail.gmail.com>
Message-ID: <39C8CCEF-21D9-4DB2-AE03-374748C6CDB7@fuhm.net>


On Dec 26, 2005, at 11:07 AM, Martin Blais wrote:
>> Then there's the whole Python list with append() and pop().  It  
>> takes a
>> method resolution and function call, but at least in Python 2.3,  
>> it is a
>> hair faster...
>
> Josiah, that's beside the point, because it is not space-efficient,
> the list is actually an dynamic array/vector, and I would expect an
> efficient array implementation to grow exponentially.  One of the
> reasons we're talking about lists is to avoid exactly this problem...

Okay, now *that* reason is a pretty crazy one. Consider what you're  
saying here.

A list made of cons cells takes at least two words per element. And  
that's if you have an efficient GC mechanism tuned for cons cells,  
like popular common lisps have. In python, a cons cell will take at  
least 8 words (gc_next, gc_prev, gc_refs, padding, ob_refcnt,  
ob_type, and finally, car, and cdr). So a list of 100 elements will  
take _at best_ 200 words, and in python, a ghastly 800 words. Plus,  
in python, the overhead per object in the pyobject memory allocation  
system, which I don't know how to quantify.

An array takes one word per element, plus some header overhead. In  
python, the header overhead will be the same as above, plus around 3  
more words, so around 9 words. So to store an array of 100 elements  
will take around 109 words. Now let's say python did overallocate by  
100%. Then the array would take up 209 words. But it doesn't  
overallocate that much. The real formula is:
new_allocated = (newsize >> 3) + (newsize < 9 ? 3 : 6) + newsize;

So, the list will generally be 1/8th of its size overallocated, or  
112 elements plus 9 words overhead is 121 words. Better than any cons- 
linked list could be, and *insanely better* than a cons-linked list  
would be in python.

James

From facundobatista at gmail.com  Mon Dec 26 18:31:10 2005
From: facundobatista at gmail.com (Facundo Batista)
Date: Mon, 26 Dec 2005 14:31:10 -0300
Subject: [Python-Dev] NotImplemented reaching top-level
In-Reply-To: <20051226162710.GA19675@code1.codespeak.net>
References: <e04bdf310512240931t1049346n@mail.gmail.com>
	<20051225113550.GA28497@code1.codespeak.net>
	<dom0ah$f29$1@sea.gmane.org>
	<20051225162325.GA32552@code1.codespeak.net>
	<43AECB86.50206@gmail.com>
	<20051226162710.GA19675@code1.codespeak.net>
Message-ID: <e04bdf310512260931p2a7d37c6o@mail.gmail.com>

2005/12/26, Armin Rigo <arigo at tunes.org>:

> Done in patch #1390657.

Fantastic, Armin, thank you!


> nb_add and nb_multiply should be tried.  I don't think that this would
> break existing C or Python code, but it should probably only go in 2.5,
> together with the patch #1390657 that relies on it.

It'd be good to know if this will be applied for the next version
2.4.x or will wait until 2.4.5, for me to search a workaround in
Decimal or not (really don't know if I can find a solution here).

Thank you!

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From tim.peters at gmail.com  Mon Dec 26 18:56:27 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Mon, 26 Dec 2005 12:56:27 -0500
Subject: [Python-Dev] deque alternative (was: Linked lists)
In-Reply-To: <39C8CCEF-21D9-4DB2-AE03-374748C6CDB7@fuhm.net>
References: <8393fff0512251810q6aa5d583s2733e265b4bcd648@mail.gmail.com>
	<1f7befae0512251946y31552befx468d1f7dda23b6e1@mail.gmail.com>
	<20051225204328.BECE.JCARLSON@uci.edu>
	<8393fff0512260807y53a73baar197e07cf8b8eaa8c@mail.gmail.com>
	<39C8CCEF-21D9-4DB2-AE03-374748C6CDB7@fuhm.net>
Message-ID: <1f7befae0512260956o695150e7x5a56c0d8972d1317@mail.gmail.com>

[restoring context and attributions lost in the original]

[Tim Peters]
>>>> Like Phillip Eby, I use 2-tuples for this when I feel the need
>>>> (usually during a backtracking graph search, to keep track of paths
>>>> back to the root in a space-efficient way), and am happy with that.

[Josiah Carlson]
>>> Then there's the whole Python list with append() and pop().  It
>>> takes a method resolution and function call, but at least in
>>> Python 2.3, it is a hair faster...

[Martin Blais]
>> Josiah, that's beside the point, because it is not space-efficient,
>> the list is actually an dynamic array/vector, and I would expect an
>> efficient array implementation to grow exponentially.  One of the
>> reasons we're talking about lists is to avoid exactly this problem...

[James Y Knight]
> Okay, now *that* reason is a pretty crazy one. Consider what you're
> saying here.

I'm sure he did ;-)  Consider a very simple graph, a skinny tree
rooted at `A` that branches once "at the end", represented by a dict
of adjacency lists:

kids[A] = [B]
kids[B] = [C]
kids[C] = [D]
...
kids[W] = [X]
kids[X] = [Y, Z]

A natural representation of the path from B back to the root is:

    Bpath = B, (A, None)

and from C back to the root:

    Cpath = C, Bpath

and from D back to the root:

    Dpath = D, Cpath

...

and from X back to the root:

    Xpath = X, Wpath

A "flat" list or tuple would certainly be more space-efficient up to
this point.  But when the graph branches, the 2-tuple representation
allows *sharing* the common path suffix (which may be very long!):

    Ypath = Y, Xpath
and
    Zpath = Z, Xpath

If there's an N-way branch, using 2-tuples allows recording the N new
paths back to the root with incremental memory burden N *
some_constant.  You can't do this kind of sharing with a flat list or
tuple, and the incremental memory burden at an N-way branch zooms to
(N + some_other_constant) * (the amount of memory needed to record the
path from the branch point back to the root), due to duplicating the
back-to-the-root info N times.   The potential memory saving from
using 2-tuples instead is unbounded.

...

> Plus, in python, the overhead per object in the pyobject memory allocation
> system, which I don't know how to quantify.

For objects requiring a number of bytes divisible by 8, a few percent
at worst.  The per-object space overhead in obmalloc consists entirely
of internal padding needed to reach a multiple of 8 bytes per
allocation unit, and that's 0 when the # of bytes needed is divisible
by 8.  The only obmalloc space overheads then are due to per-pool and
per-arena bookkeeping space, and those are small fractions of total
pool and arena sizes.

...

> So, the list will generally be 1/8th of its size overallocated, or
> 112 elements plus 9 words overhead is 121 words. Better than any cons-
> linked list could be, and *insanely better* than a cons-linked list
> would be in python.

You seem to be ignoring possiblities for sharing across lists, and
such sharing is natural in many graph algorithms.

From rwgk at cci.lbl.gov  Mon Dec 26 18:59:43 2005
From: rwgk at cci.lbl.gov (Ralf W. Grosse-Kunstleve)
Date: Mon, 26 Dec 2005 09:59:43 -0800 (PST)
Subject: [Python-Dev] Python + Visual C++ 8.0?
Message-ID: <200512261759.jBQHxhZv203530@boa.lbl.gov>

We have a bunch of C++ extensions (Boost.Python) that work fine under
Windows when compiled with Visual C++ 7.1. With a few minor tweaks
all extensions also compile with Visual C++ 8.0, but trying to "import"
any of the extensions fails with this message (shown in a pop-up box):

  This application has failed to start because MSVCP80.dll was
  not found. Re-installing the application may fix this problem.

I am using a Visual Studio 2005 Professional installation. I also ran
vcredist_x86.exe. Moving msvc[mpr]80.dll to a directory on PATH didn't
help. However, standalone executables work OK without any gymnastics.
Does anyone know what the problem could be with the extensions?

A quick attempt to compile Python from scratch using Visual C++ 8.0
produced a python.exe, but it doesn't run (the debug / send report /
don't send report box pops up). Has someone tried this before?

Cheers,
        Ralf

From jcarlson at uci.edu  Mon Dec 26 20:04:15 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 26 Dec 2005 11:04:15 -0800
Subject: [Python-Dev] deque alternative (was: Linked lists)
In-Reply-To: <1f7befae0512260956o695150e7x5a56c0d8972d1317@mail.gmail.com>
References: <39C8CCEF-21D9-4DB2-AE03-374748C6CDB7@fuhm.net>
	<1f7befae0512260956o695150e7x5a56c0d8972d1317@mail.gmail.com>
Message-ID: <20051226103320.BED1.JCARLSON@uci.edu>


> [restoring context and attributions lost in the original]
> 
> [Tim Peters]
> >>>> Like Phillip Eby, I use 2-tuples for this when I feel the need
> >>>> (usually during a backtracking graph search, to keep track of paths
> >>>> back to the root in a space-efficient way), and am happy with that.
> 
> [Josiah Carlson]
> >>> Then there's the whole Python list with append() and pop().  It
> >>> takes a method resolution and function call, but at least in
> >>> Python 2.3, it is a hair faster...
> 
> [Martin Blais]
> >> Josiah, that's beside the point, because it is not space-efficient,
> >> the list is actually an dynamic array/vector, and I would expect an
> >> efficient array implementation to grow exponentially.  One of the
> >> reasons we're talking about lists is to avoid exactly this problem...
> 
> [James Y Knight]
> > Okay, now *that* reason is a pretty crazy one. Consider what you're
> > saying here.

[Tim Peters]
> I'm sure he did ;-)  Consider a very simple graph, a skinny tree
> rooted at `A` that branches once "at the end", represented by a dict
> of adjacency lists:

Are you sure?

> A "flat" list or tuple would certainly be more space-efficient up to
> this point.  But when the graph branches, the 2-tuple representation
> allows *sharing* the common path suffix (which may be very long!):
...
> If there's an N-way branch, using 2-tuples allows recording the N new
> paths back to the root with incremental memory burden N *
> some_constant.  You can't do this kind of sharing with a flat list or
> tuple, and the incremental memory burden at an N-way branch zooms to
> (N + some_other_constant) * (the amount of memory needed to record the
> path from the branch point back to the root), due to duplicating the
> back-to-the-root info N times.   The potential memory saving from
> using 2-tuples instead is unbounded.

But one doesn't _need_ to use flat lists.  If one were to combine cons
cells with Python lists, you can get the space efficiency of lists with
the reusability of the desired cons cells (or tuples as the case may be).
How? (i, l), where i is the prefix of list l which is the desired
portion of whatever l represents.  You toss one of those anywhere in
your 'flat list', and you have your stored/saved path with a couple
dozen bytes. If you are not careful, you end up storing/saving paths
which would be stored more efficiently by copying the contents, but at
that point we are talking about a constant factor blowup, not the
(potentially) exponential blowup of storing all paths as copies, and we
can always copy to be more efficient if necessary.

I have personally used this trick to construct a union-find data
structure in which all unions are reversable regardless of find
operation. [1]


> > So, the list will generally be 1/8th of its size overallocated, or
> > 112 elements plus 9 words overhead is 121 words. Better than any cons-
> > linked list could be, and *insanely better* than a cons-linked list
> > would be in python.
> 
> You seem to be ignoring possiblities for sharing across lists, and
> such sharing is natural in many graph algorithms.

Not necessarily so as I have pointed out above.  You said yourself;
practicality beats purity.  While using cons cells are pure, as us using
only flat lists are pure, flat lists are practically smaller than cons
cells in certain cases (by a factor of ~4), and non-flat lists can be
smaller than cons cells in the rest of the cases.

 - Josiah

[1] If you remember, the ammortized running time of O(n) unions
and finds on a union-find data structure is O(n*alpha(n,n)), where
alpha(n,n) is never larger than 5 in practice.  The space overhead of
using non-flat lists as shared paths provides sufficient information to
offer reversable union operations in ammortized O(n*alpha(n,n)) space.


From fredrik at pythonware.com  Mon Dec 26 19:53:27 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 26 Dec 2005 19:53:27 +0100
Subject: [Python-Dev] that library reference, again
Message-ID: <dope79$flv$2@sea.gmane.org>

as seen on the doc-sig:

> > > javadoc's
> > >
> > >     {@link os.popen}
> > >
> > > is even shorter.
> > >
> > > hmm.  maybe a combination of rest/html/whatever and pythondoc markup
> > > would be the ultimate tool for the library reference...
> >
> > <* heavy clapping sound of my footsteps while running away screaming in fear *>
>
> really?

since everything that has people running in fear is worth investigating
further, I've spent a few hours putting together a first iteration of a
"python.org library reference to a javadoc-ish source format converter".

more info here:

    http://effbot.org/zone/pythondoc-lib.htm

including goals, non-goals, and 2.5 megabytes of converted (but not
yet properly rendered) library pages.

</F>




From tim.peters at gmail.com  Mon Dec 26 20:32:13 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Mon, 26 Dec 2005 14:32:13 -0500
Subject: [Python-Dev] deque alternative (was: Linked lists)
In-Reply-To: <20051226103320.BED1.JCARLSON@uci.edu>
References: <39C8CCEF-21D9-4DB2-AE03-374748C6CDB7@fuhm.net>
	<1f7befae0512260956o695150e7x5a56c0d8972d1317@mail.gmail.com>
	<20051226103320.BED1.JCARLSON@uci.edu>
Message-ID: <1f7befae0512261132s6e6dfec7wd58f3357406f3502@mail.gmail.com>

...

[Tim Peters]
>> I'm sure he did ;-)  Consider a very simple graph, a skinny tree
>> rooted at `A` that branches once "at the end", represented by a dict
>> of adjacency lists:

[Josiah Carlson]
> Are you sure?

Of what?

> ...

> But one doesn't _need_ to use flat lists.

Of course not.  You didn't mention that in the message of yours to
which Martin replied, and neither did James mention it in his message
to which I replied, and you shouldn't feel put upon that people
respond to what's actually written ;-)

[Tim, to James]
>> You seem to be ignoring possiblities for sharing across lists, and
>> such sharing is natural in many graph algorithms.

> Not necessarily so as I have pointed out above.

Gimme a break.  James didn't mention any such thing, and I was replying to him.

> You said yourself; practicality beats purity.  While using cons cells are pure, as
> us using only flat lists are pure, flat lists are practically smaller than cons
> cells in certain cases (by a factor of ~4), and non-flat lists can be
> smaller than cons cells in the rest of the cases.

Yes.  Nevertheless, most people are going to use flats lists or pure
two-tuples in practice, simply because the coding for each is very
simple, and the idea that using flat lists "is better" was worth
opposing (because it isn't always true, and isn't even commonly true
in many graph algorithms, but was so strongly advanced in the message
to which I replied that using two-tuples instead was characterized as
"pretty crazy").

From pje at telecommunity.com  Mon Dec 26 20:35:39 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 26 Dec 2005 14:35:39 -0500
Subject: [Python-Dev] deque alternative (was: Linked lists)
In-Reply-To: <20051226103320.BED1.JCARLSON@uci.edu>
References: <1f7befae0512260956o695150e7x5a56c0d8972d1317@mail.gmail.com>
	<39C8CCEF-21D9-4DB2-AE03-374748C6CDB7@fuhm.net>
	<1f7befae0512260956o695150e7x5a56c0d8972d1317@mail.gmail.com>
Message-ID: <5.1.1.6.0.20051226142836.04400d40@mail.telecommunity.com>

At 11:04 AM 12/26/2005 -0800, Josiah Carlson wrote:
>Not necessarily so as I have pointed out above.  You said yourself;
>practicality beats purity.  While using cons cells are pure, as us using
>only flat lists are pure, flat lists are practically smaller than cons
>cells in certain cases (by a factor of ~4), and non-flat lists can be
>smaller than cons cells in the rest of the cases.

The reason I sometimes use 2-tuples as cons cells is that it's more 
convenient and/or intuitive than the alternatives for some recursive 
algorithms.  That's the practicality I'm interested in.  Copying a list or 
doing some other hokey-pokey data structure in such cases would just be 
annoying and make the code harder to follow.

Memory consumption and performance were never my reasons for 
cons-ing.  Some algorithms just read more cleanly that way.  In any case, 
regarding the usefulness of a built-in cons type, you're preaching to the 
choir here.  My whole point here is that when you *do* need a lispy list, 
two-item tuples are *the* One Obvious Way to do it in Python, because the 
existing syntax makes them effortless to use.  There's no need for a new 
built-in, 'cause what we already have works great.


From bcannon at gmail.com  Mon Dec 26 21:02:01 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Mon, 26 Dec 2005 12:02:01 -0800
Subject: [Python-Dev] that library reference, again
In-Reply-To: <dope79$flv$2@sea.gmane.org>
References: <dope79$flv$2@sea.gmane.org>
Message-ID: <bbaeab100512261202r329ef493r4605d31356ff2dfe@mail.gmail.com>

On 12/26/05, Fredrik Lundh <fredrik at pythonware.com> wrote:
> as seen on the doc-sig:
>
> > > > javadoc's
> > > >
> > > >     {@link os.popen}
> > > >
> > > > is even shorter.
> > > >
> > > > hmm.  maybe a combination of rest/html/whatever and pythondoc markup
> > > > would be the ultimate tool for the library reference...
> > >
> > > <* heavy clapping sound of my footsteps while running away screaming in fear *>
> >
> > really?
>
> since everything that has people running in fear is worth investigating
> further, I've spent a few hours putting together a first iteration of a
> "python.org library reference to a javadoc-ish source format converter".
>
> more info here:
>
>     http://effbot.org/zone/pythondoc-lib.htm
>
> including goals, non-goals, and 2.5 megabytes of converted (but not
> yet properly rendered) library pages.
>

Not specifically advocating the solution, but the markup looks
reasonable and easy to use.  Thanks for taking a stab at this, /F.

-Brett

From noamraph at gmail.com  Mon Dec 26 23:16:19 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Tue, 27 Dec 2005 00:16:19 +0200
Subject: [Python-Dev] A few questions about setobject
Message-ID: <b348a0850512261416s5865462btf08a9613f0780d36@mail.gmail.com>

Hello,

I'm going over setobject.c/setobject.h, while trying to change them to
support cheap frozen-copying. I have a few questions.

1) This is a part of setobject.h:

typedef struct {
	long hash;
	PyObject *key;
} setentry;

typedef struct _setobject PySetObject;
struct _setobject {
	...
	setentry *table;
	setentry *(*lookup)(PySetObject *so, PyObject *key, long hash);
	setentry smalltable[PySet_MINSIZE];
	...
};

It seems to me that setentry and _setobject are defined for every file
that includes Python.h. In the Python C API, in the section about
include files, it is written that:
"All user visible names defined by Python.h (except those defined by
the included standard headers) have one of the prefixes "Py" or "_Py".
Names beginning with "_Py" are for internal use by the Python
implementation and should not be used by extension writers. Structure
member names do not have a reserved prefix."

Is this desirable? Even if it is, it seems that the second sentence
contradicts the first sentence. Perhaps the header file should stick
with writing "struct { long hash; PyObject *key; }" three times (or
define it in a macro and then undefine it), and the typedef be left to
the .c file?


2) The hash table used by sets uses a dummy element for deleted
entries. The implementation goes into the trouble of allocating it,
managing its reference count, and deallocating it at the end. What is
the reason for that? It seems to me that the only requirement of the
dummy element is that it shouldn't be a pointer to a valid PyObject,
and as such I would think that defining it like

int dummy_int;
PyObject *dummy = (PyObject *)(&dummy_int);

would be enough, and that it shouldn't be INCREFed or DECREFed every
time it is used. I think it should be ok because it's never used
really as a PyObject. Am I missing something? (Ok, I now thought that
maybe it's because some parts don't treat dummy elements specially.
But it seems to me that most parts do treat them specially, so perhaps
it would be better to make a few changes so that all parts will treat
them specially?)


3) The type of the result of a binary operator applied on a set and a
frozenset is the type of the left set. You are welcomed to ignore
this, but I just wanted to say that it seems to me better to make the
operator symmetric, and to return a frozenset only if both sets are
frozen.


If you think that these questions belong to c.l.py, then please say so
and I will go away.

Have a good day,
Noam

From martin at v.loewis.de  Mon Dec 26 23:18:39 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 26 Dec 2005 23:18:39 +0100
Subject: [Python-Dev] Python + Visual C++ 8.0?
In-Reply-To: <200512261759.jBQHxhZv203530@boa.lbl.gov>
References: <200512261759.jBQHxhZv203530@boa.lbl.gov>
Message-ID: <43B06C3F.4030203@v.loewis.de>

Ralf W. Grosse-Kunstleve wrote:
> I am using a Visual Studio 2005 Professional installation. I also ran
> vcredist_x86.exe. Moving msvc[mpr]80.dll to a directory on PATH didn't
> help. However, standalone executables work OK without any gymnastics.
> Does anyone know what the problem could be with the extensions?

Can't check right now - but could it be that the standalone executables
get an extra copy of this library in their binary directory, as part
of the build process?

> A quick attempt to compile Python from scratch using Visual C++ 8.0
> produced a python.exe, but it doesn't run (the debug / send report /
> don't send report box pops up). Has someone tried this before?

Yes; a patch to fix this problem has been checked into the trunk
(but I have no plans for backporting it to the 2.4 branch, as 2.4
will be compiled with VC7.1 "forever" - i.e. until its final release).

Regards,
Martin

P.S. I currently also plan to build Python 2.5 with VC 7.1.
P.P.S. You do know that this configuration (extension compiled
with VS2005, Python compiled wit VS.NET2003) is not supported,
right?

From tismer at stackless.com  Tue Dec 27 01:07:36 2005
From: tismer at stackless.com (Christian Tismer)
Date: Tue, 27 Dec 2005 01:07:36 +0100
Subject: [Python-Dev] deque alternative (was: Linked lists)
In-Reply-To: <5.1.1.6.0.20051226142836.04400d40@mail.telecommunity.com>
References: <1f7befae0512260956o695150e7x5a56c0d8972d1317@mail.gmail.com>	<39C8CCEF-21D9-4DB2-AE03-374748C6CDB7@fuhm.net>	<1f7befae0512260956o695150e7x5a56c0d8972d1317@mail.gmail.com>
	<5.1.1.6.0.20051226142836.04400d40@mail.telecommunity.com>
Message-ID: <43B085C8.3020608@stackless.com>

Hi all,

not addressing anybody directly here,
but this thread is about my dequeue question.
It would just be nice if you could use the original thread
topic or a different one to discuss the original question.

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From fredrik at pythonware.com  Tue Dec 27 12:40:50 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 27 Dec 2005 12:40:50 +0100
Subject: [Python-Dev] a quit that actually quits
Message-ID: <dor982$qrc$1@sea.gmane.org>

sourceforge just went off the air, so I'm posting this patch here, in order
to distract you all from Christian's deque thread.

this silly little patch changes the behaviour of the interpreter so that "quit"
and "exit" actually exits the interpreter.  it does this by installing a custom
excepthook that looks for NameErrors at the top level, in interactive mode
only.

whaddya think?

</F>

Index: Lib/site.py
===================================================================
--- Lib/site.py (revision 41831)
+++ Lib/site.py (working copy)
@@ -60,6 +60,7 @@
 import sys
 import os
 import __builtin__
+import traceback


 def makepath(*paths):
@@ -222,19 +223,20 @@


 def setquit():
-    """Define new built-ins 'quit' and 'exit'.
-    These are simply strings that display a hint on how to exit.
+    """Set default exception handler for the interactive mode."""
+    def defaultexcepthook(exc_type, exc_value, exc_info):
+        if hasattr(sys, "ps1"):
+            # interactive mode
+            if isinstance(exc_value, NameError) and not exc_info.tb_next:
+                text = exc_value[0]
+                if (text == "name 'exit' is not defined" or
+                    text == "name 'quit' is not defined"):
+                    # XXX: print helpful "Use control-D etc" message here?
+                    raise SystemExit
+                # XXX: add if text == "help" ?
+        traceback.print_exception(exc_type, exc_value, exc_info)
+    sys.excepthook = defaultexcepthook

-    """
-    if os.sep == ':':
-        exit = 'Use Cmd-Q to quit.'
-    elif os.sep == '\\':
-        exit = 'Use Ctrl-Z plus Return to exit.'
-    else:
-        exit = 'Use Ctrl-D (i.e. EOF) to exit.'
-    __builtin__.quit = __builtin__.exit = exit
-
-
 class _Printer(object):
     """interactive prompt objects for printing the license text, a list of
     contributors and the copyright notice."""




From fredrik at pythonware.com  Tue Dec 27 13:18:27 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 27 Dec 2005 13:18:27 +0100
Subject: [Python-Dev] file() vs open(), round 7
References: <20051226043831.GA24197@panix.com>
Message-ID: <dorbej$9d$1@sea.gmane.org>

Aahz wrote:

> class file(object)
> |  file(name[, mode[, buffering]]) -> file object
> |
> |  Open a file.  The mode can be 'r', 'w' or 'a' for reading (default),
> [...]
> |  Note:  open() is an alias for file().
>
> This is confusing.  I suggest that we make ``open()`` a factory function
> right now.  (I'll submit a bug report (and possibly a patch) after I get
> agreement.)

+1.

can we add a opentext factory for file/codecs.open while we're at it ?

</F> 




From skip at pobox.com  Tue Dec 27 14:07:01 2005
From: skip at pobox.com (skip@pobox.com)
Date: Tue, 27 Dec 2005 07:07:01 -0600
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <dor982$qrc$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org>
Message-ID: <17329.15477.267537.300546@montanaro.dyndns.org>


    Fredrik> whaddya think?

We're going to wind up with the same problem that spawned the atexit module:
multiple code sources wanting to fiddle with sys.excepthook step on one
another's toes.  For example, I already use an excepthook in interactive
mode to try to resolve NameErrors:

    % python
    Python 2.5a0 (#2, Dec 10 2005, 14:05:27) 
    [GCC 3.3 20030304 (Apple Computer, Inc. build 1671)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> sys.excepthook
    <function _autoload_exc at 0x3d6830>
    >>> sin(12)
    found sin in math module
    -0.53657291800043505

Other than that, seems fine to me.

Skip

From fredrik at pythonware.com  Tue Dec 27 14:06:21 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 27 Dec 2005 14:06:21 +0100
Subject: [Python-Dev] status of development documentation
References: <20051221095628.BE79.JCARLSON@uci.edu><20051221105132.BE7C.JCARLSON@uci.edu>
	<docare$tjs$1@sea.gmane.org><1135196162.14495.15.camel@geddy.wooz.org><doci50$nom$1@sea.gmane.org>
	<2m4q512vub.fsf@starship.python.net><1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net><dogduu$o1m$1@sea.gmane.org><5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net><17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
Message-ID: <dore8d$71g$1@sea.gmane.org>

Neal Norwitz wrote:

> And hopefully of interest to many here:
>
>    http://docs.python.org/dev/results/
>
> These are the results of svn update, configure, build, test, install
> and the doc run.

the "trunk" link on

    http://www.python.org/dev/doc/

still points to the old

    http://www.python.org/dev/doc/devel/

rather than the new

    http://docs.python.org/dev/

</F> 




From mal at egenix.com  Tue Dec 27 14:20:23 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 27 Dec 2005 14:20:23 +0100
Subject: [Python-Dev] file() vs open(), round 7
In-Reply-To: <dorbej$9d$1@sea.gmane.org>
References: <20051226043831.GA24197@panix.com> <dorbej$9d$1@sea.gmane.org>
Message-ID: <43B13F97.5040509@egenix.com>

Fredrik Lundh wrote:
> Aahz wrote:
> 
>> class file(object)
>> |  file(name[, mode[, buffering]]) -> file object
>> |
>> |  Open a file.  The mode can be 'r', 'w' or 'a' for reading (default),
>> [...]
>> |  Note:  open() is an alias for file().
>>
>> This is confusing.  I suggest that we make ``open()`` a factory function
>> right now.  (I'll submit a bug report (and possibly a patch) after I get
>> agreement.)
> 
> +1.
> 
> can we add a opentext factory for file/codecs.open while we're at it ?

Why a new factory function ? Can't we just redirect to codecs.open()
in case an encoding keyword argument is passed to open() ?!

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 27 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From fredrik at pythonware.com  Tue Dec 27 14:35:29 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 27 Dec 2005 14:35:29 +0100
Subject: [Python-Dev] file() vs open(), round 7
References: <20051226043831.GA24197@panix.com> <dorbej$9d$1@sea.gmane.org>
	<43B13F97.5040509@egenix.com>
Message-ID: <dorfv1$bak$1@sea.gmane.org>

M.-A. Lemburg wrote:

>> can we add a opentext factory for file/codecs.open while we're at it ?
>
> Why a new factory function ? Can't we just redirect to codecs.open()
> in case an encoding keyword argument is passed to open() ?!

I think open is overloaded enough as it is.  Using separate functions for distinct
use cases is also a lot better than keyword trickery.

Here's a rough draft:

    def textopen(name, mode="r", encoding=None):
        if "U" not in mode:
            mode += "U"
        if encoding:
            return codecs.open(name, mode, encoding)
        return file(name, mode)

</F> 




From reinhold-birkenfeld-nospam at wolke7.net  Tue Dec 27 14:42:23 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Tue, 27 Dec 2005 14:42:23 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <dor982$qrc$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org>
Message-ID: <dorgc0$ca9$1@sea.gmane.org>

Fredrik Lundh wrote:
> sourceforge just went off the air, so I'm posting this patch here, in order
> to distract you all from Christian's deque thread.
> 
> this silly little patch changes the behaviour of the interpreter so that "quit"
> and "exit" actually exits the interpreter.  it does this by installing a custom
> excepthook that looks for NameErrors at the top level, in interactive mode
> only.

What is wrong with something like this:

 >>> class Quitter:
...  def __repr__(self): raise SystemExit
...
 >>> exit = quit = Quitter()

It could optionally check for top level too, of course.

Reinhold

-- 
Mail address is perfectly valid!


From fredrik at pythonware.com  Tue Dec 27 14:57:29 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 27 Dec 2005 14:57:29 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
Message-ID: <dorh89$egi$1@sea.gmane.org>

Reinhold Birkenfeld wrote:

> What is wrong with something like this:
>
> >>> class Quitter:
> ...  def __repr__(self): raise SystemExit
> ...
> >>> exit = quit = Quitter()

>>> vars() # oops!

</F> 




From radeex at gmail.com  Tue Dec 27 14:55:27 2005
From: radeex at gmail.com (Christopher Armstrong)
Date: Wed, 28 Dec 2005 00:55:27 +1100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <dorgc0$ca9$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
Message-ID: <60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>

On 12/28/05, Reinhold Birkenfeld <reinhold-birkenfeld-nospam at wolke7.net> wrote:
> Fredrik Lundh wrote:
> > sourceforge just went off the air, so I'm posting this patch here, in order
> > to distract you all from Christian's deque thread.
> >
> > this silly little patch changes the behaviour of the interpreter so that "quit"
> > and "exit" actually exits the interpreter.  it does this by installing a custom
> > excepthook that looks for NameErrors at the top level, in interactive mode
> > only.
>
> What is wrong with something like this:
>
>  >>> class Quitter:
> ...  def __repr__(self): raise SystemExit
> ...
>  >>> exit = quit = Quitter()
>
> It could optionally check for top level too, of course.


Not sure if this is what you mean by "check for top level too", but
the obvious problem is that calling vars(__builtins__) (or similar)
will cause your interpreter to exit. :)


--
  Twisted   |  Christopher Armstrong: International Man of Twistery
   Radix    |    -- http://radix.twistedmatrix.com
            |  Release Manager, Twisted Project
  \\\V///   |    -- http://twistedmatrix.com
   |o O|    |
w----v----w-+

From mal at egenix.com  Tue Dec 27 15:04:22 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 27 Dec 2005 15:04:22 +0100
Subject: [Python-Dev] file() vs open(), round 7
In-Reply-To: <dorfv1$bak$1@sea.gmane.org>
References: <20051226043831.GA24197@panix.com>
	<dorbej$9d$1@sea.gmane.org>	<43B13F97.5040509@egenix.com>
	<dorfv1$bak$1@sea.gmane.org>
Message-ID: <43B149E6.9090205@egenix.com>

Fredrik Lundh wrote:
> M.-A. Lemburg wrote:
> 
>>> can we add a opentext factory for file/codecs.open while we're at it ?
>> Why a new factory function ? Can't we just redirect to codecs.open()
>> in case an encoding keyword argument is passed to open() ?!
> 
> I think open is overloaded enough as it is.  Using separate functions for distinct
> use cases is also a lot better than keyword trickery.

Fair enough.

> Here's a rough draft:
> 
>     def textopen(name, mode="r", encoding=None):
>         if "U" not in mode:
>             mode += "U"

The "U" is not needed when opening files using codecs -
these always break lines using .splitlines() which
breaks lines according to the Unicode rules and also
knows about the various line break variants on different
platforms.

>         if encoding:
>             return codecs.open(name, mode, encoding)
>         return file(name, mode)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 27 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From reinhold-birkenfeld-nospam at wolke7.net  Tue Dec 27 15:41:11 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Tue, 27 Dec 2005 15:41:11 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <dorh89$egi$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<dorh89$egi$1@sea.gmane.org>
Message-ID: <dorjq7$ksd$1@sea.gmane.org>

Fredrik Lundh wrote:
> Reinhold Birkenfeld wrote:
> 
>> What is wrong with something like this:
>>
>> >>> class Quitter:
>> ...  def __repr__(self): raise SystemExit
>> ...
>> >>> exit = quit = Quitter()
> 
>>>> vars() # oops!

You're right.

 >>> class Quitter:
...  def __repr__(self):
...   n = sys._getframe(1).f_code.co_names
...   if n == ("exit",) or n == ("quit",):
...    raise SystemExit


better? ;)

Reinhold


-- 
Mail address is perfectly valid!


From ronaldoussoren at mac.com  Tue Dec 27 15:27:52 2005
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 27 Dec 2005 15:27:52 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
Message-ID: <56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>


On 27-dec-2005, at 14:55, Christopher Armstrong wrote:

> On 12/28/05, Reinhold Birkenfeld <reinhold-birkenfeld- 
> nospam at wolke7.net> wrote:
>> Fredrik Lundh wrote:
>>> sourceforge just went off the air, so I'm posting this patch  
>>> here, in order
>>> to distract you all from Christian's deque thread.
>>>
>>> this silly little patch changes the behaviour of the interpreter  
>>> so that "quit"
>>> and "exit" actually exits the interpreter.  it does this by  
>>> installing a custom
>>> excepthook that looks for NameErrors at the top level, in  
>>> interactive mode
>>> only.
>>
>> What is wrong with something like this:
>>
>>>>> class Quitter:
>> ...  def __repr__(self): raise SystemExit
>> ...
>>>>> exit = quit = Quitter()
>>
>> It could optionally check for top level too, of course.
>
>
> Not sure if this is what you mean by "check for top level too", but
> the obvious problem is that calling vars(__builtins__) (or similar)
> will cause your interpreter to exit. :)

Why must quit and exit be so special in the first place? They could  
be plain functions,
or even something like::

	class _QuitOrExit:
		def __init__(self, name):
			self.name = name

		def __repr__(self):
			return "Use %(name)s() to exit."%(self.__dict__)

		def __call__(self):
			raise SystemExit

	quit = _QuitOrExit("quit")
	exit = _QuitOrExit("exit")


Ronald


From pje at telecommunity.com  Tue Dec 27 15:59:45 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 27 Dec 2005 09:59:45 -0500
Subject: [Python-Dev] file() vs open(), round 7
In-Reply-To: <dorfv1$bak$1@sea.gmane.org>
References: <20051226043831.GA24197@panix.com> <dorbej$9d$1@sea.gmane.org>
	<43B13F97.5040509@egenix.com>
Message-ID: <5.1.1.6.0.20051227095611.02241f78@mail.telecommunity.com>

At 02:35 PM 12/27/2005 +0100, Fredrik Lundh wrote:
>M.-A. Lemburg wrote:
>
> >> can we add a opentext factory for file/codecs.open while we're at it ?
> >
> > Why a new factory function ? Can't we just redirect to codecs.open()
> > in case an encoding keyword argument is passed to open() ?!
>
>I think open is overloaded enough as it is.  Using separate functions for 
>distinct
>use cases is also a lot better than keyword trickery.
>
>Here's a rough draft:
>
>     def textopen(name, mode="r", encoding=None):
>         if "U" not in mode:
>             mode += "U"
>         if encoding:
>             return codecs.open(name, mode, encoding)
>         return file(name, mode)

Nice. It should probably also check whether there's a 'b' or 't' in 'mode' 
and raise an error if so.  I'd also prefer to call it 'textfile', as that 
reads more nicely with "for line in textfile(...):" use cases, and it does 
return a file object.


From arigo at tunes.org  Tue Dec 27 16:12:54 2005
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 27 Dec 2005 16:12:54 +0100
Subject: [Python-Dev] deque alternative
In-Reply-To: <43AFE44D.6040306@stackless.com>
References: <8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>
	<ca471dc20512141237i4f09098cudb951fbe70623085@mail.gmail.com>
	<43AE96FF.2010303@stackless.com>
	<8393fff0512251810q6aa5d583s2733e265b4bcd648@mail.gmail.com>
	<43AFE44D.6040306@stackless.com>
Message-ID: <20051227151254.GA6985@code1.codespeak.net>

Hi Christian,

On Mon, Dec 26, 2005 at 01:38:37PM +0100, Christian Tismer wrote:
> I don't think your code has to decide about this. The power lies
> in the fact that you don't specify that, but just use the list
> in a different way. We do this in the PyPy implementation;
> right now it is true that we have a static analysis, but a JIT
> is to come, and I'm pretty sure it will try to use an array
> until something gets used like a list.

You are mentioning confusingly many levels of PyPy for this argument.
This is not directly related to static analysis nor to the JIT.  The
point is just that while a Python program runs, the implementation can
decide to start using a deque-like structure instead of a zero-based
array for a given user list.  This can be done in any implementation of
Python; both in PyPy and in CPython it would be done by adding checks
and cases in the code that implements list objects.

As much as I like this approach I fear that it will be rejected for
CPython, as e.g. lazily concatenated string objects were, on grounds of
code obfuscation and unpredictability of performance.  But it's PyPy's
goal to experiment here :-)


A bientot,

Armin

From mal at egenix.com  Tue Dec 27 16:20:41 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 27 Dec 2005 16:20:41 +0100
Subject: [Python-Dev] file() vs open(), round 7
In-Reply-To: <5.1.1.6.0.20051227095611.02241f78@mail.telecommunity.com>
References: <20051226043831.GA24197@panix.com>
	<dorbej$9d$1@sea.gmane.org>	<43B13F97.5040509@egenix.com>
	<5.1.1.6.0.20051227095611.02241f78@mail.telecommunity.com>
Message-ID: <43B15BC9.4070104@egenix.com>

Phillip J. Eby wrote:
> At 02:35 PM 12/27/2005 +0100, Fredrik Lundh wrote:
>> M.-A. Lemburg wrote:
>>
>>>> can we add a opentext factory for file/codecs.open while we're at it ?
>>> Why a new factory function ? Can't we just redirect to codecs.open()
>>> in case an encoding keyword argument is passed to open() ?!
>> I think open is overloaded enough as it is.  Using separate functions for 
>> distinct
>> use cases is also a lot better than keyword trickery.
>>
>> Here's a rough draft:
>>
>>     def textopen(name, mode="r", encoding=None):
>>         if "U" not in mode:
>>             mode += "U"
>>         if encoding:
>>             return codecs.open(name, mode, encoding)
>>         return file(name, mode)
> 
> Nice. It should probably also check whether there's a 'b' or 't' in 'mode' 
> and raise an error if so. 

Why should it do that ?

FYI: codecs.open() explicitly adds the 'b' to the mode since
we don't want the platform text mode interfere with the
Unicode line breaking.

> I'd also prefer to call it 'textfile', as that 
> reads more nicely with "for line in textfile(...):" use cases, and it does 
> return a file object.

Nope: open() is only guaranteed to return a file-like object,
e.g. codecs.open() will return a wrapped version of a file
object.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 27 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From pje at telecommunity.com  Tue Dec 27 16:39:58 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 27 Dec 2005 10:39:58 -0500
Subject: [Python-Dev] file() vs open(), round 7
In-Reply-To: <43B15BC9.4070104@egenix.com>
References: <5.1.1.6.0.20051227095611.02241f78@mail.telecommunity.com>
	<20051226043831.GA24197@panix.com> <dorbej$9d$1@sea.gmane.org>
	<43B13F97.5040509@egenix.com>
	<5.1.1.6.0.20051227095611.02241f78@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051227103405.04344180@mail.telecommunity.com>

At 04:20 PM 12/27/2005 +0100, M.-A. Lemburg wrote:
>Phillip J. Eby wrote:
> > At 02:35 PM 12/27/2005 +0100, Fredrik Lundh wrote:
> >> M.-A. Lemburg wrote:
> >>
> >>>> can we add a opentext factory for file/codecs.open while we're at it ?
> >>> Why a new factory function ? Can't we just redirect to codecs.open()
> >>> in case an encoding keyword argument is passed to open() ?!
> >> I think open is overloaded enough as it is.  Using separate functions for
> >> distinct
> >> use cases is also a lot better than keyword trickery.
> >>
> >> Here's a rough draft:
> >>
> >>     def textopen(name, mode="r", encoding=None):
> >>         if "U" not in mode:
> >>             mode += "U"
> >>         if encoding:
> >>             return codecs.open(name, mode, encoding)
> >>         return file(name, mode)
> >
> > Nice. It should probably also check whether there's a 'b' or 't' in 'mode'
> > and raise an error if so.
>
>Why should it do that ?

It's not necessary if both codecs.open() and file() raise an error when 
there's both a 'U' and a 't' or 'b' in the mode string, I suppose.


>FYI: codecs.open() explicitly adds the 'b' to the mode since
>we don't want the platform text mode interfere with the
>Unicode line breaking.

I think maybe you're confusing the wrapped file's mode with the passed-in 
mode, here.  The passed-in mode should contain at most one of 'b', 't', or 
'U', IIUC.  The mode used for the wrapped file should of course always be 
'b', but that's not visible to the user of the routine.


> > I'd also prefer to call it 'textfile', as that
> > reads more nicely with "for line in textfile(...):" use cases, and it does
> > return a file object.
>
>Nope: open() is only guaranteed to return a file-like object,
>e.g. codecs.open() will return a wrapped version of a file
>object.

I meant it's a "file object" in use case terms, not that isinstance(ob,file).


From fredrik at pythonware.com  Tue Dec 27 16:37:36 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 27 Dec 2005 16:37:36 +0100
Subject: [Python-Dev] file() vs open(), round 7
References: <20051226043831.GA24197@panix.com>
	<dorbej$9d$1@sea.gmane.org><43B13F97.5040509@egenix.com>
	<dorfv1$bak$1@sea.gmane.org>
	<5.1.1.6.0.20051227095611.02241f78@mail.telecommunity.com>
Message-ID: <dorn41$u5t$1@sea.gmane.org>

Phillip J. Eby wrote:

> >Here's a rough draft:
> >
> >     def textopen(name, mode="r", encoding=None):
> >         if "U" not in mode:
> >             mode += "U"
> >         if encoding:
> >             return codecs.open(name, mode, encoding)
> >         return file(name, mode)
>
> Nice. It should probably also check whether there's a 'b' or 't' in 'mode'
> and raise an error if so.  I'd also prefer to call it 'textfile', as that
> reads more nicely with "for line in textfile(...):" use cases, and it does
> return a file object.

textfile was my original proposal:

http://mail.python.org/pipermail/python-dev/2002-March/021115.html

but that was made at a time when it wasn't clear if "open" or "file" would
be the preferred way to open a file.  now that we've settled on the verb
form, I think "textopen" or "opentext" would be slightly more consistent.

but I agree that textfile looks a bit better.  how about "opentextfile" ? ;-)

</F>




From pje at telecommunity.com  Tue Dec 27 16:44:30 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 27 Dec 2005 10:44:30 -0500
Subject: [Python-Dev] file() vs open(), round 7
In-Reply-To: <dorn41$u5t$1@sea.gmane.org>
References: <20051226043831.GA24197@panix.com> <dorbej$9d$1@sea.gmane.org>
	<43B13F97.5040509@egenix.com> <dorfv1$bak$1@sea.gmane.org>
	<5.1.1.6.0.20051227095611.02241f78@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051227104310.042b2d78@mail.telecommunity.com>

At 04:37 PM 12/27/2005 +0100, Fredrik Lundh wrote:
>but that was made at a time when it wasn't clear if "open" or "file" would
>be the preferred way to open a file.  now that we've settled on the verb
>form, I think "textopen" or "opentext" would be slightly more consistent.

Sorry, I'm confused.  Who settled on the verb form?  Are you saying Guido's 
2002 post supports open() instead of file(), or is there some more recent 
reference to this?


From fredrik at pythonware.com  Tue Dec 27 16:39:59 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 27 Dec 2005 16:39:59 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
Message-ID: <dorn8g$uhk$1@sea.gmane.org>

Ronald Oussoren wrote:

> Why must quit and exit be so special in the first place? They could
> be plain functions, or even something like::
>
> class _QuitOrExit:
> def __init__(self, name):
> self.name = name
>
> def __repr__(self):
> return "Use %(name)s() to exit."%(self.__dict__)
>
> def __call__(self):
> raise SystemExit
>
> quit = _QuitOrExit("quit")
> exit = _QuitOrExit("exit")

but now we're back to today's situation:

    >>> quit
    'Use Ctrl-Z plus Return to exit.'

which violates the basic "if you know what I mean, why the /!"&/&!//%¤
don't you do what I say" usability rule.

</F>




From fdrake at acm.org  Tue Dec 27 16:53:19 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 27 Dec 2005 10:53:19 -0500
Subject: [Python-Dev] status of development documentation
In-Reply-To: <dore8d$71g$1@sea.gmane.org>
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<dore8d$71g$1@sea.gmane.org>
Message-ID: <200512271053.20254.fdrake@acm.org>

On Tuesday 27 December 2005 08:06, Fredrik Lundh wrote:
 > the "trunk" link on
 >
 >     http://www.python.org/dev/doc/

Fixed now; thanks for the reminder.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From fredrik at pythonware.com  Tue Dec 27 16:51:32 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 27 Dec 2005 16:51:32 +0100
Subject: [Python-Dev] file() vs open(), round 7
References: <20051226043831.GA24197@panix.com>
	<dorbej$9d$1@sea.gmane.org><43B13F97.5040509@egenix.com>
	<dorfv1$bak$1@sea.gmane.org><5.1.1.6.0.20051227095611.02241f78@mail.telecommunity.com>
	<dorn41$u5t$1@sea.gmane.org>
	<5.1.1.6.0.20051227104310.042b2d78@mail.telecommunity.com>
Message-ID: <dornu5$qo$1@sea.gmane.org>

Phillip J. Eby wrote:

> >but that was made at a time when it wasn't clear if "open" or "file" would
> >be the preferred way to open a file.  now that we've settled on the verb
> >form, I think "textopen" or "opentext" would be slightly more consistent.
>
> Sorry, I'm confused.  Who settled on the verb form?  Are you saying Guido's
> 2002 post supports open() instead of file(), or is there some more recent
> reference to this?

see:

http://mail.python.org/pipermail/python-dev/2004-July/045921.html

    "I recently saw a checkin that changed a call to open() into a call to
    file(), suggesting that using file() is more "politically correct"
    than open().

    I'm not sure I agree with this."

http://mail.python.org/pipermail/python-dev/2004-July/045967.html

    "Anyway, here's my future-proof distinction: use open() as the factory
    function, and file for type-testing."

</F>




From arigo at tunes.org  Tue Dec 27 17:18:53 2005
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 27 Dec 2005 17:18:53 +0100
Subject: [Python-Dev] NotImplemented reaching top-level
In-Reply-To: <e04bdf310512260931p2a7d37c6o@mail.gmail.com>
References: <e04bdf310512240931t1049346n@mail.gmail.com>
	<20051225113550.GA28497@code1.codespeak.net>
	<dom0ah$f29$1@sea.gmane.org>
	<20051225162325.GA32552@code1.codespeak.net>
	<43AECB86.50206@gmail.com>
	<20051226162710.GA19675@code1.codespeak.net>
	<e04bdf310512260931p2a7d37c6o@mail.gmail.com>
Message-ID: <20051227161853.GA7517@code1.codespeak.net>

Hi Facundo,

On Mon, Dec 26, 2005 at 02:31:10PM -0300, Facundo Batista wrote:
> > nb_add and nb_multiply should be tried.  I don't think that this would
> > break existing C or Python code, but it should probably only go in 2.5,
> > together with the patch #1390657 that relies on it.
> 
> It'd be good to know if this will be applied for the next version
> 2.4.x or will wait until 2.4.5, for me to search a workaround in
> Decimal or not (really don't know if I can find a solution here).

I completed the patch on the SF tracker, and now I believe that it could
safely be checked in the HEAD and in the 2.4 branch (after the
appropriate review).


A bientot,

Armin

From ronaldoussoren at mac.com  Tue Dec 27 17:40:20 2005
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 27 Dec 2005 17:40:20 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <dorn8g$uhk$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
Message-ID: <F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>


On 27-dec-2005, at 16:39, Fredrik Lundh wrote:

> Ronald Oussoren wrote:
>
>> Why must quit and exit be so special in the first place? They could
>> be plain functions, or even something like::
>>
>> class _QuitOrExit:
>> def __init__(self, name):
>> self.name = name
>>
>> def __repr__(self):
>> return "Use %(name)s() to exit."%(self.__dict__)
>>
>> def __call__(self):
>> raise SystemExit
>>
>> quit = _QuitOrExit("quit")
>> exit = _QuitOrExit("exit")
>
> but now we're back to today's situation:
>
>>>> quit
>     'Use Ctrl-Z plus Return to exit.'
>
> which violates the basic "if you know what I mean, why the /!"&/&!//%?
> don't you do what I say" usability rule.

I'd prefer 'def quit(): raise SystemExit', the class above just adds  
a message for someone that accidently got stuck in a python shell. I  
don't like the idea of making quit or exit special enough to cause  
side effects without parentheses, no other function does that. Anyone  
that knows how to program in Python should be able to guess that you  
have to use 'quit()' instead of 'quit'.

BTW. I do agree that the current situation is stupid.

Ronald
>
> </F>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ 
> ronaldoussoren%40mac.com


From martin at v.loewis.de  Tue Dec 27 17:58:49 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 27 Dec 2005 17:58:49 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
Message-ID: <43B172C9.4070201@v.loewis.de>

Ronald Oussoren wrote:
> I'd prefer 'def quit(): raise SystemExit', the class above just adds  
> a message for someone that accidently got stuck in a python shell. I  
> don't like the idea of making quit or exit special enough to cause  
> side effects without parentheses, no other function does that. Anyone  
> that knows how to program in Python should be able to guess that you  
> have to use 'quit()' instead of 'quit'.

The thing is there primarily for people who *don't* know how to
program in Python. If they knew, they knew how to get out of it;
they wouldn't type "quit()" but simply Ctrl-D.

So if they do

>>> quit
<function quit at 0xb7d96294>

they are just as confused as if they got a name error.

Regards,
Martin

From mal at egenix.com  Tue Dec 27 18:39:26 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 27 Dec 2005 18:39:26 +0100
Subject: [Python-Dev] file() vs open(), round 7
In-Reply-To: <5.1.1.6.0.20051227103405.04344180@mail.telecommunity.com>
References: <5.1.1.6.0.20051227095611.02241f78@mail.telecommunity.com>
	<20051226043831.GA24197@panix.com> <dorbej$9d$1@sea.gmane.org>
	<43B13F97.5040509@egenix.com>
	<5.1.1.6.0.20051227095611.02241f78@mail.telecommunity.com>
	<5.1.1.6.0.20051227103405.04344180@mail.telecommunity.com>
Message-ID: <43B17C4E.9030905@egenix.com>

Phillip J. Eby wrote:
> At 04:20 PM 12/27/2005 +0100, M.-A. Lemburg wrote:
>> Phillip J. Eby wrote:
>> > At 02:35 PM 12/27/2005 +0100, Fredrik Lundh wrote:
>> >> M.-A. Lemburg wrote:
>> >>
>> >>>> can we add a opentext factory for file/codecs.open while we're at
>> it ?
>> >>> Why a new factory function ? Can't we just redirect to codecs.open()
>> >>> in case an encoding keyword argument is passed to open() ?!
>> >> I think open is overloaded enough as it is.  Using separate
>> functions for
>> >> distinct
>> >> use cases is also a lot better than keyword trickery.
>> >>
>> >> Here's a rough draft:
>> >>
>> >>     def textopen(name, mode="r", encoding=None):
>> >>         if "U" not in mode:
>> >>             mode += "U"
>> >>         if encoding:
>> >>             return codecs.open(name, mode, encoding)
>> >>         return file(name, mode)
>> >
>> > Nice. It should probably also check whether there's a 'b' or 't' in
>> 'mode'
>> > and raise an error if so.
>>
>> Why should it do that ?
> 
> It's not necessary if both codecs.open() and file() raise an error when
> there's both a 'U' and a 't' or 'b' in the mode string, I suppose.

I see what you mean. codecs.open() doesn't work with 'U'.

>> FYI: codecs.open() explicitly adds the 'b' to the mode since
>> we don't want the platform text mode interfere with the
>> Unicode line breaking.
> 
> I think maybe you're confusing the wrapped file's mode with the
> passed-in mode, here.  The passed-in mode should contain at most one of
> 'b', 't', or 'U', IIUC.  The mode used for the wrapped file should of
> course always be 'b', but that's not visible to the user of the routine.

Thinking about this some more, I think it's better to
make encoding mandatory and to not use file() at all
in the API.

When we move to all text is Unicode in Py3k, we'll
have to require this anyway, so why not start with it
now.

That said, I think that a name "textfile" would be
more appropriate for the factory function, like you
suggested.

Valid values for mode would then be 'r', 'w' and 'a'.
'U' is not needed. 'b' and 't' neither. The '+' modes
don't work well with codecs.

>> > I'd also prefer to call it 'textfile', as that
>> > reads more nicely with "for line in textfile(...):" use cases, and
>> it does
>> > return a file object.
>>
>> Nope: open() is only guaranteed to return a file-like object,
>> e.g. codecs.open() will return a wrapped version of a file
>> object.
> 
> I meant it's a "file object" in use case terms, not that
> isinstance(ob,file).

We usually call this an "xyz-like object" (meaning that
the object provides a certain kind of interface).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 27 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From blais at furius.ca  Tue Dec 27 18:51:28 2005
From: blais at furius.ca (Martin Blais)
Date: Tue, 27 Dec 2005 12:51:28 -0500
Subject: [Python-Dev] (back to): Linked lists -- (was:  deque alternative)
Message-ID: <8393fff0512270951n77f53f71t695ac6cc3e05577@mail.gmail.com>

On 12/26/05, Josiah Carlson <jcarlson at uci.edu> wrote:
> > A "flat" list or tuple would certainly be more space-efficient up to
> > this point.  But when the graph branches, the 2-tuple representation
> > allows *sharing* the common path suffix (which may be very long!):
> ...
> > If there's an N-way branch, using 2-tuples allows recording the N new
> > paths back to the root with incremental memory burden N *
> > some_constant.  You can't do this kind of sharing with a flat list or
> > tuple, and the incremental memory burden at an N-way branch zooms to
> > (N + some_other_constant) * (the amount of memory needed to record the
> > path from the branch point back to the root), due to duplicating the
> > back-to-the-root info N times.   The potential memory saving from
> > using 2-tuples instead is unbounded.
>
> But one doesn't _need_ to use flat lists.  If one were to combine cons
> cells with Python lists, you can get the space efficiency of lists with
> the reusability of the desired cons cells (or tuples as the case may be).
> How? (i, l), where i is the prefix of list l which is the desired
> portion of whatever l represents.  You toss one of those anywhere in
> your 'flat list', and you have your stored/saved path with a couple
> dozen bytes. If you are not careful, you end up storing/saving paths
> which would be stored more efficiently by copying the contents, but at
> that point we are talking about a constant factor blowup, not the
> (potentially) exponential blowup of storing all paths as copies, and we
> can always copy to be more efficient if necessary.
>
> I have personally used this trick to construct a union-find data
> structure in which all unions are reversable regardless of find
> operation. [1]

Hmm using a single simple data type seems more elegant and less
error-prone in this case than what you suggest.  I would argue that
such solutions come about exactly because lists aren't available. 
Sure your solution works, but IMO it's a case of raising the hammer
with your arm, noticing that it's a screw and not a nail, and then
using the hammer sideways to try to turn the screw.  I want a
screwdriver.


> > > So, the list will generally be 1/8th of its size overallocated, or
> > > 112 elements plus 9 words overhead is 121 words. Better than any cons-
> > > linked list could be, and *insanely better* than a cons-linked list
> > > would be in python.
> >
> > You seem to be ignoring possiblities for sharing across lists, and
> > such sharing is natural in many graph algorithms.
>
> Not necessarily so as I have pointed out above.  You said yourself;
> practicality beats purity.  While using cons cells are pure, as us using
> only flat lists are pure, flat lists are practically smaller than cons
> cells in certain cases (by a factor of ~4), and non-flat lists can be
> smaller than cons cells in the rest of the cases.

I don't know about "practicality beats purity" type of arguments. 
Such general principles don't always apply.  Building lists and
trees/graphs with cons cells seem very, very practical to me.

Now, it's not all about storage space.  What about front-insertion? 
Everytime I have to type L.insert(0, bli), somewhere that I know runs
"often" I cringe.  If I had lists I would be able to express my
thoughts more clearly in the computer program.  Right now, I cringe,
and then I just shrug.

From martin at v.loewis.de  Tue Dec 27 18:54:30 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 27 Dec 2005 18:54:30 +0100
Subject: [Python-Dev] file() vs open(), round 7
In-Reply-To: <43B149E6.9090205@egenix.com>
References: <20051226043831.GA24197@panix.com>	<dorbej$9d$1@sea.gmane.org>	<43B13F97.5040509@egenix.com>	<dorfv1$bak$1@sea.gmane.org>
	<43B149E6.9090205@egenix.com>
Message-ID: <43B17FD6.2070702@v.loewis.de>

M.-A. Lemburg wrote:
>>Here's a rough draft:
>>
>>    def textopen(name, mode="r", encoding=None):
>>        if "U" not in mode:
>>            mode += "U"
> 
> 
> The "U" is not needed when opening files using codecs -
> these always break lines using .splitlines() which
> breaks lines according to the Unicode rules and also
> knows about the various line break variants on different
> platforms.

Still, codecs typically don't implement universal newlines
correctly. If you specify 'U', then do .read(), you deserve
to get \n (U+0010) as the line separator; with most codecs,
you get whatever line breaks where in the file.

Passing 'U' to the underlying stream is wrong, as well:
if the stream is double-byte oriented (e.g. UTF-16),
the 'U' filtering will rarely do anything, but if it does
something, it will be wrong.

I agree that it would be desirable to have textopen always
default to universal newlines, however, this is difficult
to implement.

Regards,
Martin

From aahz at pythoncraft.com  Tue Dec 27 19:21:16 2005
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 27 Dec 2005 10:21:16 -0800
Subject: [Python-Dev] (back to): Linked lists -- (was: deque alternative)
In-Reply-To: <8393fff0512270951n77f53f71t695ac6cc3e05577@mail.gmail.com>
References: <8393fff0512270951n77f53f71t695ac6cc3e05577@mail.gmail.com>
Message-ID: <20051227182116.GA9909@panix.com>

On Tue, Dec 27, 2005, Martin Blais wrote:
>
> Now, it's not all about storage space.  What about front-insertion? 
> Everytime I have to type L.insert(0, bli), somewhere that I know runs
> "often" I cringe.  If I had lists I would be able to express my
> thoughts more clearly in the computer program.  Right now, I cringe,
> and then I just shrug.

Why don't you just write your own list type?  Why does this need to go
into Python?  Why should it be one of the builtin types instead of a
library?

Please answer these questions on comp.lang.python, NOT python-dev.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Don't listen to schmucks on USENET when making legal decisions.  Hire
yourself a competent schmuck."  --USENET schmuck (aka Robert Kern)

From adal.chiriliuc at gmail.com  Tue Dec 27 17:59:08 2005
From: adal.chiriliuc at gmail.com (Adal Chiriliuc)
Date: Tue, 27 Dec 2005 18:59:08 +0200
Subject: [Python-Dev] Python + Visual C++ 8.0?
Message-ID: <153078534.20051227185908@myrealbox.com>

Python uses LoadLibraryEx with the LOAD_WITH_ALTERED_SEARCH_PATH flag
which means that DLLs used by the extension will be searched
IN THE EXTENSION FOLDER and not on PATH.

Try putting msvcp80.dll right next to your extension DLL.

It is a little strange that it is not loaded directly from the Windows
side by side folder (%WINDIR%\WinSxS). You should check if the manifest
is embedded correctly inside the DLL.

And like Martin said, mixing Python and extensions compiled with
different compilers is a bad idea.



From mal at egenix.com  Tue Dec 27 19:48:07 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 27 Dec 2005 19:48:07 +0100
Subject: [Python-Dev] file() vs open(), round 7
In-Reply-To: <43B17FD6.2070702@v.loewis.de>
References: <20051226043831.GA24197@panix.com>	<dorbej$9d$1@sea.gmane.org>	<43B13F97.5040509@egenix.com>	<dorfv1$bak$1@sea.gmane.org>	<43B149E6.9090205@egenix.com>
	<43B17FD6.2070702@v.loewis.de>
Message-ID: <43B18C67.4060900@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
>>> Here's a rough draft:
>>>
>>>    def textopen(name, mode="r", encoding=None):
>>>        if "U" not in mode:
>>>            mode += "U"
>>
>> The "U" is not needed when opening files using codecs -
>> these always break lines using .splitlines() which
>> breaks lines according to the Unicode rules and also
>> knows about the various line break variants on different
>> platforms.
> 
> Still, codecs typically don't implement universal newlines
> correctly. If you specify 'U', then do .read(), you deserve
> to get \n (U+0010) as the line separator; with most codecs,
> you get whatever line breaks where in the file.
> 
> Passing 'U' to the underlying stream is wrong, as well:
> if the stream is double-byte oriented (e.g. UTF-16),
> the 'U' filtering will rarely do anything, but if it does
> something, it will be wrong.
> 
> I agree that it would be desirable to have textopen always
> default to universal newlines, however, this is difficult
> to implement.

I think that codecs solve the problem in a better way.
If you want to read lines from a stream, you'd use
.readline() or .readlines() to read the lines, and not
expect .read() to magically apply some conversion to the
original data.

Both line methods have a parameter keepends (which defaults to
True). This parameter specifies whether you will get the
original line end markers or not, which makes it possible to let
the application implement whatever logic it finds
appropriate.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 27 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From skip at pobox.com  Tue Dec 27 20:36:44 2005
From: skip at pobox.com (skip@pobox.com)
Date: Tue, 27 Dec 2005 13:36:44 -0600
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <43B172C9.4070201@v.loewis.de>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
Message-ID: <17329.38860.916447.765575@montanaro.dyndns.org>

    Martin> So if they do

    >>> quit
    <function quit at 0xb7d96294>

    Martin> they are just as confused as if they got a name error.

Probably more so... <wink>

Skip

From fredrik at pythonware.com  Tue Dec 27 20:58:25 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 27 Dec 2005 20:58:25 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>
	<17329.15477.267537.300546@montanaro.dyndns.org>
Message-ID: <dos6d4$ep1$1@sea.gmane.org>

skip at pobox.com wrote:

>     Fredrik> whaddya think?
>
> We're going to wind up with the same problem that spawned the atexit module:
> multiple code sources wanting to fiddle with sys.excepthook step on one
> another's toes.

in this case, I'm not sure it matters that much.  if you add your own except-
hook, you take responsibility.

(it would be nice if it was possible to detect "interactive mode" when the site
code runs, though.  does anyone know if that's possible ?)

</F>




From hans at zephyrfalcon.org  Tue Dec 27 21:11:39 2005
From: hans at zephyrfalcon.org (Hans Nowak)
Date: Tue, 27 Dec 2005 15:11:39 -0500
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <43B172C9.4070201@v.loewis.de>
References: <dor982$qrc$1@sea.gmane.org>	<dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
Message-ID: <43B19FFB.4080807@zephyrfalcon.org>

Martin v. L?wis wrote:

> The thing is there primarily for people who *don't* know how to
> program in Python. If they knew, they knew how to get out of it;
> they wouldn't type "quit()" but simply Ctrl-D.

Except that on Windows, it's Ctrl-Z.  This can be quite confusing when 
you regularly use Python on both Windows and Unix, and use the wrong key 
combination.  Ctrl-D on Windows does not have the desired result, and 
Ctrl-Z on Unix suspends the process.  (And if you use a GUI version, 
they often have their own rules... on IDLE for Windows, Ctrl-D works but 
Ctrl-Z doesn't; on PyCrust, neither one works.)

Granted, it's not a big problem, but it *is* annoying.  IMHO, it would 
be more useful if you could just type 'exit' or 'quit' (like in many 
other interpreters) and be done with it, rather than having to remember 
the correct key combination.  (A key combination which has nothing to do 
with the Python language per se.)

-- 
Hans Nowak
http://zephyrfalcon.org/


From dialtone at divmod.com  Tue Dec 27 21:45:01 2005
From: dialtone at divmod.com (Valentino Volonghi aka Dialtone)
Date: Tue, 27 Dec 2005 21:45:01 +0100
Subject: [Python-Dev] Small any/all enhancement
Message-ID: <20051227204501.GA252@divmod.com>

I see that Python 2.5 will include a simple implementation of any/all.

What I would like to ask, before it's too late, is if it's possible to add an
optional test argument.

any(iterable, test=bool) and all(iterable, test=bool)

These would be the new proposed APIs. The usecase is to be able to extract
attributes from objects or simply to have arbitrary checks like:

import operator

any(some_objects, test=operator.attrgetter('some_attribute'))

or

def zerop(x):
    return x==0

all(some_objects, zerop)

instead of preprocessing the generator with a generator expression before
passing it to any/all.

Thanks.

-- 
Valentino Volonghi aka Dialtone
Now Running MacOSX 10.4
Blog: http://vvolonghi.blogspot.com
http://weever.berlios.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 186 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20051227/e76dae5b/attachment.pgp

From martin at v.loewis.de  Tue Dec 27 22:28:17 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 27 Dec 2005 22:28:17 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <43B19FFB.4080807@zephyrfalcon.org>
References: <dor982$qrc$1@sea.gmane.org>	<dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de> <43B19FFB.4080807@zephyrfalcon.org>
Message-ID: <43B1B1F1.30605@v.loewis.de>

Hans Nowak wrote:
> Granted, it's not a big problem, but it *is* annoying.  IMHO, it would
> be more useful if you could just type 'exit' or 'quit' (like in many
> other interpreters) and be done with it, rather than having to remember
> the correct key combination.  (A key combination which has nothing to do
> with the Python language per se.)

If you want to type something consistently across platforms, you can
currently do

>>> raise SystemExit

Regards,
Martin

From martin at v.loewis.de  Tue Dec 27 22:41:31 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 27 Dec 2005 22:41:31 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <dos6d4$ep1$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org>	<17329.15477.267537.300546@montanaro.dyndns.org>
	<dos6d4$ep1$1@sea.gmane.org>
Message-ID: <43B1B50B.2010905@v.loewis.de>

Fredrik Lundh wrote:
> (it would be nice if it was possible to detect "interactive mode" when the site
> code runs, though.  does anyone know if that's possible ?)

I believe checking sys.argv==[''] is a nearly reliable test (the only
other case where I could make it happen is when the script is read from
stdin). Introducing a flag into InteractiveLoop would be an option, too.

Regards,
Martin

From hans at zephyrfalcon.org  Tue Dec 27 22:44:55 2005
From: hans at zephyrfalcon.org (Hans Nowak)
Date: Tue, 27 Dec 2005 16:44:55 -0500
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <43B1B1F1.30605@v.loewis.de>
References: <dor982$qrc$1@sea.gmane.org>	<dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>	<43B172C9.4070201@v.loewis.de>
	<43B19FFB.4080807@zephyrfalcon.org> <43B1B1F1.30605@v.loewis.de>
Message-ID: <43B1B5D7.6010603@zephyrfalcon.org>

Martin v. L?wis wrote:
> If you want to type something consistently across platforms, you can
> currently do
> 
>>>>raise SystemExit

I'm not sure what to say to that.  Do you really want to type "raise 
SystemExit" every time you want to exit the interpreter?  (Your answer 
would probably be something like "No -- that's why I use Ctrl-D".  But 
that wouldn't really get us anywhere, would it?)

My point is that there is currently no acceptable, universal way to exit 
the interpreter.  Again, it's not that big of a deal... but it seems 
silly that something apparently trivial like that cannot be done right.

-- 
Hans Nowak
http://zephyrfalcon.org/


From rwgk at yahoo.com  Tue Dec 27 22:39:04 2005
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Tue, 27 Dec 2005 13:39:04 -0800 (PST)
Subject: [Python-Dev] Python + Visual C++ 8.0?
In-Reply-To: <43B06C3F.4030203@v.loewis.de>
Message-ID: <20051227213904.84090.qmail@web31507.mail.mud.yahoo.com>

--- "Martin v. L?wis" <martin at v.loewis.de> wrote:

> Ralf W. Grosse-Kunstleve wrote:
> > I am using a Visual Studio 2005 Professional installation. I also ran
> > vcredist_x86.exe. Moving msvc[mpr]80.dll to a directory on PATH didn't
> > help. However, standalone executables work OK without any gymnastics.
> > Does anyone know what the problem could be with the extensions?
> 
> Can't check right now - but could it be that the standalone executables
> get an extra copy of this library in their binary directory, as part
> of the build process?

No.

> > A quick attempt to compile Python from scratch using Visual C++ 8.0
> > produced a python.exe, but it doesn't run (the debug / send report /
> > don't send report box pops up). Has someone tried this before?
> 
> Yes; a patch to fix this problem has been checked into the trunk
> (but I have no plans for backporting it to the 2.4 branch, as 2.4
> will be compiled with VC7.1 "forever" - i.e. until its final release).

OK.

> Regards,
> Martin
> 
> P.S. I currently also plan to build Python 2.5 with VC 7.1.
> P.P.S. You do know that this configuration (extension compiled
> with VS2005, Python compiled wit VS.NET2003) is not supported,
> right?

I wasn't sure about that. In the transition from VC 6 to VC 7.x we didn't
have compatibility problems. I was hoping this trend continues ...
If that's not the case we will need two Python 2.5 installers,
one compiled with VC 7.1, one with VC 8.0.
I'll try to build SVN Python with VC 8.0. If that succeeds, is there
an equivalent of "make install" that will give me a directory structure
resembling what I get from the binary installer?

Thanks!

Cheers,
        Ralf



		
__________________________________________ 
Yahoo! DSL ? Something to write home about. 
Just $16.99/mo. or less. 
dsl.yahoo.com 


From martin at v.loewis.de  Tue Dec 27 22:49:58 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 27 Dec 2005 22:49:58 +0100
Subject: [Python-Dev] Small any/all enhancement
In-Reply-To: <20051227204501.GA252@divmod.com>
References: <20051227204501.GA252@divmod.com>
Message-ID: <43B1B706.50902@v.loewis.de>

Valentino Volonghi aka Dialtone wrote:
> What I would like to ask, before it's too late, is if it's possible to add an
> optional test argument.

I don't see why: you can easily synthesize that with map/imap.

> These would be the new proposed APIs. The usecase is to be able to extract
> attributes from objects or simply to have arbitrary checks like:
> 
> import operator
> 
> any(some_objects, test=operator.attrgetter('some_attribute'))

So write

  any(map(operator.attrgetter('some_attribute'), some_objects))
  # same number of characters to type

  any(o.some_attribute for o in some_objects)
  # fewer number of characters

> def zerop(x):
>     return x==0
> 
> all(some_objects, zerop)

So write

  all(map(some_objects, zerop))
or
  all(o==0 for o in some_objects)
  # avoids defining zerop

> instead of preprocessing the generator with a generator expression before
> passing it to any/all.

What is the disadvantage of such "preprocessing"?

Regards,
Martin

From aleaxit at gmail.com  Tue Dec 27 22:50:37 2005
From: aleaxit at gmail.com (Alex Martelli)
Date: Tue, 27 Dec 2005 13:50:37 -0800
Subject: [Python-Dev] Small any/all enhancement
In-Reply-To: <20051227204501.GA252@divmod.com>
References: <20051227204501.GA252@divmod.com>
Message-ID: <8AA572FD-50E2-4781-91A5-9C6D73B7E8EA@gmail.com>


On Dec 27, 2005, at 12:45 PM, Valentino Volonghi aka Dialtone wrote:
    ...
> any(iterable, test=bool) and all(iterable, test=bool)
    ...
> any(some_objects, test=operator.attrgetter('some_attribute'))

Why would that be better than
any(o.some_attribute for o in some_objects)
?

> def zerop(x):
>     return x==0
>
> all(some_objects, zerop)

and why would that be better than
all(o==0 for o in some_objects)
?

> instead of preprocessing the generator with a generator expression  
> before
> passing it to any/all.

I guess I just don't see the advantage, along any plane, since the  
genexp fits so snugly right there inside the any/all's parentheses.   
What am I missing?


Alex


From rwgk at yahoo.com  Tue Dec 27 22:59:49 2005
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Tue, 27 Dec 2005 13:59:49 -0800 (PST)
Subject: [Python-Dev] Python + Visual C++ 8.0?
In-Reply-To: <153078534.20051227185908@myrealbox.com>
Message-ID: <20051227215950.72784.qmail@web31514.mail.mud.yahoo.com>

--- Adal Chiriliuc <adal.chiriliuc at gmail.com> wrote:

> Python uses LoadLibraryEx with the LOAD_WITH_ALTERED_SEARCH_PATH flag
> which means that DLLs used by the extension will be searched
> IN THE EXTENSION FOLDER and not on PATH.
> 
> Try putting msvcp80.dll right next to your extension DLL.

I tried that first since that's the way we use VC 7.1 extensions with older
Python versions (compiled with VC 6).

> It is a little strange that it is not loaded directly from the Windows
> side by side folder (%WINDIR%\WinSxS). You should check if the manifest
> is embedded correctly inside the DLL.

Sorry, the manifests are new to me. How can I check if the manifest is
correctly embedded?
FWIW: I already tried copying the manifest into the directory with the
extensions.

In case it matters, here are the compiler and liker switches I am  using
(commands issued by scons):

cl /nologo /D_CRT_SECURE_NO_DEPRECATE /wd4996 /Zm800 /MD /GR /EHsc
/DBOOST_DISABLE_THREADS /DNDEBUG /Ox -DBOOST_PYTHON_MAX_BASES=2
-DBOOST_PYTHON_SOURCE

link /nologo /incremental:no /dll /out:lib\cctbx_math_ext.pyd
/implib:lib\cctbx_math_ext.lib /LIBPATH:lib

Am I missing some magic new switch?

> And like Martin said, mixing Python and extensions compiled with
> different compilers is a bad idea.

If that's really the case it will mean a lot of work.
Mixing VC6 Python and VC7.x extensions never gave us any trouble.

Thanks!

Cheers,
        Ralf



	
		
__________________________________ 
Yahoo! for Good - Make a difference this year. 
http://brand.yahoo.com/cybergivingweek2005/

From tim.peters at gmail.com  Tue Dec 27 23:05:00 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 27 Dec 2005 17:05:00 -0500
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <43B1B5D7.6010603@zephyrfalcon.org>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de> <43B19FFB.4080807@zephyrfalcon.org>
	<43B1B1F1.30605@v.loewis.de> <43B1B5D7.6010603@zephyrfalcon.org>
Message-ID: <1f7befae0512271405o1ea625f1u305a136758602b0d@mail.gmail.com>

[Martin v. L?wis]
>> If you want to type something consistently across platforms, you can
>> currently do
>>
>> >>>raise SystemExit

[Hans Nowak]
> I'm not sure what to say to that.  Do you really want to type "raise
> SystemExit" every time you want to exit the interpreter?  (Your answer
> would probably be something like "No -- that's why I use Ctrl-D".  But
> that wouldn't really get us anywhere, would it?)
>
> My point is that there is currently no acceptable, universal way to exit
> the interpreter. ...

I haven't used Python on a PDA yet, but on everything from PCs to
mainframes I get out of the interpreter just by pulling the power
plug.  Why on Earth would you want to exit Python while your machine
was running?  Aha!  The light dawns.

resolve-to-keep-python-running-24/7-in-2006-ly y'rs   - tim

From fredrik at pythonware.com  Tue Dec 27 23:16:51 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 27 Dec 2005 23:16:51 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>	<dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>	<43B172C9.4070201@v.loewis.de><43B19FFB.4080807@zephyrfalcon.org>
	<43B1B1F1.30605@v.loewis.de> <43B1B5D7.6010603@zephyrfalcon.org>
Message-ID: <dosegl$juv$1@sea.gmane.org>

Hans Nowak wrote:

> My point is that there is currently no acceptable, universal way to exit
> the interpreter.  Again, it's not that big of a deal... but it seems
> silly that something apparently trivial like that cannot be done right.

it's the usual problem: getting enough developers to agree that this
really is a problem is a lot more work than implementing it...

oh, well.  looks like SF is back up again.  I'll post the patch over there
when I find the time...

</F>




From martin at v.loewis.de  Tue Dec 27 23:20:20 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 27 Dec 2005 23:20:20 +0100
Subject: [Python-Dev] Python + Visual C++ 8.0?
In-Reply-To: <153078534.20051227185908@myrealbox.com>
References: <153078534.20051227185908@myrealbox.com>
Message-ID: <43B1BE24.5090908@v.loewis.de>

Adal Chiriliuc wrote:
> Python uses LoadLibraryEx with the LOAD_WITH_ALTERED_SEARCH_PATH flag
> which means that DLLs used by the extension will be searched
> IN THE EXTENSION FOLDER and not on PATH.

Can you please refer to the part of the documentation that documents
that PATH is not searched? In

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/dynamic-link_library_search_order.asp

the documentation says that the PATH is searched in step 6.

> It is a little strange that it is not loaded directly from the Windows
> side by side folder (%WINDIR%\WinSxS). You should check if the manifest
> is embedded correctly inside the DLL.

How do you get VS to do that?

Regards,
Martin

From martin at v.loewis.de  Tue Dec 27 23:22:20 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 27 Dec 2005 23:22:20 +0100
Subject: [Python-Dev] Python + Visual C++ 8.0?
In-Reply-To: <20051227213904.84090.qmail@web31507.mail.mud.yahoo.com>
References: <20051227213904.84090.qmail@web31507.mail.mud.yahoo.com>
Message-ID: <43B1BE9C.6070401@v.loewis.de>

Ralf W. Grosse-Kunstleve wrote:
> I wasn't sure about that. In the transition from VC 6 to VC 7.x we didn't
> have compatibility problems. I was hoping this trend continues ...

That was by pure luck only. Other people did have problems.

> If that's not the case we will need two Python 2.5 installers,
> one compiled with VC 7.1, one with VC 8.0.
> I'll try to build SVN Python with VC 8.0. If that succeeds, is there
> an equivalent of "make install" that will give me a directory structure
> resembling what I get from the binary installer?

You can create an installer yourself, through Tools/msi/msi.py.
There is no "make install" procedure.

Regards,
Martin

From martin at v.loewis.de  Tue Dec 27 23:26:26 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 27 Dec 2005 23:26:26 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <43B1B5D7.6010603@zephyrfalcon.org>
References: <dor982$qrc$1@sea.gmane.org>	<dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>	<43B172C9.4070201@v.loewis.de>	<43B19FFB.4080807@zephyrfalcon.org>
	<43B1B1F1.30605@v.loewis.de> <43B1B5D7.6010603@zephyrfalcon.org>
Message-ID: <43B1BF92.3070704@v.loewis.de>

Hans Nowak wrote:
> I'm not sure what to say to that.  Do you really want to type "raise 
> SystemExit" every time you want to exit the interpreter?  (Your answer 
> would probably be something like "No -- that's why I use Ctrl-D".  But 
> that wouldn't really get us anywhere, would it?)

Well... I understand that you want "exit" to quit the interpreter, and I
can understand why you want that. Fredrik's proposed change started that
entire thread.

> My point is that there is currently no acceptable, universal way to exit 
> the interpreter.

That clearly depends on the definition of "acceptable". If "acceptable"
means "using a four-letter keyword", then you are right, yes.

> Again, it's not that big of a deal... but it seems 
> silly that something apparently trivial like that cannot be done right.

It's silly only if you understand the background that leads to this
state of affairs.

Regards,
Martin


From adal.chiriliuc at gmail.com  Tue Dec 27 23:43:54 2005
From: adal.chiriliuc at gmail.com (Adal Chiriliuc)
Date: Wed, 28 Dec 2005 00:43:54 +0200
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <dosegl$juv$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de><43B19FFB.4080807@zephyrfalcon.org>
	<43B1B1F1.30605@v.loewis.de> <43B1B5D7.6010603@zephyrfalcon.org>
	<dosegl$juv$1@sea.gmane.org>
Message-ID: <927510309.20051228004354@myrealbox.com>

Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> exit
'Use Ctrl-Z plus Return to exit.'
>>>

I've just tried Ctrl+Z (plus Return) and some variations on Win XP and
it doesn't work! Ctrl+D does.

Beside, it should be spelled Ctrl+Z and not Ctrl-Z (at least this is
the Windows way).

I think it's intuitive for exit/quit to exit the interactive
interpreter. That's what I did the first time I've tried Python.

In another one (I think CLISP, but I'm not sure) I couldn't figure
out how to exit.

And definatly in CLISP when trying simple things I always got stuck in
some kind or different error mode (with a different prompt). That's why
I gave up playing with it.


From dialtone at divmod.com  Tue Dec 27 23:48:35 2005
From: dialtone at divmod.com (Valentino Volonghi aka Dialtone)
Date: Tue, 27 Dec 2005 23:48:35 +0100
Subject: [Python-Dev] Small any/all enhancement
In-Reply-To: <8AA572FD-50E2-4781-91A5-9C6D73B7E8EA@gmail.com>
References: <20051227204501.GA252@divmod.com>
	<8AA572FD-50E2-4781-91A5-9C6D73B7E8EA@gmail.com>
Message-ID: <20051227224835.GA16166@divmod.com>

On Tue, Dec 27, 2005 at 01:50:37PM -0800, Alex Martelli wrote:

I'll answer here for all the people who kindly answered.

> Why would that be better than
> any(o.some_attribute for o in some_objects)
> ?

I think it's because lately I've been using common lisp a lot and the approach
with the test function is pretty common there.

Of course I already knew all the alternatives using map and the generator
expression, but I felt like mine was clearer for a reader, this is probably
true but not enough to justify the change.

One reason was to hide the iteration from the user and let python handle
it, but I can see that this is both not enough and probably not even so
necessary since the iteration protocol is already 'hidden' when using for..in.

> What am I missing?

Nothing, it's just me and the bad influence that lisp has on my mind :).
Sorry to bother the list then. Thanks for the kind replies.

-- 
Valentino Volonghi aka Dialtone
Now Running MacOSX 10.4
Blog: http://vvolonghi.blogspot.com
http://weever.berlios.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 186 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20051227/ffc6dadd/attachment.pgp

From bob at redivi.com  Tue Dec 27 23:56:31 2005
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 27 Dec 2005 17:56:31 -0500
Subject: [Python-Dev] Small any/all enhancement
In-Reply-To: <20051227224835.GA16166@divmod.com>
References: <20051227204501.GA252@divmod.com>
	<8AA572FD-50E2-4781-91A5-9C6D73B7E8EA@gmail.com>
	<20051227224835.GA16166@divmod.com>
Message-ID: <39707FD5-B1D8-4EE4-B9E7-71C62063B1E0@redivi.com>


On Dec 27, 2005, at 5:48 PM, Valentino Volonghi aka Dialtone wrote:

> On Tue, Dec 27, 2005 at 01:50:37PM -0800, Alex Martelli wrote:
>
> I'll answer here for all the people who kindly answered.
>
>> Why would that be better than
>> any(o.some_attribute for o in some_objects)
>> ?
>
> I think it's because lately I've been using common lisp a lot and  
> the approach
> with the test function is pretty common there.
>
> Of course I already knew all the alternatives using map and the  
> generator
> expression, but I felt like mine was clearer for a reader, this is  
> probably
> true but not enough to justify the change.

I think that generator/list expressions are more common practice than  
attrgetter/itemgetter, so I'm not even sure it's clearer.

I don't see the harm in a "key" argument like sorted has, but without  
a key argument it could be extended to take more arguments like max/ 
min do for convenience.  e.g. any(a, b, c) instead of any((a, b, c)).

-bob


From fredrik at pythonware.com  Tue Dec 27 23:57:31 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 27 Dec 2005 23:57:31 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com><43B172C9.4070201@v.loewis.de><43B19FFB.4080807@zephyrfalcon.org><43B1B1F1.30605@v.loewis.de>
	<43B1B5D7.6010603@zephyrfalcon.org><dosegl$juv$1@sea.gmane.org>
	<927510309.20051228004354@myrealbox.com>
Message-ID: <dosgss$941$1@sea.gmane.org>

Adal Chiriliuc wrote:

> Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> exit
> 'Use Ctrl-Z plus Return to exit.'
> >>>
>
> I've just tried Ctrl+Z (plus Return) and some variations on Win XP and
> it doesn't work! Ctrl+D does.

WinXP or WinXP+Cygwin ?  here's what I get:

>python
Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> ^Z

>

>python
Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> ^D
  File "<stdin>", line 1
    ?
    ^
SyntaxError: invalid syntax
>>>

</F>




From python-dev at zesty.ca  Tue Dec 27 23:57:54 2005
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Tue, 27 Dec 2005 16:57:54 -0600 (CST)
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <17329.38860.916447.765575@montanaro.dyndns.org>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
	<17329.38860.916447.765575@montanaro.dyndns.org>
Message-ID: <Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>

It sounds to me like what is being proposed amounts to essentially
"promote sys.exit to a builtin".  So why not do that?

I see two options.  Either:

    (a) Simply let __builtins__.exit = sys.exit.  Then we get:

        >>> exit
        <built-in function exit>

        which may be enough of a clue that you type "exit()" to exit.

    (b) If more handholding seems like a good idea, then:

        class ExitHatch:
            def __call__(self): sys.exit()
            def __repr__(self): return '<Type "exit()" to exit Python.>'
        __builtins__.exit = __builtins__.quit = ExitHatch()


-- ?!ng

From adal.chiriliuc at gmail.com  Wed Dec 28 00:20:35 2005
From: adal.chiriliuc at gmail.com (Adal Chiriliuc)
Date: Wed, 28 Dec 2005 01:20:35 +0200
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <dosgss$941$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com><43B172C9.4070201@v.loewis.de><43B19FFB.4080807@zephyrfalcon.org><43B1B1F1.30605@v.loewis.de>
	<43B1B5D7.6010603@zephyrfalcon.org><dosegl$juv$1@sea.gmane.org>
	<927510309.20051228004354@myrealbox.com> <dosgss$941$1@sea.gmane.org>
Message-ID: <271407301.20051228012035@myrealbox.com>

On Wednesday, December 28, 2005 Fredrik Lundh wrote:
> WinXP or WinXP+Cygwin ?  here's what I get:

Normal WinXP, without Cygwin.

>>python
> Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> ^Z

>>

>>python
> Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> ^D
>   File "<stdin>", line 1
>     ?
>     ^
> SyntaxError: invalid syntax
>>>>

I tracked this down to having IPython installed. I was able to get the exact
behaviour you've got by disabling the "readline" package from "site-packages".

I have a solution proposal (I have no idea if it's feasible): why not
intercept "exit" and "quit" in the interpreter command line parser? A
simple regexp should do. Is the interactive interpreter implemented in
Python or on the C side?


From martin at v.loewis.de  Wed Dec 28 00:22:49 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 28 Dec 2005 00:22:49 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <271407301.20051228012035@myrealbox.com>
References: <dor982$qrc$1@sea.gmane.org>	<dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com><43B172C9.4070201@v.loewis.de><43B19FFB.4080807@zephyrfalcon.org><43B1B1F1.30605@v.loewis.de>	<43B1B5D7.6010603@zephyrfalcon.org><dosegl$juv$1@sea.gmane.org>	<927510309.20051228004354@myrealbox.com>
	<dosgss$941$1@sea.gmane.org>
	<271407301.20051228012035@myrealbox.com>
Message-ID: <43B1CCC9.70706@v.loewis.de>

Adal Chiriliuc wrote:
> I have a solution proposal (I have no idea if it's feasible): why not
> intercept "exit" and "quit" in the interpreter command line parser? A
> simple regexp should do. Is the interactive interpreter implemented in
> Python or on the C side?

In short: it's not feasible. Please don't propose such things on
python-dev without an implementation strategy in mind.

Regards,
Martin

From adal.chiriliuc at gmail.com  Tue Dec 27 23:21:58 2005
From: adal.chiriliuc at gmail.com (Adal Chiriliuc)
Date: Wed, 28 Dec 2005 00:21:58 +0200
Subject: [Python-Dev] Python + Visual C++ 8.0?
Message-ID: <1667976656.20051228002158@myrealbox.com>

On Tuesday, December 27, 2005 Ralf W. Grosse-Kunstleve wrote:

> Sorry, the manifests are new to me. How can I check if the manifest is
> correctly embedded?
> FWIW: I already tried copying the manifest into the directory with the
> extensions.

To check if you have a manifest you need to use a tool like "Resource
Hacker" or "XN Resource Editor".

http://www.wilsonc.demon.co.uk/d10resourceeditor.htm

Where did you get that manifest? You need one listing the MSVC runtime
(not one which enables the XP look). Of course, it must be named
exactly like you dll/pyd, with an additional .manifest extension.

<?xml version='1.0' encoding='UTF-8' standalone='yes'?>
<assembly xmlns='urn:schemas-microsoft-com:asm.v1' manifestVersion='1.0'>
  <dependency>
    <dependentAssembly>
      <assemblyIdentity type='win32' name='Microsoft.VC80.CRT' version='8.0.50608.0' processorArchitecture='x86' publicKeyToken='1fc8b3b9a1e18e3b' />
    </dependentAssembly>
  </dependency>
</assembly>

> In case it matters, here are the compiler and liker switches I am  using
> (commands issued by scons):

> cl /nologo /D_CRT_SECURE_NO_DEPRECATE /wd4996 /Zm800 /MD /GR /EHsc
> /DBOOST_DISABLE_THREADS /DNDEBUG /Ox -DBOOST_PYTHON_MAX_BASES=2
> -DBOOST_PYTHON_SOURCE

> link /nologo /incremental:no /dll /out:lib\cctbx_math_ext.pyd
> /implib:lib\cctbx_math_ext.lib /LIBPATH:lib

> Am I missing some magic new switch?

Add /manifest to the linker options. This will generate
cctbx_math_ext.pyd.manifest either in the output folder or in the
intermediate one.

Then you need to run mt.exe to embedd the manifest:
mt.exe /outputresource:"cctbx_math_ext.pyd;#2" /manifest cctbx_math_ext.pyd.manifest


From bcannon at gmail.com  Wed Dec 28 01:08:06 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Tue, 27 Dec 2005 16:08:06 -0800
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
	<17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
Message-ID: <bbaeab100512271608t7dfac458y8b8e7d12e0df57d@mail.gmail.com>

On 12/27/05, Ka-Ping Yee <python-dev at zesty.ca> wrote:
> It sounds to me like what is being proposed amounts to essentially
> "promote sys.exit to a builtin".  So why not do that?
>
> I see two options.  Either:
>
>     (a) Simply let __builtins__.exit = sys.exit.  Then we get:
>
>         >>> exit
>         <built-in function exit>
>
>         which may be enough of a clue that you type "exit()" to exit.
>
>     (b) If more handholding seems like a good idea, then:
>
>         class ExitHatch:
>             def __call__(self): sys.exit()
>             def __repr__(self): return '<Type "exit()" to exit Python.>'
>         __builtins__.exit = __builtins__.quit = ExitHatch()
>

I prefer (b) since this does need to be newbie-friendly and thus self
explaining.  I would prefer the name ExitInterpreter for the class and
including the keyboard shortcut in the message as well.

And Tim had a good point about PDAs and such; how are they supposed to
exit?  What if someone picked up Python for their Nokia S60 phone and
tried to exit from the interpreter?  Unless Nokia has tweaked
something I don't know how they would know to exit without knowing
about sys.exit() or raising SystemExit since I wouldn't know how to do
the equivalent Ctrl-D on a cell phone.

-Brett

From fredrik at pythonware.com  Wed Dec 28 01:20:42 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 28 Dec 2005 01:20:42 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com><43B172C9.4070201@v.loewis.de><17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
Message-ID: <doslos$kop$1@sea.gmane.org>

Ka-Ping Yee wrote

> It sounds to me like what is being proposed amounts to essentially
> "promote sys.exit to a builtin".

no, what's being proposed is what the subject says: a quit/exit command
that actually quits, instead of printing a "you didn't say please!" message.

</F>




From gh at ghaering.de  Wed Dec 28 00:52:54 2005
From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=)
Date: Wed, 28 Dec 2005 00:52:54 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>	<43B172C9.4070201@v.loewis.de>	<17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
Message-ID: <43B1D3D6.9000206@ghaering.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ka-Ping Yee wrote:
> [...]
>     (b) If more handholding seems like a good idea, then:
> 
>         class ExitHatch:
>             def __call__(self): sys.exit()
>             def __repr__(self): return '<Type "exit()" to exit Python.>'
>         __builtins__.exit = __builtins__.quit = ExitHatch()

That looks like a good compromise to me.

- -- Gerhard
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDsdPWdIO4ozGCH14RAsTyAKCW5mwCJ/cN+UbKICufXwmDIX9/tgCfQLJa
LaEL4a4pV7Jhnh3ry/b+CrU=
=/FQl
-----END PGP SIGNATURE-----

From alan.mcintyre at gmail.com  Wed Dec 28 01:39:48 2005
From: alan.mcintyre at gmail.com (Alan McIntyre)
Date: Tue, 27 Dec 2005 19:39:48 -0500
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <bbaeab100512271608t7dfac458y8b8e7d12e0df57d@mail.gmail.com>
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>	<43B172C9.4070201@v.loewis.de>	<17329.38860.916447.765575@montanaro.dyndns.org>	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
	<bbaeab100512271608t7dfac458y8b8e7d12e0df57d@mail.gmail.com>
Message-ID: <43B1DED4.6000000@gmail.com>

Brett Cannon wrote:

>And Tim had a good point about PDAs and such; how are they supposed to
>exit?  What if someone picked up Python for their Nokia S60 phone and
>tried to exit from the interpreter?  Unless Nokia has tweaked
>something I don't know how they would know to exit without knowing
>about sys.exit() or raising SystemExit since I wouldn't know how to do
>the equivalent Ctrl-D on a cell phone.
>  
>
The version of Python on my phone (2.2.2) doesn't recognize "quit" or
"exit," but there is a soft key conveniently labeled "Exit" which drops
out of the interactive console.

From jimjjewett at gmail.com  Wed Dec 28 02:03:36 2005
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 27 Dec 2005 20:03:36 -0500
Subject: [Python-Dev] Keep default comparisons - or add a second set?
Message-ID: <fb6fbf560512271703i29f7e24co22e2bd7aaa9c3d27@mail.gmail.com>

    Today, x < y will return a consistent result,
    even if the two objects are not comparable
    in any meaningful manner.[*]  The current plan
    for Python 3 is to instead raise TypeError.

    If this really needs to be changed (and I don't
    think it does), then there should still be a standard
    way to get a canonical ordering.

Jim Fulton asked how this canonical ordering
should be defined.

The obvious answer is "Just like the __lt__
operator today".   But it doesn't actually *need*
to be that sensible, if there are strong reasons
to prefer a simpler ordering.

The more I think about it, though, __eq__ really
does need to be honored; I don't want an
    "Are these two containers equivalent?"
test to depend on how well the memory
subsystem happened to be working.

Given that, also honoring an explicit __lt__
isn't much of an extra burden, and will make
the ordering much more useful for debugging
and output.

The only real question left is what to do when the
two objects' classes do not define a comparison
between them.

Current CPython eventually falls back to the
objects' locations in memory.  Unfortunately,
this isn't stable across different sessions, and
can mask some wrong-type bugs.

There is no general answer to the lack of stability;
if objects from multiple sessions might be compared
(Jim Fulton's BTree example), then whatever
communication channel mixes the objects will have
to provide its own wrapper for fallback comparisons.
Taking away the default fallback will (at best) change
where bugs are caught, and it won't always be earlier.

There is some value in saying "unlike objects should
not be compared by accident", so I would understand
a decision to create two types of sorting.  "Comparison"
would raise a TypeError if the pair of objects were
not comparable; "Ordering" would continue on to
"something consistent", just like sorts do today.

The catch is that both operations are needed,
and the obvious syntax (<) should go with the
operation whose semantics are not backwards-
compatible.  Given that, I don't think the
separation of operations would clarify code
in practice.

[*]  Paul Moore pointed out that comparing
Numeric arrays to other objects does raise
a TypeError.  But this was an explicit choice,
and the Numeric authors had to jump through
enough hoops to get this behavior that it isn't
likely to be a common problem.

-jJ

From ark at acm.org  Wed Dec 28 01:45:50 2005
From: ark at acm.org (Andrew Koenig)
Date: Tue, 27 Dec 2005 19:45:50 -0500
Subject: [Python-Dev] Small any/all enhancement
In-Reply-To: <20051227224835.GA16166@divmod.com>
Message-ID: <007701c60b48$100bd6c0$6402a8c0@arkdesktop>

> Of course I already knew all the alternatives using map and the generator
> expression, but I felt like mine was clearer for a reader, this is
> probably true but not enough to justify the change.

If there is to be any enhancement, I think I would prefer it to be an
enhancement to map, so that map's second and subsequent arguments can be
sequences or iterables, and its result is an iterable if any of its
arguments is an iterable.

Unfortunately, that would be an incompatible change :-(




From tismer at stackless.com  Wed Dec 28 02:13:28 2005
From: tismer at stackless.com (Christian Tismer)
Date: Wed, 28 Dec 2005 02:13:28 +0100
Subject: [Python-Dev] deque alternative
In-Reply-To: <20051227151254.GA6985@code1.codespeak.net>
References: <8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>
	<ca471dc20512141237i4f09098cudb951fbe70623085@mail.gmail.com>
	<43AE96FF.2010303@stackless.com>
	<8393fff0512251810q6aa5d583s2733e265b4bcd648@mail.gmail.com>
	<43AFE44D.6040306@stackless.com>
	<20051227151254.GA6985@code1.codespeak.net>
Message-ID: <43B1E6B8.60706@stackless.com>

Dear Armin,

> You are mentioning confusingly many levels of PyPy for this argument.

This is true, I didn't want to care.

> This is not directly related to static analysis nor to the JIT.  The
> point is just that while a Python program runs, the implementation can
> decide to start using a deque-like structure instead of a zero-based
> array for a given user list.  This can be done in any implementation of
> Python; both in PyPy and in CPython it would be done by adding checks
> and cases in the code that implements list objects.

Correct. It is neither directly related to static analysis nor
to JIT, I just drew some line of analogy between how we decide whether
to implement a list or a straight array.

Mentioning many levels of PyPy might be confusing, but these levels
are more related to each other for me than you might expect, or you
might just not expect the kind of picture I think to have. I'm
of course not confused, but tend to put different levels into
relation.

PyPy has several levels to attack/approach certain problems. It
thinks to insulate them. Actually, it is less insulating but more
structuring, trying to attach valid levels of approaches to a problem.

This crystallizes as certain ways to implement some ideas in RPython,
while it comes up in a different flavor when JIT approaches for one
level up are discussed. But all these different approaches have one
similar mindset in common, which is always seeking for the best balance
between feasibility, simplicity, comprehensibility and elegance,
in conjunction with the confidence that we can do better if we do it
right. And there are similar patterns to be observed, at every level.

PyPy gives me the advantage that I can emit a more or less wild idea
and check if it is doable, makes sense, or leads to more problems
than it solves. I cannot discuss this on the Python list, because
this is always bearing lots of other considerations, where the
hardest ones are "can we implement it", "do people like it", and
"is it worth it", where the latter was the point why I brought
the issue up, at all, trying to ride on Guido's arguments.

> As much as I like this approach I fear that it will be rejected for
> CPython, as e.g. lazily concatenated string objects were, on grounds of
> code obfuscation and unpredictability of performance.  But it's PyPy's
> goal to experiment here :-)

No fears needed, I'm not putting my person into it, again.

Guido asked me to shut up, a while ago. I did!
If I'm showing up from time to time, this is not meant
to make him happy, just to remind him on contradictions to himself.

You know CPython and its community almost as good as I know, and that
I left it to some extent quite a while ago. One reason was the
ambiguous decision of what is considerable and what not. This is
too much of personality going into it for me. PyPy gives me the
chance to discuss things, try things, go wrong, correct myself, and
in case of disagreement, nobody would object to create a branch
for a new idea until it is approved or I'm healed. This is what
I'm missing in CPython, and why I'm feeling well being with PyPy.

Stackless is a good example. In PyPy, everybody can decide to go
for Stackless and set a compiler option. There is no question
asked, everybody agrees that it can be a good thing if you need it,
so use it or not. A new feature may be a good thing as well, so
let's see if it makes sense, and categorize it if it is of general
use, or a special case, which should be treated differently, maybe
even by a real branch. The big advantage that we have is that we are
not bound to a single, hand-written implementation which has to be
maintained by hand. We can try and let the code generator get
astray, automatically. We'll fix it and don't worry.

Don't worry, I'm not expecting anything positive from python-dev,
and the only thing that makes me still unhappy is unreflected
abuses of my changed topic, but that's a minor matter of taste :-))

all the best -- chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From raymond.hettinger at verizon.net  Wed Dec 28 04:24:06 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Tue, 27 Dec 2005 22:24:06 -0500
Subject: [Python-Dev] (back to): Linked lists -- (was: deque alternative)
In-Reply-To: <8393fff0512270951n77f53f71t695ac6cc3e05577@mail.gmail.com>
Message-ID: <02d801c60b5e$29429aa0$78fdcc97@oemcomputer>

[Martin Blais]
> Now, it's not all about storage space.  What about front-insertion?
> Everytime I have to type L.insert(0, bli), somewhere that I know runs
> "often" I cringe.  If I had lists I would be able to express my
> thoughts more clearly in the computer program.  Right now, I cringe,
> and then I just shrug.

Doesn't collections.deque() meet your front-insertion needs?


Raymond


From ncoghlan at gmail.com  Wed Dec 28 05:13:17 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 28 Dec 2005 14:13:17 +1000
Subject: [Python-Dev] Small any/all enhancement
In-Reply-To: <39707FD5-B1D8-4EE4-B9E7-71C62063B1E0@redivi.com>
References: <20051227204501.GA252@divmod.com>	<8AA572FD-50E2-4781-91A5-9C6D73B7E8EA@gmail.com>	<20051227224835.GA16166@divmod.com>
	<39707FD5-B1D8-4EE4-B9E7-71C62063B1E0@redivi.com>
Message-ID: <43B210DD.7000001@gmail.com>

Bob Ippolito wrote:
> I don't see the harm in a "key" argument like sorted has, but without  
> a key argument it could be extended to take more arguments like max/ 
> min do for convenience.  e.g. any(a, b, c) instead of any((a, b, c)).

Hmm, I think you just found the use case for fixing the current lack of 
support for keyword-only arguments - it allows conveniences like this, while 
still allowing keyword arguments to tailor function behaviour.

For example, min & max could grow a "key" argument analogous to sorted's (e.g. 
to find the person with the highest score in a list of players).

(Guido's already approved the concept of permitting keyword arguments to be 
supplied after a * entry in a function call. I don't remember if he expressed 
an opinion on allowing the same syntax in a function definition to define 
keyword only arguments).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From skip at pobox.com  Wed Dec 28 05:40:56 2005
From: skip at pobox.com (skip@pobox.com)
Date: Tue, 27 Dec 2005 22:40:56 -0600
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <doslos$kop$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
	<17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
	<doslos$kop$1@sea.gmane.org>
Message-ID: <17330.5976.560419.584425@montanaro.dyndns.org>


    Fredrik> a quit/exit command that actually quits, instead of printing a
    Fredrik> "you didn't say please!" message.

I like Fredrik's idea more and more.  Without my Unix bifocals it wouldn't
occur to me that Ctrl-D is the way to exit.  Knowing Ctrl-Z is EOF on
Windows, it wouldn't occur to me that I'd also have to hit Return.  Without
my Python shades I'd never guess to exit via "raise SystemExit".  While the
raise command is "one true way", it certainly won't occur to newbies.  I
have no idea how I'd exit from Pippy or from the interpreter prompt on a
Nokia phone without it.

In short, I think it makes a lot of sense to support a bare "exit" and/or
"quit" as a completely intuitive platform-independent newbie-friendly way to
exit the interpreter.

Skip

From python-dev at zesty.ca  Wed Dec 28 06:25:13 2005
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Tue, 27 Dec 2005 23:25:13 -0600 (CST)
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <doslos$kop$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com><43B172C9.4070201@v.loewis.de><17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
	<doslos$kop$1@sea.gmane.org>
Message-ID: <Pine.LNX.4.58.0512272258520.6831@server1.LFW.org>

On Wed, 28 Dec 2005, Fredrik Lundh wrote:
> Ka-Ping Yee wrote
> > It sounds to me like what is being proposed amounts to essentially
> > "promote sys.exit to a builtin".
> no, what's being proposed is what the subject says: a quit/exit command
> that actually quits, instead of printing a "you didn't say please!" message.

Okay, that would be fine.  It's just that the solutions so far that
work without parentheses are a bit too magical for me.

Fredrik's NameError-based proposal (exit when there's an exception
with no tb_next that says "name 'exit' is not defined") causes the
interpreter to quit when you enter any expression involving 'exit'.

    print exit                  # seems surprising
    [3, 4, 5, exit]             # seems surprising
    'a' in 'xyz' or exit        # desirable or undesirable?
    del exit                    # actually happened to me
    x = exit                    # seems surprising

Reinhold's proposal (exit when sys._getframe(1).f_code.co_names
is ("exit",)) causes the interpreter to quit whenever any function
attempts to print out or represent 'exit', as long as it doesn't
mention any other variables.

    print exit                  # seems surprising
    [3, 4, 5, exit]             # seems surprising
    'a' in 'xyz' or exit        # desirable or undesirable?
    def foo():
        print exit
    foo()                       # seems surprising

I'd be happy with having Python exit when the user types just plain
'exit' without parentheses, but only in that case, not others.
However, i'm starting to think that may be impossible to implement.
I can't think of any way to make 'print exit' not exit, for example.

("'exit'" is a shorthand for "'exit' or 'quit'" above.)


-- ?!ng

From rwgk at yahoo.com  Wed Dec 28 07:32:56 2005
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Tue, 27 Dec 2005 22:32:56 -0800 (PST)
Subject: [Python-Dev] Python + Visual C++ 8.0?
In-Reply-To: <1667976656.20051228002158@myrealbox.com>
Message-ID: <20051228063256.16569.qmail@web31505.mail.mud.yahoo.com>

--- Adal Chiriliuc <adal.chiriliuc at gmail.com> wrote:
> Then you need to run mt.exe to embedd the manifest:
> mt.exe /outputresource:"cctbx_math_ext.pyd;#2" /manifest
> cctbx_math_ext.pyd.manifest

That is the magic trick! After applying the mt command to all our extensions
most of our unit tests work even with the VC7.1 compiled Python. I am very
optimistic I can get all our tests to work with a few C++ adjustments (related
to the new way STL iterators are implemented). Thanks a lot! I could not have
figured this out myself in a million years.

Cheers,
        Ralf



	
		
__________________________________ 
Yahoo! for Good - Make a difference this year. 
http://brand.yahoo.com/cybergivingweek2005/

From Scott.Daniels at Acm.Org  Wed Dec 28 07:53:37 2005
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Tue, 27 Dec 2005 22:53:37 -0800
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <fb6fbf560512271703i29f7e24co22e2bd7aaa9c3d27@mail.gmail.com>
References: <fb6fbf560512271703i29f7e24co22e2bd7aaa9c3d27@mail.gmail.com>
Message-ID: <dotcp2$ju3$1@sea.gmane.org>

Jim Jewett wrote:
> Jim Fulton asked how this canonical ordering
> should be defined.
> 
> The obvious answer is "Just like the __lt__
> operator today".   But it doesn't actually *need*
> to be that sensible, if there are strong reasons
> to prefer a simpler ordering.
> 
> The more I think about it, though, __eq__ really
> does need to be honored; I don't want an
>     "Are these two containers equivalent?"
> test to depend on how well the memory
> subsystem happened to be working.
> 
> Given that, also honoring an explicit __lt__
> isn't much of an extra burden, and will make
> the ordering much more useful for debugging
> and output.

Tell me:
 >>> a = [0] * 3
 >>> b = [0] * 3
 >>> a[0] = b
 >>> b[0] = a

What order should a and b have?

--Scott David Daniels
Scott.Daniels at Acm.Org


From eric.nieuwland at xs4all.nl  Wed Dec 28 08:06:09 2005
From: eric.nieuwland at xs4all.nl (Eric Nieuwland)
Date: Wed, 28 Dec 2005 08:06:09 +0100
Subject: [Python-Dev] Small any/all enhancement
In-Reply-To: <8AA572FD-50E2-4781-91A5-9C6D73B7E8EA@gmail.com>
References: <20051227204501.GA252@divmod.com>
	<8AA572FD-50E2-4781-91A5-9C6D73B7E8EA@gmail.com>
Message-ID: <efdf14b4ecbb0e972dac7899ff88a761@xs4all.nl>

Alex Martelli wrote:
> On Dec 27, 2005, at 12:45 PM, Valentino Volonghi aka Dialtone wrote:
>     ...
>> any(iterable, test=bool) and all(iterable, test=bool)
>     ...
>> any(some_objects, test=operator.attrgetter('some_attribute'))
>
> Why would that be better than
> any(o.some_attribute for o in some_objects)
> ?
>
>> def zerop(x):
>>     return x==0
>>
>> all(some_objects, zerop)
>
> and why would that be better than
> all(o==0 for o in some_objects)
> ?

all() can be terminated at the first false element. For very long 
sequences this has important performance benefits. Besides, it makes 
all(seq,pred) the equivalent of pred(seq[0]) and  pred(seq[1]) and 
pred(seq[2]) and ...


From fredrik at pythonware.com  Wed Dec 28 08:56:14 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 28 Dec 2005 08:56:14 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org><dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com><43B172C9.4070201@v.loewis.de><17329.38860.916447.765575@montanaro.dyndns.org><Pine.LNX.4.58.0512271651040.6831@server1.LFW.org><doslos$kop$1@sea.gmane.org>
	<Pine.LNX.4.58.0512272258520.6831@server1.LFW.org>
Message-ID: <dotgf0$qn2$1@sea.gmane.org>

Ka-Ping Yee wrote:

> Fredrik's NameError-based proposal (exit when there's an exception
> with no tb_next that says "name 'exit' is not defined") causes the
> interpreter to quit when you enter any expression involving 'exit'.
>
>     print exit                  # seems surprising
>     [3, 4, 5, exit]             # seems surprising
>     'a' in 'xyz' or exit        # desirable or undesirable?
>     del exit                    # actually happened to me
>     x = exit                    # seems surprising

the easiest way to solve this that I can think of right now is to add a
new sys variable that contains a copy of the most recent line read by
the interactive prompt.

    if sys.commandline.strip() in ("exit", "quit"):
        sys.exit()

</F>




From reinhold-birkenfeld-nospam at wolke7.net  Wed Dec 28 10:24:57 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Wed, 28 Dec 2005 10:24:57 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <17330.5976.560419.584425@montanaro.dyndns.org>
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>	<43B172C9.4070201@v.loewis.de>	<17329.38860.916447.765575@montanaro.dyndns.org>	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>	<doslos$kop$1@sea.gmane.org>
	<17330.5976.560419.584425@montanaro.dyndns.org>
Message-ID: <dotll9$7bg$1@sea.gmane.org>

skip at pobox.com wrote:
>     Fredrik> a quit/exit command that actually quits, instead of printing a
>     Fredrik> "you didn't say please!" message.
> 
> I like Fredrik's idea more and more.  Without my Unix bifocals it wouldn't
> occur to me that Ctrl-D is the way to exit.  Knowing Ctrl-Z is EOF on
> Windows, it wouldn't occur to me that I'd also have to hit Return.  Without
> my Python shades I'd never guess to exit via "raise SystemExit".  While the
> raise command is "one true way", it certainly won't occur to newbies.  I
> have no idea how I'd exit from Pippy or from the interpreter prompt on a
> Nokia phone without it.
> 
> In short, I think it makes a lot of sense to support a bare "exit" and/or
> "quit" as a completely intuitive platform-independent newbie-friendly way to
> exit the interpreter.

+1.

Reinhold

-- 
Mail address is perfectly valid!


From martin at v.loewis.de  Wed Dec 28 10:54:27 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 28 Dec 2005 10:54:27 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <17330.5976.560419.584425@montanaro.dyndns.org>
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>	<43B172C9.4070201@v.loewis.de>	<17329.38860.916447.765575@montanaro.dyndns.org>	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>	<doslos$kop$1@sea.gmane.org>
	<17330.5976.560419.584425@montanaro.dyndns.org>
Message-ID: <43B260D3.8070601@v.loewis.de>

skip at pobox.com wrote:
> In short, I think it makes a lot of sense to support a bare "exit" and/or
> "quit" as a completely intuitive platform-independent newbie-friendly way to
> exit the interpreter.

I can readily agree to this part of Fredrik's proposal. What slightly
bothers me is the hackish nature of the proposed implementation.

Regards,
Martin

From martin at v.loewis.de  Wed Dec 28 11:00:41 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 28 Dec 2005 11:00:41 +0100
Subject: [Python-Dev] Small any/all enhancement
In-Reply-To: <efdf14b4ecbb0e972dac7899ff88a761@xs4all.nl>
References: <20051227204501.GA252@divmod.com>	<8AA572FD-50E2-4781-91A5-9C6D73B7E8EA@gmail.com>
	<efdf14b4ecbb0e972dac7899ff88a761@xs4all.nl>
Message-ID: <43B26249.6070406@v.loewis.de>

Eric Nieuwland wrote:
>>all(o==0 for o in some_objects)
>>?
> 
> 
> all() can be terminated at the first false element. For very long 
> sequences this has important performance benefits. Besides, it makes 
> all(seq,pred) the equivalent of pred(seq[0]) and  pred(seq[1]) and 
> pred(seq[2]) and ...

And so does the version with generator expressions: Alex' expression
will also terminate with the first false statement; it is equivalent
to some_objects[0]==0 and some_objects[1]==0 and ...

Regards,
Martin

From rhamph at gmail.com  Wed Dec 28 11:01:39 2005
From: rhamph at gmail.com (Adam Olsen)
Date: Wed, 28 Dec 2005 03:01:39 -0700
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <dotcp2$ju3$1@sea.gmane.org>
References: <fb6fbf560512271703i29f7e24co22e2bd7aaa9c3d27@mail.gmail.com>
	<dotcp2$ju3$1@sea.gmane.org>
Message-ID: <aac2c7cb0512280201k5a725223id0885d417f376b42@mail.gmail.com>

On 12/27/05, Scott David Daniels <Scott.Daniels at acm.org> wrote:
> Tell me:
>  >>> a = [0] * 3
>  >>> b = [0] * 3
>  >>> a[0] = b
>  >>> b[0] = a
>
> What order should a and b have?

More gems:
>>> Decimal(3) < 4.0
False
>>> Decimal(3) == 3.0
False
>>> Decimal(3) > 4.0
True

And the pi?ce de r?sistance:
>>> l = [2.0, Decimal(3), 3.5, Decimal(4), 5, 3]
>>> for i in range(5): random.shuffle(l); print sorted(l)
[2.0, Decimal("3"), 3, 3.5, Decimal("4"), 5]
[2.0, 3.5, Decimal("3"), 3, Decimal("4"), 5]
[2.0, Decimal("3"), 3, 3.5, Decimal("4"), 5]
[2.0, 3.5, Decimal("3"), 3, Decimal("4"), 5]
[2.0, 3, 3.5, Decimal("3"), Decimal("4"), 5]

(The positions of 3 and Decimal("3") should be swapping, but the
position of 3.5 should not.)

--
Adam Olsen, aka Rhamphoryncus

From fredrik at pythonware.com  Wed Dec 28 11:04:55 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 28 Dec 2005 11:04:55 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org><dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>	<43B172C9.4070201@v.loewis.de>	<17329.38860.916447.765575@montanaro.dyndns.org>	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>	<doslos$kop$1@sea.gmane.org><17330.5976.560419.584425@montanaro.dyndns.org>
	<43B260D3.8070601@v.loewis.de>
Message-ID: <doto08$elm$1@sea.gmane.org>

Martin v. Löwis wrote:

> > In short, I think it makes a lot of sense to support a bare "exit" and/or
> > "quit" as a completely intuitive platform-independent newbie-friendly way to
> > exit the interpreter.
>
> I can readily agree to this part of Fredrik's proposal. What slightly
> bothers me is the hackish nature of the proposed implementation.

here's my current proposal (see ping's post and my reply for background):

    if isinstance(exc_value, NameError) and not exc_info.tb_next:
        text = exc_value[0]
        name = ... extract name from nameerror string ...
        if sys.commandline.strip() == name:
            if name in ("exit", "quit"):
                raise SystemExit
            if name == "help":
                help()
                return
    ...

any suggestions on how to improve this ?

</F>




From martin at v.loewis.de  Wed Dec 28 11:24:45 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 28 Dec 2005 11:24:45 +0100
Subject: [Python-Dev] A few questions about setobject
In-Reply-To: <b348a0850512261416s5865462btf08a9613f0780d36@mail.gmail.com>
References: <b348a0850512261416s5865462btf08a9613f0780d36@mail.gmail.com>
Message-ID: <43B267ED.6020804@v.loewis.de>

Noam Raphael wrote:
> Is this desirable? 

Not sure what "this" refers to in your message: the text of the C API
documentation certainly is desirable as it stands (although it should
be clearer as to whether struct names should be prefixed).

The setentry typedef clearly violates the principles of the API, so
it should be renamed.

> Even if it is, it seems that the second sentence
> contradicts the first sentence.

Why does that seem so? To quote, the first sentence is

'All user visible names defined by Python.h (except those defined by
the included standard headers) have one of the prefixes "Py" or "_Py".'

and the second sentence is

'Names beginning with "_Py" are for internal use by the Python
implementation and should not be used by extension writers.'

I cannot see any contradiction between these.

> Perhaps the header file should stick
> with writing "struct { long hash; PyObject *key; }" three times (or
> define it in a macro and then undefine it), and the typedef be left to
> the .c file?

That would not be conforming to the C language standard (although
accepted by most compilers).

> I think it should be ok because it's never used
> really as a PyObject. Am I missing something? (Ok, I now thought that
> maybe it's because some parts don't treat dummy elements specially.
> But it seems to me that most parts do treat them specially, so perhaps
> it would be better to make a few changes so that all parts will treat
> them specially?)

In principle, you are right. One place that doesn't special-case the
dummy is set_clear_internal (in fact, the Py_DEBUG assertion is
completely useless there, AFAICT).

The tricky question is: can we be certain that we find all places,
in all code paths, where we have to special-case the dummy? Having
PyObject* which don't point to PyObject is error-prone.

Also, what would we gain? If you think it is speed: I doubt it. In
one place, a comment suggests that actually seeing the dummy element
is so much more unlikely than the other cases that, for performance,
the test for the dummy is done last. We would lose additional speed
in the cases where the dummy isn't yet considered.

> 3) The type of the result of a binary operator applied on a set and a
> frozenset is the type of the left set. You are welcomed to ignore
> this, but I just wanted to say that it seems to me better to make the
> operator symmetric, and to return a frozenset only if both sets are
> frozen.

How would you implement this? The result is obtained from copying the
left operand, and then applying the other operand. This is done so
that set subtyping becomes possible:

>>> class myset(set):pass
...
>>> x=myset([2,6])
>>> y=set([2,6])
>>> x.union(y)
myset([2, 6])

So if the result is not obtained by copying the left operand first,
how would you compute the result type, so that this example still
works?

Regards,
Martin

From jeremy.kloth at fourthought.com  Wed Dec 28 11:29:16 2005
From: jeremy.kloth at fourthought.com (Jeremy Kloth)
Date: Wed, 28 Dec 2005 03:29:16 -0700
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org><dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com><43B172C9.4070201@v.loewis.de><17329.38860.916447.765575@montanaro.dyndns.org><Pine.LNX.4.58.0512271651040.6831@server1.LFW.org><doslos$kop$1@sea.gmane.org>
	<Pine.LNX.4.58.0512272258520.6831@server1.LFW.org>
Message-ID: <dotpb4$i01$1@sea.gmane.org>

Ka-Ping Yee wrote:
> I'd be happy with having Python exit when the user types just plain
> 'exit' without parentheses, but only in that case, not others.
> However, i'm starting to think that may be impossible to implement.
> I can't think of any way to make 'print exit' not exit, for example.

OK, here's one:

def quithook(obj, _exit=__builtins__.exit, _displayhook=sys.displayhook):
    if obj is _exit: raise SystemExit
    _displayhook(obj)
sys.displayhook = quithook

It does, however, fall into the whole issue of chaining that Skip brought up
earlier.

--
Jeremy Kloth
Fourthought, Inc.
http://fourthought.com/
http://4suite.org/ 




From martin at v.loewis.de  Wed Dec 28 11:38:31 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 28 Dec 2005 11:38:31 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <doto08$elm$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org><dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>	<43B172C9.4070201@v.loewis.de>	<17329.38860.916447.765575@montanaro.dyndns.org>	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>	<doslos$kop$1@sea.gmane.org><17330.5976.560419.584425@montanaro.dyndns.org>	<43B260D3.8070601@v.loewis.de>
	<doto08$elm$1@sea.gmane.org>
Message-ID: <43B26B27.4030008@v.loewis.de>

Fredrik Lundh wrote:
> any suggestions on how to improve this ?

Introducing sys.commandline is fine; overriding sys.excepthook
still worrisome.

What's wrong with triggering this in some __repr__ implementation?

If an excepthook must be installed, why couldn't the previous
excepthook be preserved, and called in case the exception is
not what you are looking for?
Of course, even if this excepthook behaves friendly, some other
package might overwrite excepthook without preserving yours,
in which case people would "sometimes" get a NameError when
they try to exit.

Regards,
Martin

From fredrik at pythonware.com  Wed Dec 28 11:40:33 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 28 Dec 2005 11:40:33 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org><dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com><43B172C9.4070201@v.loewis.de><17329.38860.916447.765575@montanaro.dyndns.org><Pine.LNX.4.58.0512271651040.6831@server1.LFW.org><doslos$kop$1@sea.gmane.org><Pine.LNX.4.58.0512272258520.6831@server1.LFW.org>
	<dotpb4$i01$1@sea.gmane.org>
Message-ID: <dotq33$k07$1@sea.gmane.org>

Jeremy Kloth wrote:

> Ka-Ping Yee wrote:
> > I'd be happy with having Python exit when the user types just plain
> > 'exit' without parentheses, but only in that case, not others.
> > However, i'm starting to think that may be impossible to implement.
> > I can't think of any way to make 'print exit' not exit, for example.
>
> OK, here's one:
>
> def quithook(obj, _exit=__builtins__.exit, _displayhook=sys.displayhook):
>     if obj is _exit: raise SystemExit
>     _displayhook(obj)
> sys.displayhook = quithook
>
> It does, however, fall into the whole issue of chaining that Skip brought up
> earlier.

as well as various introspection-related issues:

    >>> import foo
    >>> cb = foo.getcallback()
    >>> # let's see what it is
    >>> cb

    $

(by the way, it's "__builtin__", not "__builtins__".  the former is a module,
the latter a CPython implementation detail)

</F>




From martin at v.loewis.de  Wed Dec 28 11:55:31 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 28 Dec 2005 11:55:31 +0100
Subject: [Python-Dev] deque alternative
In-Reply-To: <1f7befae0512260956o695150e7x5a56c0d8972d1317@mail.gmail.com>
References: <8393fff0512251810q6aa5d583s2733e265b4bcd648@mail.gmail.com>	<1f7befae0512251946y31552befx468d1f7dda23b6e1@mail.gmail.com>	<20051225204328.BECE.JCARLSON@uci.edu>	<8393fff0512260807y53a73baar197e07cf8b8eaa8c@mail.gmail.com>	<39C8CCEF-21D9-4DB2-AE03-374748C6CDB7@fuhm.net>
	<1f7befae0512260956o695150e7x5a56c0d8972d1317@mail.gmail.com>
Message-ID: <43B26F23.6020004@v.loewis.de>

Tim Peters wrote:
> You seem to be ignoring possiblities for sharing across lists, and
> such sharing is natural in many graph algorithms.

No doubt cons cells are a useful construct. I think Martin Blais
(and others) advocated a plain list container type, only implemented
as a linked list, instead of a as a vector. The subject is
"deque alternative", after all.

Typically, you either have sharing, or you have appending, but not
both. If sharing is what you want, 2-tuples (pairs) provide an
adequate API (IMO). If you want a deque alternative, sharing won't
happen.

Regards,
Martin

From martin at v.loewis.de  Wed Dec 28 11:57:49 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 28 Dec 2005 11:57:49 +0100
Subject: [Python-Dev] deque alternative
In-Reply-To: <5.1.1.6.0.20051225215827.02233c30@mail.telecommunity.com>
References: <43AE96FF.2010303@stackless.com>	<8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>	<ca471dc20512141237i4f09098cudb951fbe70623085@mail.gmail.com>	<43AE96FF.2010303@stackless.com>
	<5.1.1.6.0.20051225215827.02233c30@mail.telecommunity.com>
Message-ID: <43B26FAD.9010700@v.loewis.de>

Phillip J. Eby wrote:
> Since I routinely use 2-item tuples (twoples?)

I've been using "pairs" to describe that datatype. Not sure
how common it is in English, but in German, "Zweitupel"
is often called "Paar".

Regards,
Martin

From mwh at python.net  Wed Dec 28 11:59:16 2005
From: mwh at python.net (Michael Hudson)
Date: Wed, 28 Dec 2005 10:59:16 +0000
Subject: [Python-Dev] status of development documentation
In-Reply-To: <ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
	(Neal Norwitz's message of "Fri, 23 Dec 2005 23:14:48 -0800")
References: <20051221095628.BE79.JCARLSON@uci.edu>
	<1135196162.14495.15.camel@geddy.wooz.org>
	<doci50$nom$1@sea.gmane.org> <2m4q512vub.fsf@starship.python.net>
	<1FBF0E3A-0980-47BD-9318-56F8EB369FF2@lag.net>
	<dogduu$o1m$1@sea.gmane.org>
	<5A27B555-36CB-46E4-A4EE-6F14FDC3AF3D@lag.net>
	<17324.31929.979592.958222@montanaro.dyndns.org>
	<ee2a432c0512231828l6df48235s4587e29488947400@mail.gmail.com>
	<1f7befae0512232244t16598252y1deefcf2d46f1b99@mail.gmail.com>
	<ee2a432c0512232314w2492dcf4sce83969736193a13@mail.gmail.com>
Message-ID: <2mek3x1o8r.fsf@starship.python.net>

Neal Norwitz <nnorwitz at gmail.com> writes:

> On 12/23/05, Tim Peters <tim.peters at gmail.com> wrote:
>> _assumed_ this was known damage everywhere so was waiting for someone
>> else to fix it ;-)  (A parenthentical question:  is there a reason you
>> don't pass -uall to regrtest.py?)
>
> It's calling make test.

You could call "make testall" instead?

Cheers,
mwh
(catching up, so: thanks!)

-- 
  I never disputed the Perl hacking skill of the Slashdot creators. 
  My objections are to the editors' taste, the site's ugly visual 
  design, and the Slashdot community's raging stupidity.
     -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq

From fredrik at pythonware.com  Wed Dec 28 11:51:51 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 28 Dec 2005 11:51:51 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org><dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>	<43B172C9.4070201@v.loewis.de>	<17329.38860.916447.765575@montanaro.dyndns.org>	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>	<doslos$kop$1@sea.gmane.org><17330.5976.560419.584425@montanaro.dyndns.org>	<43B260D3.8070601@v.loewis.de><doto08$elm$1@sea.gmane.org>
	<43B26B27.4030008@v.loewis.de>
Message-ID: <dotqo9$ljq$1@sea.gmane.org>

Martin v. Löwis wrote:

> Introducing sys.commandline is fine; overriding sys.excepthook
> still worrisome.
>
> What's wrong with triggering this in some __repr__ implementation?

because simple introspection may exit your program.  unexpected
exits are a lot more annoying than unexpected non-exits.

> If an excepthook must be installed, why couldn't the previous
> excepthook be preserved, and called in case the exception is
> not what you are looking for?

this is done in site.py, before sitecustomize is loaded.  I'm not sure
how anyone else would be able to squeeze in an excepthook at this
point, even if they wanted...

> Of course, even if this excepthook behaves friendly, some other
> package might overwrite excepthook without preserving yours,
> in which case people would "sometimes" get a NameError when
> they try to exit.

sure, but people may sometimes get another result if they rebind
exit (e.g. from sys import *), plug in a broken displayhook, or other-
wise mess up their system.

as long the documentation mentions how this works, and urges
excepthook developers to be careful, I don't really see this as a
problem.

</F>




From fredrik at pythonware.com  Wed Dec 28 12:09:37 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 28 Dec 2005 12:09:37 +0100
Subject: [Python-Dev] floating point literals don't work in non-US locale in
	2.5
Message-ID: <dotrpj$o9o$1@sea.gmane.org>

someone recently broke floating point literals in a rather spectacular
way:

$ export LANG=sv_SE.utf8
$ ./python
Python 2.5a0 (41806M, Dec 25 2005, 12:12:29)
Type "help", "copyright", "credits" or "license" for more information.
>>> 3.14
3.1400000000000001
>>> import locale
>>> locale.setlocale(locale.LC_ALL, "")
'sv_SE.utf8'
>>> 3.14
3.0

more here:

http://www.python.org/sf/1391872

</F>




From mwh at python.net  Wed Dec 28 12:24:23 2005
From: mwh at python.net (Michael Hudson)
Date: Wed, 28 Dec 2005 11:24:23 +0000
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <17330.5976.560419.584425@montanaro.dyndns.org> (skip@pobox.com's
	message of "Tue, 27 Dec 2005 22:40:56 -0600")
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
	<17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
	<doslos$kop$1@sea.gmane.org>
	<17330.5976.560419.584425@montanaro.dyndns.org>
Message-ID: <2m7j9p1n2w.fsf@starship.python.net>

skip at pobox.com writes:

>     Fredrik> a quit/exit command that actually quits, instead of printing a
>     Fredrik> "you didn't say please!" message.
>
> I like Fredrik's idea more and more.

The thing that bothers me about it is that the standard way you tell
python to do something is "call a function" -- to me, a special case
for exiting the interpreter seems out of proportion.

In other news, clever hacks with tb_next and so on also seem
excessive.  Why not have the equivalent of "if input.rstrip() ==
'exit': sys.exit()" in the implementation of the interactive
interpreter?

Cheers,
mwh

-- 
  My first thought was someone should offer Mr. Bush elocution
  lessons.  But he'd probably say he knew how to work them chairs
  already.                            -- Internet Oracularity #1294-01

From pedronis at strakt.com  Wed Dec 28 12:57:29 2005
From: pedronis at strakt.com (Samuele Pedroni)
Date: Wed, 28 Dec 2005 12:57:29 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <2m7j9p1n2w.fsf@starship.python.net>
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>	<43B172C9.4070201@v.loewis.de>	<17329.38860.916447.765575@montanaro.dyndns.org>	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>	<doslos$kop$1@sea.gmane.org>	<17330.5976.560419.584425@montanaro.dyndns.org>
	<2m7j9p1n2w.fsf@starship.python.net>
Message-ID: <43B27DA9.2080009@strakt.com>

Michael Hudson wrote:
> skip at pobox.com writes:
> 
> 
>>    Fredrik> a quit/exit command that actually quits, instead of printing a
>>    Fredrik> "you didn't say please!" message.
>>
>>I like Fredrik's idea more and more.
> 
> 
> The thing that bothers me about it is that the standard way you tell
> python to do something is "call a function" -- to me, a special case
> for exiting the interpreter seems out of proportion.
> 

also worth remarking is that is quite a FAQ at least for Jython,
how to convince python to accept bare words (plus args) as function
calls. So this magic is confusing and in the wrong direction
from that point of view, given that we don't plan to support that
at all.



> In other news, clever hacks with tb_next and so on also seem
> excessive.  Why not have the equivalent of "if input.rstrip() ==
> 'exit': sys.exit()" in the implementation of the interactive
> interpreter?
> 
> Cheers,
> mwh
> 


From fredrik at pythonware.com  Wed Dec 28 12:57:55 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 28 Dec 2005 12:57:55 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com><43B172C9.4070201@v.loewis.de><17329.38860.916447.765575@montanaro.dyndns.org><Pine.LNX.4.58.0512271651040.6831@server1.LFW.org><doslos$kop$1@sea.gmane.org><17330.5976.560419.584425@montanaro.dyndns.org>
	<2m7j9p1n2w.fsf@starship.python.net>
Message-ID: <dotuk5$v1u$1@sea.gmane.org>

Michael Hudson wrote:

> In other news, clever hacks with tb_next and so on also seem
> excessive.  Why not have the equivalent of "if input.rstrip() ==
> 'exit': sys.exit()" in the implementation of the interactive
> interpreter?

that would turn exit and quit into reserved keywords.

</F>




From noamraph at gmail.com  Wed Dec 28 12:58:38 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Wed, 28 Dec 2005 13:58:38 +0200
Subject: [Python-Dev] A few questions about setobject
In-Reply-To: <43B267ED.6020804@v.loewis.de>
References: <b348a0850512261416s5865462btf08a9613f0780d36@mail.gmail.com>
	<43B267ED.6020804@v.loewis.de>
Message-ID: <b348a0850512280358x27c93c3n8be2c99d9416fc95@mail.gmail.com>

On 12/28/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> The setentry typedef clearly violates the principles of the API, so
> it should be renamed.
(And so will _setobject, although I think it will be much easier to remove it.)
>
> > Perhaps the header file should stick
> > with writing "struct { long hash; PyObject *key; }" three times (or
> > define it in a macro and then undefine it), and the typedef be left to
> > the .c file?
>
> That would not be conforming to the C language standard (although
> accepted by most compilers).

Can you explain why it is not conforming to the standard? Can't a
typedef be used intechangably with the original type?

> Not sure what "this" refers to in your message: the text of the C API
> documentation certainly is desirable as it stands (although it should
> be clearer as to whether struct names should be prefixed).

> > Even if it is, it seems that the second sentence
> > contradicts the first sentence.
>
> Why does that seem so? To quote, the first sentence is
>
> 'All user visible names defined by Python.h (except those defined by
> the included standard headers) have one of the prefixes "Py" or "_Py".'
>
> and the second sentence is
>
> 'Names beginning with "_Py" are for internal use by the Python
> implementation and should not be used by extension writers.'
>
> I cannot see any contradiction between these.
>
Oops. It's the first and the third:
The first: All user visible names defined by Python.h (except those
defined by the included standard headers) have one of the prefixes
"Py" or "_Py".
The third: Structure member names do not have a reserved prefix.

I think that "structure member names" refers to things like setentry.
The third sentence contradicts the first since structure member names
are user visible names. Anyway, it seems to me best if the third
sentence will be removed and all names will start with Py or _Py.
>
> > I think it should be ok because it's never used
> > really as a PyObject. Am I missing something? (Ok, I now thought that
> > maybe it's because some parts don't treat dummy elements specially.
> > But it seems to me that most parts do treat them specially, so perhaps
> > it would be better to make a few changes so that all parts will treat
> > them specially?)
>
> In principle, you are right. One place that doesn't special-case the
> dummy is set_clear_internal (in fact, the Py_DEBUG assertion is
> completely useless there, AFAICT).
>
> The tricky question is: can we be certain that we find all places,
> in all code paths, where we have to special-case the dummy? Having
> PyObject* which don't point to PyObject is error-prone.
>
> Also, what would we gain? If you think it is speed: I doubt it. In
> one place, a comment suggests that actually seeing the dummy element
> is so much more unlikely than the other cases that, for performance,
> the test for the dummy is done last. We would lose additional speed
> in the cases where the dummy isn't yet considered.
>
Ok, I tried. It took me 25 minutes to change the code, while going
over every occurence of "key" and "decref" in setobject.c. (It seems
to me that the dummy element can only be accessed from entry->key.)
Most of the code isn't bothered by the dummy element, since it uses
the data structure in a more abstract way. I think that it simplifies
the code, while adding a condition in only two places, which I don't
think should make anything noticably slower. The result passes the
complete test suite. In one sentence: I think that it makes the code
slightly better, and the risk is pretty low.

I thought to post it as a patch, but sourceforge didn't work for me,
and it's not that long, so I paste it at the end of this message. Feel
free to do whatever you want with it.

> > 3) The type of the result of a binary operator applied on a set and a
> > frozenset is the type of the left set. You are welcomed to ignore
> > this, but I just wanted to say that it seems to me better to make the
> > operator symmetric, and to return a frozenset only if both sets are
> > frozen.
>
> How would you implement this? The result is obtained from copying the
> left operand, and then applying the other operand. This is done so
> that set subtyping becomes possible:
>
> >>> class myset(set):pass
> ...
> >>> x=myset([2,6])
> >>> y=set([2,6])
> >>> x.union(y)
> myset([2, 6])
>
> So if the result is not obtained by copying the left operand first,
> how would you compute the result type, so that this example still
> works?
>
The behaviour will change to work like in other types - the returned
value will be of the base type:
>>> class MyInt(int): pass
...
>>> x = MyInt(3)
>>> y = 5
>>> x.__add__(y)
8

I'm not talking about backwards compatibility - I'm just currently
asking if others also feel that the symmetric version is preferable.

Ok, here's the diff:

=== modified file 'Objects/setobject.c'
--- Objects/setobject.c
+++ Objects/setobject.c
@@ -13,8 +13,12 @@
 /* This must be >= 1. */
 #define PERTURB_SHIFT 5

-/* Object used as dummy key to fill deleted entries */
-static PyObject *dummy = NULL; /* Initialized by first call to
make_new_set() */
+/* Object used as dummy key to fill deleted entries.
+ * The only requirement is that it won't be a valid pointer to a
+ * PyObject, so it is instead a pointer to a dummy int.
+ */
+static int dummy_int;
+static PyObject *dummy = (PyObject *)(&dummy_int);

 #define INIT_NONZERO_SET_SLOTS(so) do {                                \
        (so)->table = (so)->smalltable;                         \
@@ -199,7 +203,6 @@
                entry->key = key;
                entry->hash = hash;
                so->used++;
-               Py_DECREF(dummy);
        } else {
                /* ACTIVE */
                Py_DECREF(key);
@@ -283,8 +286,6 @@
                } else if (entry->key == dummy) {
                        /* DUMMY */
                        --i;
-                       assert(entry->key == dummy);
-                       Py_DECREF(entry->key);
                } else {
                        /* ACTIVE */
                        --i;
@@ -356,7 +357,6 @@
        if (entry->key == NULL  ||  entry->key == dummy)
                return DISCARD_NOTFOUND;
        old_key = entry->key;
-       Py_INCREF(dummy);
        entry->key = dummy;
        so->used--;
        Py_DECREF(old_key);
@@ -383,7 +383,6 @@
        if (entry->key == NULL  ||  entry->key == dummy)
                return DISCARD_NOTFOUND;
        old_key = entry->key;
-       Py_INCREF(dummy);
        entry->key = dummy;
        so->used--;
        Py_DECREF(old_key);
@@ -439,14 +438,12 @@
                assert(i < n);
                ++i;
 #endif
-               if (entry->key) {
+               if (entry->key != NULL) {
                        --fill;
-                       Py_DECREF(entry->key);
-               }
-#ifdef Py_DEBUG
-               else
-                       assert(entry->key == NULL || entry->key == dummy);
-#endif
+                       if (entry->key != dummy) {
+                               Py_DECREF(entry->key);
+                       }
+               }
        }

        if (table_is_malloced)
@@ -499,9 +496,11 @@
                PyObject_ClearWeakRefs((PyObject *) so);

        for (entry = so->table; fill > 0; entry++) {
-               if (entry->key) {
+               if (entry->key != NULL) {
                        --fill;
-                       Py_DECREF(entry->key);
+                       if (entry->key != dummy) {
+                               Py_DECREF(entry->key);
+                       }
                }
        }
        if (so->table != so->smalltable)
@@ -661,7 +660,6 @@
                }
        }
        key = entry->key;
-       Py_INCREF(dummy);
        entry->key = dummy;
        so->used--;
        so->table[0].hash = i + 1;  /* next place to start */
@@ -889,12 +887,6 @@
 {
        register PySetObject *so = NULL;

-       if (dummy == NULL) { /* Auto-initialize dummy */
-               dummy = PyString_FromString("<dummy key>");
-               if (dummy == NULL)
-                       return NULL;
-       }
-
        /* create PySetObject structure */
        if (num_free_sets &&
            (type == &PySet_Type  ||  type == &PyFrozenSet_Type)) {
@@ -971,7 +963,6 @@
                so = free_sets[num_free_sets];
                PyObject_GC_Del(so);
        }
-       Py_XDECREF(dummy);
        Py_XDECREF(emptyfrozenset);
 }

From martin at v.loewis.de  Wed Dec 28 13:27:06 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 28 Dec 2005 13:27:06 +0100
Subject: [Python-Dev] A few questions about setobject
In-Reply-To: <b348a0850512280358x27c93c3n8be2c99d9416fc95@mail.gmail.com>
References: <b348a0850512261416s5865462btf08a9613f0780d36@mail.gmail.com>	
	<43B267ED.6020804@v.loewis.de>
	<b348a0850512280358x27c93c3n8be2c99d9416fc95@mail.gmail.com>
Message-ID: <43B2849A.1060204@v.loewis.de>

Noam Raphael wrote:
>>The setentry typedef clearly violates the principles of the API, so
>>it should be renamed.
> 
> (And so will _setobject, although I think it will be much easier to remove it.)

That's not that certain. setentry is a typedef; _setobject is a tag
name of a struct. The documentation is silent as to whether tag names
also follow the prefixing rules. Many tag names don't, e.g. PyObject
is a typedef for struct _object.

>>That would not be conforming to the C language standard (although
>>accepted by most compilers).
> 
> 
> Can you explain why it is not conforming to the standard? Can't a
> typedef be used intechangably with the original type?

6.2.7 of ISO C says

       6.2.7  Compatible type and composite type

       [#1] Two types have compatible type if their types  are  the
       same.   Additional  rules  for determining whether two types
       are compatible are described in 6.7.2 for  type  specifiers,
       in   6.7.3   for   type   qualifiers,   and   in  6.7.5  for
       declarators.46)   Moreover,   two   structure,   union,   or
       enumerated  types declared in separate translation units ...

We are not dealing with separate translation units here, so I cut
the paragraph. I'm also ommitting several other exceptions where
types are compatible (e.g. enum types are compatible with the
underlying integral type). Pointer types are compatible if the
underlying types are compatible, and the types have the same
qualifiers.

So if you mention an equally-layouted struct twice in the
same translation unit, you still get incompatible types. As a result,
pointers to these structs are also incompatible, so you cannot
assign between them.

> The first: All user visible names defined by Python.h (except those
> defined by the included standard headers) have one of the prefixes
> "Py" or "_Py".
> The third: Structure member names do not have a reserved prefix.
> 
> I think that "structure member names" refers to things like setentry.

Ah. It doesn't. It means things like hash and key (for setentry),
or fill, used, mask, table, ... (for struct _setobject).

That they don't use a name prefix means that there could be a
collision with macro names (and indeed, that has happened).

> The third sentence contradicts the first since structure member names
> are user visible names. Anyway, it seems to me best if the third
> sentence will be removed and all names will start with Py or _Py.

No. Do you really want ob_type to be renamed to Py_ob_type?

Regards,
Martin

From mwh at python.net  Wed Dec 28 13:30:07 2005
From: mwh at python.net (Michael Hudson)
Date: Wed, 28 Dec 2005 12:30:07 +0000
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <dotuk5$v1u$1@sea.gmane.org> (Fredrik Lundh's message of "Wed,
	28 Dec 2005 12:57:55 +0100")
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
	<17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
	<doslos$kop$1@sea.gmane.org>
	<17330.5976.560419.584425@montanaro.dyndns.org>
	<2m7j9p1n2w.fsf@starship.python.net> <dotuk5$v1u$1@sea.gmane.org>
Message-ID: <2mzmmlz9o0.fsf@starship.python.net>

"Fredrik Lundh" <fredrik at pythonware.com> writes:

> Michael Hudson wrote:
>
>> In other news, clever hacks with tb_next and so on also seem
>> excessive.  Why not have the equivalent of "if input.rstrip() ==
>> 'exit': sys.exit()" in the implementation of the interactive
>> interpreter?
>
> that would turn exit and quit into reserved keywords.

In what sense?  Not in the sense of "things in single quotes in
Grammar"...

Cheers,
mwh

-- 
  There are two kinds of large software systems: those that evolved
  from small systems and those that don't work.
                           -- Seen on slashdot.org, then quoted by amk

From CAMBO9 at aol.com  Mon Dec 26 19:21:26 2005
From: CAMBO9 at aol.com (CAMBO9@aol.com)
Date: Mon, 26 Dec 2005 13:21:26 EST
Subject: [Python-Dev]  JOB OPENING: Implementor for Python and Search
Message-ID: <154.5e83ca72.30e18ea6@aol.com>

"THINK OUTSIDE THE BOX"

...Rules managing rules in a rules processing system!!!!!
Strong technical skills and capable of developing and driving application 
software strategy    
A track record of strong leadership and motivational skills resulting in high 
retention and morale within project teams. 

This is my consulting email intro........


Senior ERP Process Flow CONSULTANT.
  It's all about my: methodology, techniques and tools.


Below is an example of a successful ERP system installation. It successfully 
demonstrates the use of knowledge engineering and the new paradigm for 
knowledge normalization, to produce an ERP Initiative blueprint, outlining the 
direction and rationale for pursuing an ERP methodology, as a solution to managing 
an enterprise's process flow. It's critical that all management and staff 
involved in an ERP solution know exactly why they are doing it!


"example of a successful ERP system installation"

*** A VIDEO PRESENTATION OF THIS SYSTEM IS AVAILABLE***

In 2001 Industrial Design Corporation, Tempe, AZ, asked International 
Cognitive Computing to design an ERP system that would capture the rules by which 
their engineers conducted business? The ERP initiative for this project is 3 fold.

First, that due to the cyclical nature of the semiconductor industry, IDC has 
been required to hire and layoff engineers, placing them in an almost 
constant state of talent searching (a bottleneck for developing an ERP application).

Second, the uniqueness of the skills required is compounded by the varying 
engineering and scientific disciplines involved in a single customer order 
(another bottleneck for developing an ERP application).

Third, IDC management has a strong ERP strategic direction that includes the 
automation and retrieval of the rules by which their senior engineers (in all 
disciplines and sciences) make decisions about elements of a customer 
requirement. 


SUCCESS:



The IDC Story: The First Successful AI Based Multi-Expert System In Arizona. 

Examine a multi-expert system generator, Rose Navigator, and an Enterprise 
Resource Plan to help manage the need for human engineers against the dynamics 
of customer expectations and orders. 
Pages: 39 through 45, also pages 1 and 5. 

www.pcai.com/web/6a72/522.11.42/TOC.htm
COPY and PASTE to BROWSER   Contents page or AOL users just click on above 
link.
..............................................................................
..............................................................................
........................
This next PCAI magazine article is a behind the scenes look at the technology 
utilized in the Tempe project ................


Natural Language Processing begins with knowledge normalization.
JUNE 2004   Publication: PCAI Magazine.
The Heuristic Life Cycle of a multi-Expert system.

Introduction, the purpose of this article is to introduce a new paradigm in 
the discipline of engineering human knowledge.
Abstract This article introduces a new paradigm to the discipline of 
engineering human knowledge, one that we divide into four tenets of knowledge 
representation:
  1. The four prime domains of knowledge.
  2. All human knowledge has, at its root, a language to communicate the 
knowledge.
  3. A single language sentence contains the smallest unit of knowledge, and 
it is possible to normalize and codify this unit of knowledge into a 
multi-expert computer system (Language representation). 
  4. A knowledge based computer system can learn as well as teach.

This paradigm, as illustrated in this article, is the result of research and 
development and the resulting creation of a multi expert system generator. The 
methodology of the multi-expert system generator is a self-designing system ? 
it constructs and designs attributes that are an integral part of the 
methodology, process and architecture used to generate the multi-expert system.

Magazine Article:
http://www.pcai.com/web/6t6y6t/6t6y6y.7.02/TOC.htm 
copy and paste to your browser.


Nicholas J. Zendelbach
Knowledge is a person's understanding of how the world around them operates.
Chief Technologist
International Cognitive Computing
Century Country Club Estates
5640 East Shea Boulevard
Scottsdale, Arizona
Lab: 480-948-9240
email: cambo9 at aol.com
web:  www.cambo1.com

2005  International Cognitive Computing, Scottsdale, Arizona.
Title: Chief Technologist and ERP Consultant.
An independent R&D company established to, design and develop next generation 
artificial intelligent computer systems. Authored CAMBO a multi-expert system 
generator, pioneering a new paradigm, for 'knowledge normalization'. 

Enterprise Resource Planning. 
" Engineering and business knowledge retention and on-line access.
" Personnel activity mapping.
" Process flow modeling,
" Reengineering and performance evaluation.
" Transaction Pathing.
" Workstation integration.

Consulting agreements with

Flex Solutions, Scottsdale, AZ.  
Andersen Consulting, to LTV Steel, Ohio.
Boeing MDTSC, Phoenix, Arizona
American Express Corp. Phoenix, AZ.
EDS Corporation Monterey, CA.
Presbyterian Hospital, Information Systems, Albuquerque, New Mexico.
Scottsdale Insurance Corp. Scottsdale, AZ.
Walsh America, Camelback Road, Phoenix, Arizona.
International Data Systems Phoenix, AZ. partnership: ORACLE and AION.
Jet Propulsion Labs, aerospace div., Spacecraft Voyager.
County of Santa Clara Assessor's Office, Silicon Valley, San Jose, CA.
General Electric, CALMA Company, Santa Clara, CA.
Stanford University, Palo Alto, CA.
MicroAge Corporation, Phoenix, AZ.
I.L.X. Corporation, Phoenix, AZ.
ABL Incorporated, Mesa, AZ.
JGC Corporation of Japan.
General Electric, CALMA Company, Santa Clara, CA.
Bank Of America, San Francisco, CA.
Wells Fargo Bank, San Francisco, CA.

Application technologies:

EXSYS-CORVID, CLIPS.
ASP/VAR business model, Point-Of-Vote technology.
Business Continuity, Business Impact Analysis and Disaster Management.
ORACLE, AION, MS/DOS,VISUAL BASIC, Replication and Data Warehousing.
SYBASE, POWERBUILDER.
Distributed Data Base and LANMAN.
TERADATA.
Spacecraft Voyager, DEVISER.
TODAY 4GL.
Project Management EXPERT system.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20051226/8207db87/attachment.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2in1speak.doc
Type: application/octet-stream
Size: 695296 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20051226/8207db87/2in1speak-0001.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pdf
Size: 941989 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20051226/8207db87/attachment-0001.pdf

From andrea at suse.de  Tue Dec 27 15:05:16 2005
From: andrea at suse.de (Andrea Arcangeli)
Date: Tue, 27 Dec 2005 15:05:16 +0100
Subject: [Python-Dev] suggestion for smarter garbage collection in function
	of size (gc.set_collect_mem_growth(2))
Message-ID: <20051227140516.GB27200@opteron.random>

Hello,

I run into a problem recently with a reconnectingclientfactory with
twisted while write some spare time software, that turned out to be a gc
inefficiency.

In short the protocol memory wasn't released after the reconnect and the
protocol had about 50M attached to it. So with frequent reconnects the
rss of the task grown to >1G and it pushed the system into heavy swap.
In reality only 50M were allocated, so 950M were wasted by python.

A gc.collect() explicit invocation at every retry of the factory fixed
it. However this means there is significant room for improvement in the
default behaviour of the python gc.

In short the real bug is that the python gc isn't working in function of
size in any way. It doesn't matter the number of objects, it matters
their size!

The whole story can be found in this thread:

	http://twistedmatrix.com/pipermail/twisted-python/2005-December/012233.html

My suggestion to fix this problem in autopilot mode (without requiring
explicit gc.collect()) is to invoke a gc.collect() (or anyway to go deep
down freeing everything possible) at least once every time the amount of
anonymous memory allocated by the interpreter doubles. The tunable should
be a float >= 1. When the tunable is 1 the feature is disabled (so it
works like current python today). Default should be 2 (which means to
invoke gc.collect() after a 100% increase of the anonymous memory
allocated by the interpreter). We could also have yet another threshold
that sets a minimum of ram after which this heuristic in function of
size kicks in, but it's not very important and it may not be worth it
(whem little memory is involved gc.collect() should be fast anyway).

To implement this we need two hooks, in the malloc and free that
allocate python objects. Then we have to store the minimum of this
value (i.e. the last minimum of memory allocated by the interpreter).

The algorithm I'd suggest is like this (supposedly readable pseudocode):

	gc.set_collect_mem_growth(v):
		assert float(v) >= 1
		gc.collect_mem_growth = v

	python_malloc(size):
		ram_size += size
		if ram_size > min_ram_size * gc.collect_mem_growth:
			gc.collect() # python_free runs inside it
			min_ram_size = ram_size # ram size after gc.collect()

	python_free(size):
		ram_size -= size
		min_ram_size = min(min_ram_size, size)

The overhead of this should be zero, and it'll fix my testcase just
fine. I believe the default should be 2 (equivalent to 100% growth of
rss to trigger a full collect) even though it alters the behaviour of
the gc, I think it's a bug that so much memory can be leaked when it
could be reclaimed istantly.

I wouldn't change other parameters, this heuristic in function of size
would be completely orthogonal and disconnected by the current
heuristics in function of the number of elements.
 
It has taken me a day and precious help from the twisted folks to
realize it wasn't a memleak in my twisted spare time application (but
well, it was good since I learnt about the fact I created an heisenbug
by using __del__ to debug the apparent memleak ;).

Thanks.

From fredrik at pythonware.com  Wed Dec 28 13:37:19 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 28 Dec 2005 13:37:19 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com><43B172C9.4070201@v.loewis.de><17329.38860.916447.765575@montanaro.dyndns.org><Pine.LNX.4.58.0512271651040.6831@server1.LFW.org><doslos$kop$1@sea.gmane.org><17330.5976.560419.584425@montanaro.dyndns.org><2m7j9p1n2w.fsf@starship.python.net>
	<dotuk5$v1u$1@sea.gmane.org> <2mzmmlz9o0.fsf@starship.python.net>
Message-ID: <dou0u0$51n$1@sea.gmane.org>

Michael Hudson wrote:

> > that would turn exit and quit into reserved keywords.
>
> In what sense?  Not in the sense of "things in single quotes in
> Grammar"...

no, but in the sense of "names that can no longer be used in code"

    >>> def exit():
    ...      print "bye"

    >>> # what is it?
    >>> exit

    $ oops!

the whole nameerror thing is there to avoid problems if you create
your own exit variables/functions/classes/modules.

</F>




From skip at pobox.com  Wed Dec 28 14:08:12 2005
From: skip at pobox.com (skip@pobox.com)
Date: Wed, 28 Dec 2005 07:08:12 -0600
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <doto08$elm$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
	<17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
	<doslos$kop$1@sea.gmane.org>
	<17330.5976.560419.584425@montanaro.dyndns.org>
	<43B260D3.8070601@v.loewis.de> <doto08$elm$1@sea.gmane.org>
Message-ID: <17330.36412.884898.419281@montanaro.dyndns.org>


    Fredrik>     if isinstance(exc_value, NameError) and not exc_info.tb_next:
    Fredrik>         text = exc_value[0]
    Fredrik>         name = ... extract name from nameerror string ...
    Fredrik>         if sys.commandline.strip() == name:
    Fredrik>             if name in ("exit", "quit"):
    Fredrik>                 raise SystemExit
    Fredrik>             if name == "help":
    Fredrik>                 help()
    Fredrik>                 return
    Fredrik>     ...

    Fredrik> any suggestions on how to improve this ?

I'd make it sys._last_input or something similar to make it clear that a)
this is magic, don't mess with it, and that b) this is the last user input,
not the raw command line.

Skip

From noamraph at gmail.com  Wed Dec 28 14:29:03 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Wed, 28 Dec 2005 15:29:03 +0200
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <aac2c7cb0512280201k5a725223id0885d417f376b42@mail.gmail.com>
References: <fb6fbf560512271703i29f7e24co22e2bd7aaa9c3d27@mail.gmail.com>
	<dotcp2$ju3$1@sea.gmane.org>
	<aac2c7cb0512280201k5a725223id0885d417f376b42@mail.gmail.com>
Message-ID: <b348a0850512280529k65e16b62ye482161e7f9d60b4@mail.gmail.com>

And another example:

>>> a = 1+2j
>>> b = 2+1j
>>> a > b
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: no ordering relation is defined for complex numbers

I came to think that, when forgetting backwards compatibility for a
while, the best thing for comparison operators to do is to raise a
TypeError by default, and work only for types that it makes sense to
compare. I think it is more "explicit is better than implicit", and I
have now two reasons for that:
1. Things like "Decimal(3.0) == 3.0" will make more sense (raise an
exception which explains that Decimals should not be compared to
floats, instead of returning false constantly).
2. It is more forward compatible - when it is discovered that two
types can sensibly be compared, the comparison can be defined, without
changing an existing behaviour which doesn't raise an exception.

Perhaps you can explain to me again why arbitrary objects should be
comparable? I don't see why sorting objects according to values should
work when the order has no real meaning. I don't see why you need all
objects to be comparable for storing them in containers with the
behaviour of dict or set.

If the reason is that you want containers that work among multiple
sessions, and are "identity-based" (that is, only one object can be
used as a key), you can attach to each object an id that isn't session
dependent, and use that instead of the default memory address.

It may be a reason for dropping the default "hash is id": suppose that
you want a persistent storage that will work like dicts but will not
be associated with one Python session (it may be exactly Zope's BTrees
- I don't know). Currently you have a problem with using __hash__ for
that purpose, since the hash value of an object can change between
sessions - that happens when it's the id of the object. Now suppose
that we have a "persistent" id function - it can be implemented by
using weakrefs to associate a unique value with an object on the first
time that the function is called, and storing it with the object when
serializing it. Also suppose that we drop the default hash method, so
where currently hash(x) is id(x), hash(x) will raise a TypeError. Then
our persistent storage can use the persistent id instead of the
default id, and it will work. (You might not want mutable objects to
be used as keys, but that's another problem - the data structure will
be consistent anyway.)

In fewer words, the built-in id() is just one way to assign identities
to objects. __hash__ shouldn't use it implicitly when there's no
value-based hash value - if it wouldn't, the rule that x == y ->
hash(x) == hash(y) will be preserved also between different sessions,
so persistent objects would be able to use hash values.

Does it make sense to you?

Noam

From aahz at pythoncraft.com  Wed Dec 28 14:52:06 2005
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 28 Dec 2005 05:52:06 -0800
Subject: [Python-Dev] suggestion for smarter garbage collection in
	function of size (gc.set_collect_mem_growth(2))
In-Reply-To: <20051227140516.GB27200@opteron.random>
References: <20051227140516.GB27200@opteron.random>
Message-ID: <20051228135206.GA18974@panix.com>

On Tue, Dec 27, 2005, Andrea Arcangeli wrote:
>
> My suggestion to fix this problem in autopilot mode (without requiring
> explicit gc.collect()) is to invoke a gc.collect() (or anyway to go
> deep down freeing everything possible) at least once every time the
> amount of anonymous memory allocated by the interpreter doubles. The
> tunable should be a float >= 1. When the tunable is 1 the feature
> is disabled (so it works like current python today). Default should
> be 2 (which means to invoke gc.collect() after a 100% increase of
> the anonymous memory allocated by the interpreter). We could also
> have yet another threshold that sets a minimum of ram after which
> this heuristic in function of size kicks in, but it's not very
> important and it may not be worth it (whem little memory is involved
> gc.collect() should be fast anyway).

If you feel comfortable with C code, the best way to get this to happen
would be to make the change yourself, then test to find out what effects
this has on Python (in terms of speed and memory usage and whether it
breaks any of the regression tests).  Once you've satisfied yourself
that it works, submit a patch, and post here again with the SF number.

Note that since your tunable parameter is presumably accessible from
Python code, you'll also need to submit doc patches and tests to verify
that it's working correctly.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Don't listen to schmucks on USENET when making legal decisions.  Hire
yourself a competent schmuck."  --USENET schmuck (aka Robert Kern)

From adal.chiriliuc at gmail.com  Wed Dec 28 15:02:08 2005
From: adal.chiriliuc at gmail.com (Adal Chiriliuc)
Date: Wed, 28 Dec 2005 16:02:08 +0200
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <b348a0850512280529k65e16b62ye482161e7f9d60b4@mail.gmail.com>
References: <fb6fbf560512271703i29f7e24co22e2bd7aaa9c3d27@mail.gmail.com>
	<dotcp2$ju3$1@sea.gmane.org>
	<aac2c7cb0512280201k5a725223id0885d417f376b42@mail.gmail.com>
	<b348a0850512280529k65e16b62ye482161e7f9d60b4@mail.gmail.com>
Message-ID: <108891446.20051228160208@myrealbox.com>

On Wednesday, December 28, 2005 Noam Raphael wrote:
> I came to think that, when forgetting backwards compatibility for a
> while, the best thing for comparison operators to do is to raise a
> TypeError by default, and work only for types that it makes sense to
> compare. I think it is more "explicit is better than implicit", and I
> have now two reasons for that:
> 1. Things like "Decimal(3.0) == 3.0" will make more sense (raise an
> exception which explains that Decimals should not be compared to
> floats, instead of returning false constantly).
> 2. It is more forward compatible - when it is discovered that two
> types can sensibly be compared, the comparison can be defined, without
> changing an existing behaviour which doesn't raise an exception.

I agree.

Here's what happens if you try to compare a string and an int in .NET
(it's a compiler error):
error CS0019: Operator '<' cannot be applied to operands of type 'string' and 'int'
error CS0019: Operator '==' cannot be applied to operands of type 'string' and 'int'

Of course, C# is not a dynamic language so things are stricter in
general.

Maybe having == and != work for whatever combination of types is not
such a bad thing. After all, 10 != "text", but then what about Decimal(3.0)
== 3.0?

It doesn't make sense to say that a function object is smaller or
bigger than a string.

And if you sort a list of different objects based on theirs address or
something similar, can you really consider that list sorted? If you
want to normalize a list, you can sort it by using id(item) as a key
and not rely on default comparasions.



From martin at v.loewis.de  Wed Dec 28 15:20:18 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 28 Dec 2005 15:20:18 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <dotqo9$ljq$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org><dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>	<43B172C9.4070201@v.loewis.de>	<17329.38860.916447.765575@montanaro.dyndns.org>	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>	<doslos$kop$1@sea.gmane.org><17330.5976.560419.584425@montanaro.dyndns.org>	<43B260D3.8070601@v.loewis.de><doto08$elm$1@sea.gmane.org>	<43B26B27.4030008@v.loewis.de>
	<dotqo9$ljq$1@sea.gmane.org>
Message-ID: <43B29F22.60905@v.loewis.de>

Fredrik Lundh wrote:
> this is done in site.py, before sitecustomize is loaded.  I'm not sure
> how anyone else would be able to squeeze in an excepthook at this
> point, even if they wanted...

I see. Still, I think the Python code should give a good example.
There *is* an excepthook installed at that point, anyway, so we
should use it.

Otherwise, I think the approach is fine.

Regards,
Martin

From martin at v.loewis.de  Wed Dec 28 15:24:11 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 28 Dec 2005 15:24:11 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <2m7j9p1n2w.fsf@starship.python.net>
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>	<43B172C9.4070201@v.loewis.de>	<17329.38860.916447.765575@montanaro.dyndns.org>	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>	<doslos$kop$1@sea.gmane.org>	<17330.5976.560419.584425@montanaro.dyndns.org>
	<2m7j9p1n2w.fsf@starship.python.net>
Message-ID: <43B2A00B.1080001@v.loewis.de>

Michael Hudson wrote:
> The thing that bothers me about it is that the standard way you tell
> python to do something is "call a function" -- to me, a special case
> for exiting the interpreter seems out of proportion.

That would assume that the user knows that exit is a function:
apparently, people expect it to be a statement (like print), or
they are entirely unaware of the distinctions between statements,
expressions, and functions. I often find that my students call
them "commands" collectively (if they want to be more specific,
they call the functions "methods").

Regards,
Martin

From martin at v.loewis.de  Wed Dec 28 15:32:29 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 28 Dec 2005 15:32:29 +0100
Subject: [Python-Dev] suggestion for smarter garbage collection in
 function of size (gc.set_collect_mem_growth(2))
In-Reply-To: <20051227140516.GB27200@opteron.random>
References: <20051227140516.GB27200@opteron.random>
Message-ID: <43B2A1FD.5010300@v.loewis.de>

Andrea Arcangeli wrote:
> To implement this we need two hooks, in the malloc and free that
> allocate python objects. Then we have to store the minimum of this
> value (i.e. the last minimum of memory allocated by the interpreter).

I would like to underline Aahz' comment: it is unlikely that anything
will happen about this unless you make it happen. This specific problem
is not frequent, and the current strategy (collect if 1000 new objects
are allocated) works fine for most people. So if you want a change,
you should really consider comming up with a patch yourself. Bonus
points if the code integrates with the current strategies, instead
of replacing them.

Regards,
Martin

From rhamph at gmail.com  Wed Dec 28 15:59:27 2005
From: rhamph at gmail.com (Adam Olsen)
Date: Wed, 28 Dec 2005 07:59:27 -0700
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <b348a0850512280529k65e16b62ye482161e7f9d60b4@mail.gmail.com>
References: <fb6fbf560512271703i29f7e24co22e2bd7aaa9c3d27@mail.gmail.com>
	<dotcp2$ju3$1@sea.gmane.org>
	<aac2c7cb0512280201k5a725223id0885d417f376b42@mail.gmail.com>
	<b348a0850512280529k65e16b62ye482161e7f9d60b4@mail.gmail.com>
Message-ID: <aac2c7cb0512280659j4ca39f28yfa3a688fb89c71ff@mail.gmail.com>

On 12/28/05, Noam Raphael <noamraph at gmail.com> wrote:
> And another example:
>
> >>> a = 1+2j
> >>> b = 2+1j
> >>> a > b
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: no ordering relation is defined for complex numbers
>
> I came to think that, when forgetting backwards compatibility for a
> while, the best thing for comparison operators to do is to raise a
> TypeError by default, and work only for types that it makes sense to
> compare. I think it is more "explicit is better than implicit", and I
> have now two reasons for that:
> 1. Things like "Decimal(3.0) == 3.0" will make more sense (raise an
> exception which explains that Decimals should not be compared to
> floats, instead of returning false constantly).
> 2. It is more forward compatible - when it is discovered that two
> types can sensibly be compared, the comparison can be defined, without
> changing an existing behaviour which doesn't raise an exception.

I agree for greaterthan and lessthan operators but I'm not convinced
for equality.  Consider this in contrast to your example:
>>> a = 1+2j
>>> b = 1+2j
>>> a is b
False
>>> a == b
True

Complex numbers can't be sorted but they can be tested for equality. 
Decimal numbers can't currently be tested for equality against a float
but there's no loss-of-accuracy argument to prevent it (just a
difficulty of implementation one.)

If the comparison is to fail I would prefer an exception rather than
an id-based fallback though.

Speaking of id, there's no reason why "id(a) == id(b)" has to fail for
mismatched types in the face of persistence so long as the result of
id() has the same lifetime as the target object.  This could be done
using weakrefs or by making an id type with a strong reference to the
target object.

--
Adam Olsen, aka Rhamphoryncus

From noamraph at gmail.com  Wed Dec 28 16:13:44 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Wed, 28 Dec 2005 17:13:44 +0200
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <aac2c7cb0512280659j4ca39f28yfa3a688fb89c71ff@mail.gmail.com>
References: <fb6fbf560512271703i29f7e24co22e2bd7aaa9c3d27@mail.gmail.com>
	<dotcp2$ju3$1@sea.gmane.org>
	<aac2c7cb0512280201k5a725223id0885d417f376b42@mail.gmail.com>
	<b348a0850512280529k65e16b62ye482161e7f9d60b4@mail.gmail.com>
	<aac2c7cb0512280659j4ca39f28yfa3a688fb89c71ff@mail.gmail.com>
Message-ID: <b348a0850512280713p4a71b76fo1cb5aa87d6c5a984@mail.gmail.com>

On 12/28/05, Adam Olsen <rhamph at gmail.com> wrote:
> I agree for greaterthan and lessthan operators but I'm not convinced
> for equality.  Consider this in contrast to your example:
> >>> a = 1+2j
> >>> b = 1+2j
> >>> a is b
> False
> >>> a == b
> True
>
> Complex numbers can't be sorted but they can be tested for equality.
> Decimal numbers can't currently be tested for equality against a float
> but there's no loss-of-accuracy argument to prevent it (just a
> difficulty of implementation one.)
>
> If the comparison is to fail I would prefer an exception rather than
> an id-based fallback though.

I think we agree. I don't think that every type that supports equality
comparison should support order comparison. I think that if there's no
meaningful comparison (whether equality or order), an exception should
be raised.
>
> Speaking of id, there's no reason why "id(a) == id(b)" has to fail for
> mismatched types in the face of persistence so long as the result of
> id() has the same lifetime as the target object.  This could be done
> using weakrefs or by making an id type with a strong reference to the
> target object.

I don't mean to change the current behaviour of id() - I just meant
that an additional one may be implemented, possible by a specific
library (Zope, for instance), so the built-in one shouldn't be used as
a fallback default.

Noam

From goodger at python.org  Wed Dec 28 16:07:34 2005
From: goodger at python.org (David Goodger)
Date: Wed, 28 Dec 2005 10:07:34 -0500
Subject: [Python-Dev] PyCon TX 2006: Early-bird registration ends Dec. 31!
Message-ID: <43B2AA36.7060808@python.org>

Early bird registration for PyCon TX 2006 ends on December 31st,
so there are only a few days LEFT. To register, please visit:

	http://us.pycon.org/TX2006/Registration

You can still register after Dec. 31st, but the cost will go up by
US$65 (US$25 for students).

This year PyCon will feature a day of tutorials before the three days
of regular presentations.  Course outlines for all the tutorials have
been posted; see

	http://wiki.python.org/moin/PyCon2006/Tutorials

All of the PyCon tutorials are still open for new registrations, but
space is limited, and we suspect they'll all be filled up by the time
early-bird registration closes.

Don't forget to book your hotel room, too.  PyCon TX 2006 is being
held at a Dallas/Addison hotel, and we have negotiated a special low
rate:

	http://us.pycon.org/Addison/Hotels

We hope to see you in Texas!

-- David Goodger
   (on behalf of A.M. Kuchling, Chair, PyCon 2006)



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 249 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20051228/b5c22f9f/signature.pgp

From rhamph at gmail.com  Wed Dec 28 16:27:10 2005
From: rhamph at gmail.com (Adam Olsen)
Date: Wed, 28 Dec 2005 08:27:10 -0700
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <b348a0850512280713p4a71b76fo1cb5aa87d6c5a984@mail.gmail.com>
References: <fb6fbf560512271703i29f7e24co22e2bd7aaa9c3d27@mail.gmail.com>
	<dotcp2$ju3$1@sea.gmane.org>
	<aac2c7cb0512280201k5a725223id0885d417f376b42@mail.gmail.com>
	<b348a0850512280529k65e16b62ye482161e7f9d60b4@mail.gmail.com>
	<aac2c7cb0512280659j4ca39f28yfa3a688fb89c71ff@mail.gmail.com>
	<b348a0850512280713p4a71b76fo1cb5aa87d6c5a984@mail.gmail.com>
Message-ID: <aac2c7cb0512280727q1f987ea5oc38660bf50fb227f@mail.gmail.com>

On 12/28/05, Noam Raphael <noamraph at gmail.com> wrote:
> On 12/28/05, Adam Olsen <rhamph at gmail.com> wrote:
> > Speaking of id, there's no reason why "id(a) == id(b)" has to fail for
> > mismatched types in the face of persistence so long as the result of
> > id() has the same lifetime as the target object.  This could be done
> > using weakrefs or by making an id type with a strong reference to the
> > target object.
>
> I don't mean to change the current behaviour of id() - I just meant
> that an additional one may be implemented, possible by a specific
> library (Zope, for instance), so the built-in one shouldn't be used as
> a fallback default.

Why aim low when you can aim for the face instead? :)

--
Adam Olsen, aka Rhamphoryncus

From aleaxit at gmail.com  Wed Dec 28 16:55:13 2005
From: aleaxit at gmail.com (Alex Martelli)
Date: Wed, 28 Dec 2005 07:55:13 -0800
Subject: [Python-Dev] Small any/all enhancement
In-Reply-To: <efdf14b4ecbb0e972dac7899ff88a761@xs4all.nl>
References: <20051227204501.GA252@divmod.com>
	<8AA572FD-50E2-4781-91A5-9C6D73B7E8EA@gmail.com>
	<efdf14b4ecbb0e972dac7899ff88a761@xs4all.nl>
Message-ID: <D4CF0102-8D89-4B16-84D7-0703D627281C@gmail.com>


On Dec 27, 2005, at 11:06 PM, Eric Nieuwland wrote:
    ...
>>> def zerop(x):
>>>     return x==0
>>>
>>> all(some_objects, zerop)
>>
>> and why would that be better than
>> all(o==0 for o in some_objects)
>> ?
>
> all() can be terminated at the first false element. For very long
> sequences this has important performance benefits. Besides, it makes

Of course it can -- in both formulations.  genexp's are also computed  
"as needed", only one item at a time: you appear to imply they don't,  
maybe you're confusing them with list comprehensions.  What I'm  
asking is, what are the ADVANTAGES of the pred form, that make it  
worth paying the conceptual cost of having "two obvious ways" to do  
one task.

> all(seq,pred) the equivalent of pred(seq[0]) and  pred(seq[1]) and
> pred(seq[2]) and ...

...and also the equivalent of all(pred(s) for s in seq) -- which is  
exactly my problem: I don't like redundant good ways of expressing  
identical tasks.  The genexp will often be more compact, whenever the  
'pred' requires a def, a lambda, or something like  
operator.attrgetter, anyway.

Alex



From chris at kateandchris.net  Wed Dec 28 16:41:36 2005
From: chris at kateandchris.net (Chris Lambacher)
Date: Wed, 28 Dec 2005 10:41:36 -0500
Subject: [Python-Dev] Automated Python testing (was Re: status of
	development documentation)
In-Reply-To: <1f7befae0512250954x4dee672w54efca5ebe0bd80d@mail.gmail.com>
References: <1f7befae0512250954x4dee672w54efca5ebe0bd80d@mail.gmail.com>
Message-ID: <20051228154136.GA23168@kateandchris.net>

On Sun, Dec 25, 2005 at 12:54:32PM -0500, Tim Peters wrote:
> This really helps at Zope Corp.  One downside is that we seem unable
> to get an in-house Windows buildbot slave to work reliably, and so far
> don't even know whether that's because of Windows, the buildbot code,
> or flakiness in our internal network.  It seems quite reliable on
> Linux, though.

At work we build both py2exe'd apps and Visual C++ apps on windows with
buildbot with no issues.  The only problem we has was when we were first
setting it up, it was not clear that you need to explicitly tell twistd to use
the win32 reactor.

-Chris

From aleaxit at gmail.com  Wed Dec 28 16:59:41 2005
From: aleaxit at gmail.com (Alex Martelli)
Date: Wed, 28 Dec 2005 07:59:41 -0800
Subject: [Python-Dev] deque alternative
In-Reply-To: <43B26FAD.9010700@v.loewis.de>
References: <43AE96FF.2010303@stackless.com>	<8393fff0512141107x691bb085yd846cbf9b96cde33@mail.gmail.com>	<ca471dc20512141237i4f09098cudb951fbe70623085@mail.gmail.com>	<43AE96FF.2010303@stackless.com>
	<5.1.1.6.0.20051225215827.02233c30@mail.telecommunity.com>
	<43B26FAD.9010700@v.loewis.de>
Message-ID: <64AA6E67-F798-4C17-8133-2C97FC3D3120@gmail.com>


On Dec 28, 2005, at 2:57 AM, Martin v. L?wis wrote:

> Phillip J. Eby wrote:
>> Since I routinely use 2-item tuples (twoples?)
>
> I've been using "pairs" to describe that datatype. Not sure
> how common it is in English, but in German, "Zweitupel"
> is often called "Paar".

I use 'pair', too, admittedly by analogy with C++'s std::pair type.   
(The Italian closest word "paio" would imply that the two items are  
in some sense "homogeneous"; to avoid that implication one would use  
"coppia", which differently from English "a couple" ALWAYS, not just  
sometimes, implies exactly two items; so I can't base myself on  
analogies with my own mothertongue;-).


Alex


From aleaxit at gmail.com  Wed Dec 28 17:06:53 2005
From: aleaxit at gmail.com (Alex Martelli)
Date: Wed, 28 Dec 2005 08:06:53 -0800
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <2m7j9p1n2w.fsf@starship.python.net>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
	<17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
	<doslos$kop$1@sea.gmane.org>
	<17330.5976.560419.584425@montanaro.dyndns.org>
	<2m7j9p1n2w.fsf@starship.python.net>
Message-ID: <B7EE1E00-6FC7-4404-AF12-90B8444BCD6F@gmail.com>


On Dec 28, 2005, at 3:24 AM, Michael Hudson wrote:

> skip at pobox.com writes:
>
>>     Fredrik> a quit/exit command that actually quits, instead of  
>> printing a
>>     Fredrik> "you didn't say please!" message.
>>
>> I like Fredrik's idea more and more.
>
> The thing that bothers me about it is that the standard way you tell
> python to do something is "call a function" -- to me, a special case
> for exiting the interpreter seems out of proportion.

Just brainstorming, but -- maybe this means we should generalize the  
idea?  I.e., allow other cases in which "just mentioning X" means  
"call function Y [with the following arguments]", at least at the  
interactive prompt if not more generally.  If /F's idea gets  
implemented by binding to names 'exit' and 'quit' the result of some  
factory-call with "function to be called" set to sys.exit and  
"arguments for it" set to () [[as opposed to specialcasing checks of  
last commandline for equality to 'exit' &c]] then the implementation  
of the generalization would be no harder.  I do find myself in  
sessions in which I want to perform some action repeatedly, and  
currently the least typing is 4 characters (x()<enter>) while this  
would reduce it to two (iPython does allow such handy shortcuts, but  
I'm often using plain interactive interpreters).

If this generalization means a complicated implementation, by all  
means let's scrap it, but if implementation is roughly as easy, it  
may be worth considering to avoid making a too-special "special  
case" (or maybe not, but brainstorming means never having to say  
you're sorry;-).


Alex



From fdrake at acm.org  Wed Dec 28 17:10:15 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 28 Dec 2005 11:10:15 -0500
Subject: [Python-Dev] deque alternative
In-Reply-To: <64AA6E67-F798-4C17-8133-2C97FC3D3120@gmail.com>
References: <43AE96FF.2010303@stackless.com> <43B26FAD.9010700@v.loewis.de>
	<64AA6E67-F798-4C17-8133-2C97FC3D3120@gmail.com>
Message-ID: <200512281110.16074.fdrake@acm.org>

On Wednesday 28 December 2005 10:59, Alex Martelli wrote:
 > (The Italian closest word "paio" would imply that the two items are
 > in some sense "homogeneous"; to avoid that implication one would use
 > "coppia", which differently from English "a couple" ALWAYS, not just
 > sometimes, implies exactly two items; so I can't base myself on
 > analogies with my own mothertongue;-).

Language is a pain, isn't it?

Growing up, "pair" always meant exactly two.  I teach my kids that.  We'll see 
if it sticks.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From martin at v.loewis.de  Wed Dec 28 17:43:04 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 28 Dec 2005 17:43:04 +0100
Subject: [Python-Dev] Automated Python testing (was Re: status
 of	development documentation)
In-Reply-To: <1f7befae0512250954x4dee672w54efca5ebe0bd80d@mail.gmail.com>
References: <1f7befae0512250954x4dee672w54efca5ebe0bd80d@mail.gmail.com>
Message-ID: <43B2C098.5070104@v.loewis.de>

Tim Peters wrote:
> Someone sets up a "buildbot master" 

That's what I now did:

http://www.python.org/dev/buildbot/

I'm not quite sure on a number of concepts: should there
be multiple "slaves" per "builder"? Should I have multiple
factories? How should I pass build-machine specific information
(like: what compiler to use) in the master, or is that a slave
thing? How should I deal with branches?

Anyhow, I created two "builders", both with the same "slave",
on sparc-sun-solaris2.10, 32-bit mode, gcc. The "python-full"
builder does "svn export", whereas the "python-quick" builder
does "svn update". Each of these is associated with an
AnyBranchScheduler, for both 'trunk' and 'tags/release24-maint'.
Not sure whether this means that the 2.4 release branch
will ever be built (as the SVN steps have a defaultBranch of
"trunk").

If anybody wants to contribute additional builders, or has
ideas for organizing this all, please let me know.

Regards,
Martin

From walter at livinglogic.de  Wed Dec 28 18:04:55 2005
From: walter at livinglogic.de (=?iso-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed, 28 Dec 2005 18:04:55 +0100 (CET)
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <B7EE1E00-6FC7-4404-AF12-90B8444BCD6F@gmail.com>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
	<17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
	<doslos$kop$1@sea.gmane.org>
	<17330.5976.560419.584425@montanaro.dyndns.org>
	<2m7j9p1n2w.fsf@starship.python.net>
	<B7EE1E00-6FC7-4404-AF12-90B8444BCD6F@gmail.com>
Message-ID: <61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>

Alex Martelli wrote:

> On Dec 28, 2005, at 3:24 AM, Michael Hudson wrote:
>
>> skip at pobox.com writes:
>>
>>>     Fredrik> a quit/exit command that actually quits, instead of
>>> printing a
>>>     Fredrik> "you didn't say please!" message.
>>>
>>> I like Fredrik's idea more and more.
>>
>> The thing that bothers me about it is that the standard way you tell python to do something is "call a function" -- to me, a
>> special case for exiting the interpreter seems out of proportion.
>
> Just brainstorming, but -- maybe this means we should generalize the   idea?  I.e., allow other cases in which "just
> mentioning X" means   "call function Y [with the following arguments]", at least at the   interactive prompt if not more
> generally.  If /F's idea gets
> implemented by binding to names 'exit' and 'quit' the result of some   factory-call with "function to be called" set to
> sys.exit and
> "arguments for it" set to () [[as opposed to specialcasing checks of   last commandline for equality to 'exit' &c]]

We have sys.displayhook and sys.excepthook. Why not add a sys.inputhook? sys.inputhook gets passed each line entered and may
return True if it has processed the line inself and False if normal handling of the input should be done.
This allows special treatment of "quit", "exit", "help" and it might make implementing alternative shells for Python easier
(without having to subclass code.InteractiveConsole).
> then the
> implementation   of the generalization would be no harder.  I do find myself in
> sessions in which I want to perform some action repeatedly, and
> currently the least typing is 4 characters (x()<enter>) while this   would reduce it to two

What's wrong with <cursor up>, <return>?

> (iPython does allow such handy
> shortcuts, but   I'm often using plain interactive interpreters).
>
> If this generalization means a complicated implementation, by all   means let's scrap it, but if implementation is roughly as
> easy, it   may be worth considering to avoid making a too-special "special
> case" (or maybe not, but brainstorming means never having to say   you're sorry;-).

Bye,
   Walter D?rwald




From ncoghlan at gmail.com  Wed Dec 28 18:28:22 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 29 Dec 2005 03:28:22 +1000
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>	<43B172C9.4070201@v.loewis.de>	<17329.38860.916447.765575@montanaro.dyndns.org>	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>	<doslos$kop$1@sea.gmane.org>	<17330.5976.560419.584425@montanaro.dyndns.org>	<2m7j9p1n2w.fsf@starship.python.net>	<B7EE1E00-6FC7-4404-AF12-90B8444BCD6F@gmail.com>
	<61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>
Message-ID: <43B2CB36.1080701@gmail.com>

[Alex]
>> Just brainstorming, but -- maybe this means we should generalize the   idea?  I.e., allow other cases in which "just
>> mentioning X" means   "call function Y [with the following arguments]", at least at the   interactive prompt if not more
>> generally.  If /F's idea gets
>> implemented by binding to names 'exit' and 'quit' the result of some   factory-call with "function to be called" set to
>> sys.exit and
>> "arguments for it" set to () [[as opposed to specialcasing checks of   last commandline for equality to 'exit' &c]]
> 

[Walter]
> We have sys.displayhook and sys.excepthook. Why not add a sys.inputhook? sys.inputhook gets passed each line entered and may
> return True if it has processed the line inself and False if normal handling of the input should be done.
> This allows special treatment of "quit", "exit", "help" and it might make implementing alternative shells for Python easier
> (without having to subclass code.InteractiveConsole).

[Alex]
>> then the
>> implementation   of the generalization would be no harder.  I do find myself in
>> sessions in which I want to perform some action repeatedly, and
>> currently the least typing is 4 characters (x()<enter>) while this   would reduce it to two

Hmm. . .

   def default_inputhook(statement):
       try:
           aliased = sys.aliases[statement]
       except KeyError:
           return False
       else:
           aliased()
           return True

   sys.aliases = dict(exit=sys.exit, quit=sys.exit)
   sys.inputhook = default_inputhook

I think Walter's idea may have merit (although I believe the input hook should 
be passed whole statements, rather than individual lines).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From skip at pobox.com  Wed Dec 28 18:38:05 2005
From: skip at pobox.com (skip@pobox.com)
Date: Wed, 28 Dec 2005 11:38:05 -0600
Subject: [Python-Dev] Automated Python testing (was Re: status of
 development documentation)
In-Reply-To: <43B2C098.5070104@v.loewis.de>
References: <1f7befae0512250954x4dee672w54efca5ebe0bd80d@mail.gmail.com>
	<43B2C098.5070104@v.loewis.de>
Message-ID: <17330.52605.50178.566883@montanaro.dyndns.org>


    Martin> If anybody wants to contribute additional builders, or has ideas
    Martin> for organizing this all, please let me know.

Martin,

I installed buildbot on a dual-processor Powermac G5 running OSX 10.3.9
(montanaro-g5.dyndns.org).  Let me know the parameters I need to give to the
buildbot command.  It might take a couple cycles to get things set up since
I don't normally develop on that machine.  I'll also have to punch a hole in
my Airport to allow access.  OTOH, it's more stationary and a lot faster
than my laptop.

Skip

From exarkun at divmod.com  Wed Dec 28 18:36:33 2005
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Wed, 28 Dec 2005 12:36:33 -0500
Subject: [Python-Dev] Automated Python testing (was Re: status
 of	development documentation)
In-Reply-To: <43B2C098.5070104@v.loewis.de>
Message-ID: <20051228173633.1217.205642351.divmod.quotient.8563@ohm>

On Wed, 28 Dec 2005 17:43:04 +0100, "\"Martin v. L?wis\"" <martin at v.loewis.de> wrote:
>Tim Peters wrote:
>> Someone sets up a "buildbot master"
>
>That's what I now did:
>
>http://www.python.org/dev/buildbot/
>
>I'm not quite sure on a number of concepts: should there
>be multiple "slaves" per "builder"? Should I have multiple
>factories? How should I pass build-machine specific information
>(like: what compiler to use) in the master, or is that a slave
>thing? How should I deal with branches?

A slave is an entity capable of performing tasks.  It can be 
asked to perform any task you like, though it may not be able 
to perform them all if it lacks some requirements.

A builder is a particular job.  It can be performed by any 
slave (so long as its requirements are met), or by multiple 
slaves.

A factory defines the work to be done by a builder.  If which 
compiler is being used is an important part of the purpose of 
a builder (for example, there might be a builder the purpose of 
which is to test a Python built with GCC 3.4 and another the 
purpose of which is to test a Python built with GCC 4.0), then 
it should be specified in the master configuration.  If it is 
not important, then it should be left as general as possible, 
to allow for the most slaves to be able to complete the task.

I have most often seen branches supported by allowing commits 
to trunk to automatically trigger a build on trunk, while 
commits to branches do not automatically trigger any builds.
Builds on branches can then be explicitly requested when a 
developer wants to know the state of their branch on various 
platforms/configurations (often before merging to trunk to 
know if they are introducing any regressions, but valuable 
at other times as well).

To support branches in this way, you want to use a PBChangeSource 
with a prefix of "trunk" and when creating build steps, specify 
a a baseURL of "svn//svn.python.org/projects/python/", and 
defaultBranch of "trunk".

>
>Anyhow, I created two "builders", both with the same "slave",
>on sparc-sun-solaris2.10, 32-bit mode, gcc. The "python-full"
>builder does "svn export", whereas the "python-quick" builder
>does "svn update". Each of these is associated with an
>AnyBranchScheduler, for both 'trunk' and 'tags/release24-maint'.
>Not sure whether this means that the 2.4 release branch
>will ever be built (as the SVN steps have a defaultBranch of
>"trunk").

I think this means tags/release24-maint won't ever be built 
automatically (I haven't used AnyBranchScheduler, and I don't 
know much about schedulers in buildbot in general).  You 
should be able to manually request a build, but for some 
reason I don't see the form for doing so on the master website 
(<http://twistedmatrix.com/buildbot/full-2.3> for an example 
of what this looks like).  I'm not sure if this is a buildbot
version problem, or if there is just another piece of 
configuration that needs to be set.

Jean-Paul

From raymond.hettinger at verizon.net  Wed Dec 28 19:08:14 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Wed, 28 Dec 2005 13:08:14 -0500
Subject: [Python-Dev] A few questions about setobject
In-Reply-To: <43B267ED.6020804@v.loewis.de>
Message-ID: <006a01c60bd9$acad86e0$78fdcc97@oemcomputer>

> The setentry typedef clearly violates the principles of the API, so
> it should be renamed.

In my next update, will rename it to match the Py or _Py convention.


Raymond


From aleaxit at gmail.com  Wed Dec 28 19:10:19 2005
From: aleaxit at gmail.com (Alex Martelli)
Date: Wed, 28 Dec 2005 10:10:19 -0800
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>
References: <dor982$qrc$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
	<17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
	<doslos$kop$1@sea.gmane.org>
	<17330.5976.560419.584425@montanaro.dyndns.org>
	<2m7j9p1n2w.fsf@starship.python.net>
	<B7EE1E00-6FC7-4404-AF12-90B8444BCD6F@gmail.com>
	<61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>
Message-ID: <e8a0972d0512281010t4853f3a8r936f277c28d83ab9@mail.gmail.com>

On 12/28/05, Walter D?rwald <walter at livinglogic.de> wrote:
   ...
> We have sys.displayhook and sys.excepthook. Why not add a sys.inputhook?

Sure, particularly with Nick's suggestion for a default input hook it
would be fine.

> > sessions in which I want to perform some action repeatedly, and
> > currently the least typing is 4 characters (x()<enter>) while this   would reduce it to two
>
> What's wrong with <cursor up>, <return>?

The fact that there is no upper bound to the number of cursor-up
keystrokes needed here -- by "perform some action repeatedly" I don't
mean "half a dozen times right after each other with nothing
in-between" (sorry for the ambiguous phrasing), but "numerous times,
repeatedly in the course of an interactive interpreter session".


Alex

From raymond.hettinger at verizon.net  Wed Dec 28 19:15:51 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Wed, 28 Dec 2005 13:15:51 -0500
Subject: [Python-Dev] A few questions about setobject
In-Reply-To: <b348a0850512280358x27c93c3n8be2c99d9416fc95@mail.gmail.com>
Message-ID: <006b01c60bda$bcc45f80$78fdcc97@oemcomputer>

> > > I think it should be ok because it's never used
> > > really as a PyObject. Am I missing something? (Ok, I now thought
that
> > > maybe it's because some parts don't treat dummy elements
specially.
> > > But it seems to me that most parts do treat them specially, so
perhaps
> > > it would be better to make a few changes so that all parts will
treat
> > > them specially?)
> >
> > In principle, you are right. One place that doesn't special-case the
> > dummy is set_clear_internal (in fact, the Py_DEBUG assertion is
> > completely useless there, AFAICT).
> >
> > The tricky question is: can we be certain that we find all places,
> > in all code paths, where we have to special-case the dummy? Having
> > PyObject* which don't point to PyObject is error-prone.
> >
> > Also, what would we gain? If you think it is speed: I doubt it. In
> > one place, a comment suggests that actually seeing the dummy element
> > is so much more unlikely than the other cases that, for performance,
> > the test for the dummy is done last. We would lose additional speed
> > in the cases where the dummy isn't yet considered.
> >
> Ok, I tried. It took me 25 minutes to change the code, while going
> over every occurence of "key" and "decref" in setobject.c. (It seems
> to me that the dummy element can only be accessed from entry->key.)
> Most of the code isn't bothered by the dummy element, since it uses
> the data structure in a more abstract way. I think that it simplifies
> the code, while adding a condition in only two places, which I don't
> think should make anything noticably slower. The result passes the
> complete test suite. In one sentence: I think that it makes the code
> slightly better, and the risk is pretty low.
> 
> I thought to post it as a patch, but sourceforge didn't work for me,
> and it's not that long, so I paste it at the end of this message. Feel
> free to do whatever you want with it.

Feel free to send me your patch (as an attachment, not the body of an
email) and I'll take another look at it.

We discussed this a few months ago and rejected it.  I'll look back to
find the reason why (perhaps to keep parallel with dictobject.c or
because it was thought to be an error-prone micro-optimization).



Raymond


From eric.nieuwland at xs4all.nl  Wed Dec 28 19:17:50 2005
From: eric.nieuwland at xs4all.nl (Eric Nieuwland)
Date: Wed, 28 Dec 2005 19:17:50 +0100
Subject: [Python-Dev] Small any/all enhancement
In-Reply-To: <43B26249.6070406@v.loewis.de>
References: <20051227204501.GA252@divmod.com>	<8AA572FD-50E2-4781-91A5-9C6D73B7E8EA@gmail.com>
	<efdf14b4ecbb0e972dac7899ff88a761@xs4all.nl>
	<43B26249.6070406@v.loewis.de>
Message-ID: <4a87b3155e4794a5791d01b6a0c86061@xs4all.nl>

I wrote:
> all() can be terminated at the first false element. For very long
> sequences this has important performance benefits. Besides, it makes
> all(seq,pred) the equivalent of pred(seq[0]) and  pred(seq[1]) and
> pred(seq[2]) and ...

then, Martin v. L?wis wrote:
>
> And so does the version with generator expressions: Alex' expression
> will also terminate with the first false statement; it is equivalent
> to some_objects[0]==0 and some_objects[1]==0 and ...

and Alex Martelli wrote:
> Of course it can -- in both formulations.  genexp's are also computed 
> "as needed", only one item at a time: you appear to imply they don't, 
> maybe you're confusing them with list comprehensions.  What I'm asking 
> is, what are the ADVANTAGES of the pred form, that make it worth 
> paying the conceptual cost of having "two obvious ways" to do one 
> task.
>
>> all(seq,pred) the equivalent of pred(seq[0]) and  pred(seq[1]) and
>> pred(seq[2]) and ...
>
> ...and also the equivalent of all(pred(s) for s in seq) -- which is 
> exactly my problem: I don't like redundant good ways of expressing 
> identical tasks.  The genexp will often be more compact, whenever the 
> 'pred' requires a def, a lambda, or something like 
> operator.attrgetter, anyway.

Oops! Right you are. I was a bit too quick after seeing the use of 
map() proposed.

--eric


From fredrik at pythonware.com  Wed Dec 28 19:19:44 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 28 Dec 2005 19:19:44 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com><43B172C9.4070201@v.loewis.de><17329.38860.916447.765575@montanaro.dyndns.org><Pine.LNX.4.58.0512271651040.6831@server1.LFW.org><doslos$kop$1@sea.gmane.org><17330.5976.560419.584425@montanaro.dyndns.org><2m7j9p1n2w.fsf@starship.python.net><B7EE1E00-6FC7-4404-AF12-90B8444BCD6F@gmail.com>
	<61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>
Message-ID: <doul02$8ja$1@sea.gmane.org>

Walter Dörwald wrote:

> We have sys.displayhook and sys.excepthook. Why not add a sys.inputhook?
> sys.inputhook gets passed each line entered and may return True if it has
> processed the line inself and False if normal handling of the input should be
> done. This allows special treatment of "quit", "exit", "help" /.../

so how would such a hook deal with the

    >>> def exit():
    ...     pass
    >>> exit

case ?

</F>




From fperez.net at gmail.com  Wed Dec 28 19:36:35 2005
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 28 Dec 2005 11:36:35 -0700
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
	<17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
	<doslos$kop$1@sea.gmane.org>
	<17330.5976.560419.584425@montanaro.dyndns.org>
	<2m7j9p1n2w.fsf@starship.python.net>
	<B7EE1E00-6FC7-4404-AF12-90B8444BCD6F@gmail.com>
Message-ID: <doulvl$bnf$1@sea.gmane.org>

Alex Martelli wrote:

> 
> On Dec 28, 2005, at 3:24 AM, Michael Hudson wrote:

>> The thing that bothers me about it is that the standard way you tell
>> python to do something is "call a function" -- to me, a special case
>> for exiting the interpreter seems out of proportion.
> 
> Just brainstorming, but -- maybe this means we should generalize the
> idea?  I.e., allow other cases in which "just mentioning X" means
> "call function Y [with the following arguments]", at least at the
> interactive prompt if not more generally.  If /F's idea gets
> implemented by binding to names 'exit' and 'quit' the result of some
> factory-call with "function to be called" set to sys.exit and
> "arguments for it" set to () [[as opposed to specialcasing checks of
> last commandline for equality to 'exit' &c]] then the implementation
> of the generalization would be no harder.  I do find myself in
> sessions in which I want to perform some action repeatedly, and
> currently the least typing is 4 characters (x()<enter>) while this
> would reduce it to two (iPython does allow such handy shortcuts, but
> I'm often using plain interactive interpreters).
> 
> If this generalization means a complicated implementation, by all
> means let's scrap it, but if implementation is roughly as easy, it
> may be worth considering to avoid making a too-special "special
> case" (or maybe not, but brainstorming means never having to say
> you're sorry;-).

Allow me to add a few comments, here: as the ipython author, I happen to have
thought an awful lot about all these issues.  

First, your suggestion of incorporating 'autocalling' ( the automatic 'foo a' ->
'foo(a)' transformation) into the core python interpreter may not be a very
good idea.  The code that does this is precisely the most brittle, delicate
part of ipython, a little regexp/eval/introspection dance that tries really,
really hard to understand whether 'foo' is a string that will point to a
callable once evaluated, but without eating generators, causing side effects,
or anything else.  I know it sounds silly, and perhaps it's just my
limitations, but it has taken several years to flesh out all the corner cases
where this code could fail (and in the past, a few really surprising failure
cases have been found).

In ipython, this functionality can still be turned off (via %autocall) at any
time, in case it is not working correctly.  You are welcome to look at the
code, it's the _prefilter method here:

http://projects.scipy.org/ipython/ipython/file/ipython/trunk/IPython/iplib.py

So while I think that this is extremely useful in a third-party library like
ipython, it's probably a little too delicate for the core, official interpreter
where reliability is so absolutely critical.  In fact, my own standard is that
I trust the CPython prompt as a test of 'doing the right thing', so I like that
it remains simple and solid.

Now, on to the wider discussion started by the quit/exit debate: the problem is
that we are here trying to make some particular words 'interactive commands' in
a language that doesn't have such a notion to begin with.  The tension of how
best to implement it, the various low-level tricks proposed, etc, stems (I
think) from this underlying fact.  

In IPyhton, I've convinced myself that this is a problem whose proper solution
requires an orthogonal command subsystem, the 'magics' (credit where credit is
due: the magic system was already in IPP, Janko Hauser's library which was one
of ipython's three original components; Janko understood the issue early on and
got it right).  This creates a separate namespace, controlled by a
disambiguation character (the % prefix in ipython), and therefore allows you to
cleanly offer the kind of behavior and semantics which are more convenient for
command-line use (whitespace argument separation, --dashed-options) without
colliding with the underlying language.  

By having the 'magic' command system be separate, it is also trivially
extensible by the users (see
http://ipython.scipy.org/doc/manual/node6.html#SECTION00062000000000000000 for
details).  This means that instead of going into a never-ending rabbit-chase of
special cases, you now have a well-defined API (IPython's is primitive but it
works, and we're cleaning it up as part of a major rework) where users can add
arbitrary command functionality which remains separate from the language
itself.

The main point here is, I think, that any good interactive environment for a
programming language requires at least TWO separate kinds of syntax:

1. the language itself, perhaps with aids to economize typing at the command
line (ipython offers many: auto-calling, auto-quoting, readline tricks, input
history as a Python list, output caching, macros, and more).

2. a set of control commands, meant to manipulate the environment itself.  These
can obviously be implemented in the underlying language, but there should be a
way to keep them in a separate namespace (so they don't collide with user
variables and can be always called).


I'll be glad to discuss this further, either here or on the ipython lists where
this is probably more appropriate.  But I wanted to give the python-devs some
perspective on this, from having spent a lot of time dealing directly with the
isues.  

Finally, I'd like to provide a bit of a plea: I hope that python itself doesn't
end up polluting the core sys namespace with too many special things for
interactive use.  I think that a proper interactive console is best implemented
as a separate extension (ipython, pycrust, idle, etc).  Putting all these
special-case things into sys ends up propagating everywhere, and potentially
colliding with such systems (ipython has its own sys.excepthook, for example). 
Please keep the core (sys is very much part of the core) as clean as possible,
and let the interactive shell writers add the functionality outside.

Best,

f

ps - thanks to this thread, I've changed ipython from having the old-style
quit/exit strings with the message, to being magics which exit.  It had always
had %Quit/%Exit for unconditional exit, but now (SVN) it also has %quit/%exit
which honor the confirm_exit rc setting.


From pinard at iro.umontreal.ca  Wed Dec 28 19:42:16 2005
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Wed, 28 Dec 2005 13:42:16 -0500
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <B7EE1E00-6FC7-4404-AF12-90B8444BCD6F@gmail.com>
References: <56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
	<17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
	<doslos$kop$1@sea.gmane.org>
	<17330.5976.560419.584425@montanaro.dyndns.org>
	<2m7j9p1n2w.fsf@starship.python.net>
	<B7EE1E00-6FC7-4404-AF12-90B8444BCD6F@gmail.com>
Message-ID: <20051228184216.GA17771@phenix.sram.qc.ca>

[Alex Martelli]

>On Dec 28, 2005, at 3:24 AM, Michael Hudson wrote:

>> skip at pobox.com writes:

>>>     Fredrik> a quit/exit command that actually quits, instead of 
>>>     Fredric> printing a "you didn't say please!" message.

>>> I like Fredrik's idea more and more.

>> The thing that bothers me about it is that the standard way you tell
>> python to do something is "call a function" -- to me, a special case
>> for exiting the interpreter seems out of proportion.

> Just brainstorming, but -- maybe this means we should generalize the  
> idea?  I.e., allow other cases in which "just mentioning X" means  
> "call function Y [with the following arguments]", at least at the  
> interactive prompt if not more generally.

Just brainstorming then! :-)

The fact that dot notation is used both for accessing globals from 
a module, and attributes from a class or an instance, does not mean that 
modules are classes, but surely, they are instances of something, and 
maybe we could build more on the similarities despite the differences.

Through __getattr__ (and also properties and more generally 
descriptors), it is possible to create instance attributes triggering 
functional behaviour.  Maybe it could be useful creating some (vaguely) 
similar mechanism for modules' globals.  Then, `quit' and `exit' could 
be mere cases of that mechanism.

It is desirable, in my opinion, that interactive behaviour be as 
identical as possible as script behaviour: the automatic printing of 
interactive mode is already much magic that newcomers need to sort out.

-- 
Fran?ois Pinard   http://pinard.progiciels-bpi.ca

From martin at v.loewis.de  Wed Dec 28 19:45:45 2005
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 28 Dec 2005 19:45:45 +0100
Subject: [Python-Dev] Automated Python testing (was Re: status
 of	development documentation)
In-Reply-To: <20051228173633.1217.205642351.divmod.quotient.8563@ohm>
References: <20051228173633.1217.205642351.divmod.quotient.8563@ohm>
Message-ID: <43B2DD59.6060702@v.loewis.de>

Jean-Paul Calderone wrote:
> A slave is an entity capable of performing tasks.  It can be 
> asked to perform any task you like, though it may not be able 
> to perform them all if it lacks some requirements.

This is clear in principle. However, what constitutes a "task"?
I see that you can send it shell commands to execute, arbitrary
ones, with environment variables. What else? Can you send it
Python code?

> A builder is a particular job.  It can be performed by any 
> slave (so long as its requirements are met), or by multiple 
> slaves.

That I'm not so sure of. In a builder, I give a single value
"slavename", and I understand that only that single slave will
ever get jobs from the builder - not any slave. I also read
that I can give "slavenames" instead, but that I should do so
only if the slaves are sufficiently similar (for some metric
which I apparently have to define myself).

> A factory defines the work to be done by a builder.  If which 
> compiler is being used is an important part of the purpose of 
> a builder (for example, there might be a builder the purpose of 
> which is to test a Python built with GCC 3.4 and another the 
> purpose of which is to test a Python built with GCC 4.0), then 
> it should be specified in the master configuration.  If it is 
> not important, then it should be left as general as possible, 
> to allow for the most slaves to be able to complete the task.

So would the assumption be that I use the same factory for
multiple builders? I'm gravitating towards a 1:1:1 relationship
between factories, builders, and slaves. For example, on OSX,
I think I have to give --with-suffix=.exe to configure. This
means I have to create a separate factory, which then only
applies to OSX builders (of which I have only one), which
has just a single slavename.

> I have most often seen branches supported by allowing commits 
> to trunk to automatically trigger a build on trunk, while 
> commits to branches do not automatically trigger any builds.

I think I want it differently: commits to release24-maint should
also trigger builds; commits to other branches should not trigger
builds. Does that mean I need twice as many builders? How
do I tell them what branch they should build?

> Builds on branches can then be explicitly requested when a 
> developer wants to know the state of their branch on various 
> platforms/configurations (often before merging to trunk to 
> know if they are introducing any regressions, but valuable 
> at other times as well).

Some people advised me that supporting web-initiated builds
is not a good idea. So I turned off all manual triggering of
builds for now. I would like to give committers permission to
trigger builds, but I'm not sure how to do that.

> I think this means tags/release24-maint won't ever be built 
> automatically (I haven't used AnyBranchScheduler, and I don't 
> know much about schedulers in buildbot in general).  You 
> should be able to manually request a build, but for some 
> reason I don't see the form for doing so on the master website 
> (<http://twistedmatrix.com/buildbot/full-2.3> for an example 
> of what this looks like).  I'm not sure if this is a buildbot
> version problem, or if there is just another piece of 
> configuration that needs to be set.

Ah, no: that's configuration that would need to be removed.
Jeff Pitmann suggested that this feature be disabled
(Waterfall(allowForce=False)). Before I did that, there
was indeed a form to request building of a branch.

Regards,
Martin

From reinhold-birkenfeld-nospam at wolke7.net  Wed Dec 28 20:44:52 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Wed, 28 Dec 2005 20:44:52 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <doul02$8ja$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org>	<dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com><43B172C9.4070201@v.loewis.de><17329.38860.916447.765575@montanaro.dyndns.org><Pine.LNX.4.58.0512271651040.6831@server1.LFW.org><doslos$kop$1@sea.gmane.org><17330.5976.560419.584425@montanaro.dyndns.org><2m7j9p1n2w.fsf@starship.python.net><B7EE1E00-6FC7-4404-AF12-90B8444BCD6F@gmail.com>	<61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>
	<doul02$8ja$1@sea.gmane.org>
Message-ID: <doupvk$oa4$1@sea.gmane.org>

Fredrik Lundh wrote:
> Walter D?rwald wrote:
> 
>> We have sys.displayhook and sys.excepthook. Why not add a sys.inputhook?
>> sys.inputhook gets passed each line entered and may return True if it has
>> processed the line inself and False if normal handling of the input should be
>> done. This allows special treatment of "quit", "exit", "help" /.../
> 
> so how would such a hook deal with the
> 
>     >>> def exit():
>     ...     pass
>     >>> exit
> 
> case ?

In the inputhook one would have to check for "exit" being defined at
interpreter level.

Reinhold

-- 
Mail address is perfectly valid!


From mal at egenix.com  Wed Dec 28 21:56:43 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 28 Dec 2005 21:56:43 +0100
Subject: [Python-Dev] NotImplemented reaching top-level
In-Reply-To: <20051225113550.GA28497@code1.codespeak.net>
References: <e04bdf310512240931t1049346n@mail.gmail.com>
	<20051225113550.GA28497@code1.codespeak.net>
Message-ID: <43B2FC0B.6070809@egenix.com>

Armin Rigo wrote:
> Hi Facundo,
> 
> On Sat, Dec 24, 2005 at 02:31:19PM -0300, Facundo Batista wrote:
>>>>> d += 1.2
>>>>> d
>> NotImplemented
> 
> The situation appears to be a mess.  Some combinations of specific
> operators fail to convert NotImplemented to a TypeError, depending on
> old- or new-style-class-ness, although this is clearly a bug (e.g. in an
> example like yours but using -= instead of +=, we get the correct
> TypeError.)
> 
> Obviously, we need to write some comprehensive tests about this.  But
> now I just found out that the old, still-pending SF bug #847024 about
> A()*5 in new-style classes hasn't been given any attention; my theory is
> that nobody fully understands the convoluted code paths of abstract.c
> any more :-(

The PEP documenting the coercion logic has complete tables
for what should happen:

http://www.python.org/peps/pep-0208.html

Looking at the code in abstract.c the above problem appears
to be related to the special cases applied to += and *=
in case both operands cannot deal with the type combination.

In such a case, a check is done whether the operation could
be interpreted as sequence operation (concat or repeat) and
then delegated to the appropriate handlers.

But then again, looking in typeobject.c, the following code
could be the cause for leaking a NotImplemented singleton
reference:

#define SLOT1BINFULL(FUNCNAME, TESTFUNC, SLOTNAME, OPSTR, ROPSTR) \
static PyObject * \
FUNCNAME(PyObject *self, PyObject *other) \
{ \
	static PyObject *cache_str, *rcache_str; \
	int do_other = self->ob_type != other->ob_type && \
	    other->ob_type->tp_as_number != NULL && \
	    other->ob_type->tp_as_number->SLOTNAME == TESTFUNC; \
	if (self->ob_type->tp_as_number != NULL && \
	    self->ob_type->tp_as_number->SLOTNAME == TESTFUNC) { \
		PyObject *r; \
		if (do_other && \
		    PyType_IsSubtype(other->ob_type, self->ob_type) && \
		    method_is_overloaded(self, other, ROPSTR)) { \
			r = call_maybe( \
				other, ROPSTR, &rcache_str, "(O)", self); \
			if (r != Py_NotImplemented) \
				return r; \
			Py_DECREF(r); \
			do_other = 0; \
		} \
		r = call_maybe( \
			self, OPSTR, &cache_str, "(O)", other); \
		if (r != Py_NotImplemented || \
		    other->ob_type == self->ob_type) \
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If both types are of the same type, then a NotImplemented returng
value would be returned.

			return r; \
		Py_DECREF(r); \
	} \
	if (do_other) { \
		return call_maybe( \
			other, ROPSTR, &rcache_str, "(O)", self); \
	} \
	Py_INCREF(Py_NotImplemented); \
	return Py_NotImplemented; \
}

Strange enough, the SLOT1BINFULL macro is not used by the
inplace concat or repeat slots...

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 28 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From martin at v.loewis.de  Wed Dec 28 22:51:28 2005
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 28 Dec 2005 22:51:28 +0100
Subject: [Python-Dev] Automated Python testing (was Re: status
 of	development documentation)
In-Reply-To: <20051228194543.1217.425460361.divmod.quotient.8597@ohm>
References: <20051228194543.1217.425460361.divmod.quotient.8597@ohm>
Message-ID: <43B308E0.20205@v.loewis.de>

Jean-Paul Calderone wrote:
> I guess the config for this particular behavior would look something
> like...

You were right that I needed two schedulers for that.

Unfortunately, it doesn't work at all, because svn_buildbot.py does
not report branches on which a change happened, so if you have multiple
schedulers for a subversion source, they either all build when a change
occurs, or none of them.

If svn_version knew about branches (which I'll have to implement,
or incorporate the patch that I saw somewhere), it would probably
work - I have now code to create builders and schedulers in a nested
loop.

> Builds can also be forced using the IRC bot, and there may be a
> commandline tool for doing this as well.  I doubt there's any
> authentication required when using the IRC bot, so it doesn't really
> help restrict forcing to commits only.

Currently, my buildbot isn't connected to IRC at all. If I ever
enable that aspect, I'll use allowForce=False again to disable
remotely invoking builds.

> Another possibility might be to place the form (or just the form action)
> behind HTTP auth.  I'm not sure if this is feasible with the
> authentication mechanism used to restrict access to the svn repository.

Not easily, no. We don't have passwords from the committers, and
authentication through SSH keys is not supported in the Web.

If people really need to be able to force a build, I can activate
that, of course - but not with explicit consent of the operators
of the build slaves.

Regards,
Martin

From fumanchu at amor.org  Wed Dec 28 23:01:09 2005
From: fumanchu at amor.org (Robert Brewer)
Date: Wed, 28 Dec 2005 14:01:09 -0800
Subject: [Python-Dev] Keep default comparisons - or add a second set?
Message-ID: <6949EC6CD39F97498A57E0FA55295B214115C5@ex9.hostedexchange.local>

Noam Raphael wrote:
> I don't think that every type that supports equality
> comparison should support order comparison. I think
> that if there's no meaningful comparison (whether
> equality or order), an exception should be raised.

Just to keep myself sane...

    def date_range(start=None, end=None):
        if start == None:
            start = datetime.date.today()
        if end == None:
            end = datetime.date.today()
        return end - start

Are you saying the "if" statements will raise TypeError if start or end are dates? That would be a sad day for Python. Perhaps you're saying that there is a "meaningful comparison" between None and anything else, but please clarify if so.


Robert Brewer
System Architect
Amor Ministries
fumanchu at amor.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20051228/be413cce/attachment.htm

From noamraph at gmail.com  Wed Dec 28 23:11:24 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Thu, 29 Dec 2005 00:11:24 +0200
Subject: [Python-Dev] Keep default comparisons - or add a second set?
In-Reply-To: <6949EC6CD39F97498A57E0FA55295B214115C5@ex9.hostedexchange.local>
References: <6949EC6CD39F97498A57E0FA55295B214115C5@ex9.hostedexchange.local>
Message-ID: <b348a0850512281411u5e6f5c86wc861fb30b70f1ea6@mail.gmail.com>

On 12/29/05, Robert Brewer <fumanchu at amor.org> wrote:
>
>  Just to keep myself sane...
>
>      def date_range(start=None, end=None):
>          if start == None:
>              start = datetime.date.today()
>          if end == None:
>              end = datetime.date.today()
>          return end - start
>
>  Are you saying the "if" statements will raise TypeError if start or end are
> dates? That would be a sad day for Python. Perhaps you're saying that there
> is a "meaningful comparison" between None and anything else, but please
> clarify if so.

Yes, I'm suggesting that they will raise a TypeError. Your example
shows that the change is not compatible with a lot of existing Python
code, which means that it's a Python 3000 thing. The following code
will continue to work:

    def date_range(start=None, end=None):
        if start is None:
            start = datetime.date.today()
        if end is None:
            end = datetime.date.today()
        return end - start

Using "is None" instead of "== None" is considered a better style even now.

Noam

From tim.peters at gmail.com  Wed Dec 28 23:12:41 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 28 Dec 2005 17:12:41 -0500
Subject: [Python-Dev] Automated Python testing (was Re: status of
	development documentation)
In-Reply-To: <43B308E0.20205@v.loewis.de>
References: <20051228194543.1217.425460361.divmod.quotient.8597@ohm>
	<43B308E0.20205@v.loewis.de>
Message-ID: <1f7befae0512281412q596176d3ma4d965e627d6221d@mail.gmail.com>

[Martin v. L?wis]
> ...
> Unfortunately, it doesn't work at all, because svn_buildbot.py does
> not report branches on which a change happened, so if you have multiple
> schedulers for a subversion source, they either all build when a change
> occurs, or none of them.
>
> If svn_version knew about branches (which I'll have to implement,
> or incorporate the patch that I saw somewhere), it would probably
> work - I have now code to create builders and schedulers in a nested
> loop.

Branches are working sensibly on buildbot.zope.org, so maybe Benji
York (who did most or all of the heavy buildbot lifting at Zope Corp)
could share some clues.  I see that Benji is subscribed to python-dev,
but many people in this part of the world are on vacation this week.

> If people really need to be able to force a build, I can activate
> that, of course - but not with explicit consent of the operators
> of the build slaves.

You can't force a build from:

    http://buildbot.zope.org

either, but there's a different (and "secret") URL from which you can
force builds.   That's been handy for me primarily for experimenting
with theories about why our Windows buildbot slaves get wedged after a
build or two.  The ZC buildbots have also (IMO) been configured to
ignore too much, which is a problem Python doesn't have (e.g., ZODB is
stitched into Zope via svn externals, and for some time the ZC
buildbot didn't think replacing ZODB was "a reason" to run the Zope
tests again).  IOW, I doubt there's much need for it here.

From arigo at tunes.org  Wed Dec 28 23:14:30 2005
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 28 Dec 2005 23:14:30 +0100
Subject: [Python-Dev] NotImplemented reaching top-level
In-Reply-To: <43B2FC0B.6070809@egenix.com>
References: <e04bdf310512240931t1049346n@mail.gmail.com>
	<20051225113550.GA28497@code1.codespeak.net>
	<43B2FC0B.6070809@egenix.com>
Message-ID: <20051228221430.GA2148@code1.codespeak.net>

Hi Marc,

On Wed, Dec 28, 2005 at 09:56:43PM +0100, M.-A. Lemburg wrote:
> >>>>> d += 1.2
> >>>>> d
> >> NotImplemented
> 
> The PEP documenting the coercion logic has complete tables
> for what should happen:

Well, '+=' does not invoke coercion at all, with new-style classes like
Decimal.

> Looking at the code in abstract.c the above problem appears
> to be related to the special cases applied to += and *=
> in case both operands cannot deal with the type combination.
>
> In such a case, a check is done whether the operation could
> be interpreted as sequence operation (concat or repeat) and
> then delegated to the appropriate handlers.

Indeed.  The bug was caused by this delegation, which (prior to my
patch) would also return a Py_NotImplemented that would leak through
abstract.c.  My patch is to remove this unnecessary delegation by not
defining sq_concat/sq_repeat for user-defined classes, and restoring the
original expectation that the sq_concat/sq_repeat slots should not
return Py_NotImplemented.  How does this relate to coercion?

> But then again, looking in typeobject.c, the following code
> could be the cause for leaking a NotImplemented singleton
> reference:
> 
> #define SLOT1BINFULL(FUNCNAME, TESTFUNC, SLOTNAME, OPSTR, ROPSTR) \
> static PyObject * \
> FUNCNAME(PyObject *self, PyObject *other) \
> { \
> 	static PyObject *cache_str, *rcache_str; \
> 	int do_other = self->ob_type != other->ob_type && \
> 	    other->ob_type->tp_as_number != NULL && \
> 	    other->ob_type->tp_as_number->SLOTNAME == TESTFUNC; \
> 	if (self->ob_type->tp_as_number != NULL && \
> 	    self->ob_type->tp_as_number->SLOTNAME == TESTFUNC) { \
> 		PyObject *r; \
> 		if (do_other && \
> 		    PyType_IsSubtype(other->ob_type, self->ob_type) && \
> 		    method_is_overloaded(self, other, ROPSTR)) { \
> 			r = call_maybe( \
> 				other, ROPSTR, &rcache_str, "(O)", self); \
> 			if (r != Py_NotImplemented) \
> 				return r; \
> 			Py_DECREF(r); \
> 			do_other = 0; \
> 		} \
> 		r = call_maybe( \
> 			self, OPSTR, &cache_str, "(O)", other); \
> 		if (r != Py_NotImplemented || \
> 		    other->ob_type == self->ob_type) \
>                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> If both types are of the same type, then a NotImplemented returng
> value would be returned.

Indeed, however:

> 
> 			return r; \
> 		Py_DECREF(r); \
> 	} \
> 	if (do_other) { \
> 		return call_maybe( \
> 			other, ROPSTR, &rcache_str, "(O)", self); \
> 	} \
> 	Py_INCREF(Py_NotImplemented); \
> 	return Py_NotImplemented; \
> }

This last statement also returns Py_NotImplemented.  So it's expected of
this function to be able to return Py_NotImplemented, isn't it?  The
type slots like nb_add can return Py_NotImplemented; the code that
converts it to a TypeError is in the caller, which is abstract.c.


A bientot,

Armin

From noamraph at gmail.com  Wed Dec 28 23:40:14 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Thu, 29 Dec 2005 00:40:14 +0200
Subject: [Python-Dev] set.copy documentation string
Message-ID: <b348a0850512281440j561654c6s54320dc2033e0c1a@mail.gmail.com>

is currently "Return a shallow copy of a set."

Perhaps "shallow" should be removed, since set members are supposed to
be immutable so there's no point in a deep copy?

Noam

From rwgk at yahoo.com  Wed Dec 28 23:58:28 2005
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Wed, 28 Dec 2005 14:58:28 -0800 (PST)
Subject: [Python-Dev] Python + Visual C++ 8.0?
In-Reply-To: <43B06C3F.4030203@v.loewis.de>
Message-ID: <20051228225828.85055.qmail@web31505.mail.mud.yahoo.com>

--- "Martin v. L?wis" <martin at v.loewis.de> wrote:
> P.P.S. You do know that this configuration (extension compiled
> with VS2005, Python compiled wit VS.NET2003) is not supported,
> right?

Thanks to Adal's help I got all our C++ extensions (about 50) to work with VC8.
I am using Python 2.4.2 compiled with VC7.1. We have more than 150 unit tests
which all pass. Our applications also work under Windows 2000 even if the
platform has never seen any Visual Studio installation. Also, in over two years
I never had problems mixing VC6 Python with VC7.1 extensions; our binary
installers worked on any Windows 2000 or XP system I've ever tried. Since we
are using Boost.Python, I believe we are heavily exercising the compiler, the
C/C++ libraries, and the DLL machinery. However, we don't have any home-grown
C++ GUI code. Could it be that problems due to mixing objects from different
compiler versions are restricted to certain areas, like GUI libraries?

Cheers,
        Ralf



	
		
__________________________________ 
Yahoo! for Good - Make a difference this year. 
http://brand.yahoo.com/cybergivingweek2005/

From andrea at suse.de  Wed Dec 28 15:59:17 2005
From: andrea at suse.de (Andrea Arcangeli)
Date: Wed, 28 Dec 2005 15:59:17 +0100
Subject: [Python-Dev] suggestion for smarter garbage collection in
	function of size (gc.set_collect_mem_growth(2))
In-Reply-To: <43B2A1FD.5010300@v.loewis.de>
References: <20051227140516.GB27200@opteron.random>
	<43B2A1FD.5010300@v.loewis.de>
Message-ID: <20051228145917.GI27200@opteron.random>

On Wed, Dec 28, 2005 at 03:32:29PM +0100, "Martin v. L?wis" wrote:
> you should really consider comming up with a patch yourself. Bonus
> points if the code integrates with the current strategies, instead
> of replacing them.

As I wrote in the first email, I've no intention to replace anything.
The new heuristic would be completely orthogonal to the current
strategy. Current strategy is in function of the number, the new
heuristic would be in function of size, and they would co-exist
perfectly.

From andrea at suse.de  Wed Dec 28 15:57:23 2005
From: andrea at suse.de (Andrea Arcangeli)
Date: Wed, 28 Dec 2005 15:57:23 +0100
Subject: [Python-Dev] suggestion for smarter garbage collection in
	function of size (gc.set_collect_mem_growth(2))
In-Reply-To: <20051228135206.GA18974@panix.com>
References: <20051227140516.GB27200@opteron.random>
	<20051228135206.GA18974@panix.com>
Message-ID: <20051228145723.GH27200@opteron.random>

On Wed, Dec 28, 2005 at 05:52:06AM -0800, Aahz wrote:
> If you feel comfortable with C code, the best way to get this to happen
> would be to make the change yourself, then test to find out what effects

I'm more confortable with C code than with python code, that's not the
problem (infact I think that everyone should be confortable with C ;).
The only problem is that my time on this is quite limited, but I will be
really be happy to give it a try.

> this has on Python (in terms of speed and memory usage and whether it
> breaks any of the regression tests).  Once you've satisfied yourself
> that it works, submit a patch, and post here again with the SF number.

Ok.

> Note that since your tunable parameter is presumably accessible from
> Python code, you'll also need to submit doc patches and tests to verify
> that it's working correctly.

Ok.

If there's anybody willing to suggest the files to hook into (the
location where the interpreter allocates all anonymous memory) and how
to invoke gc.collect() from C, that would help. thanks!

From martin at v.loewis.de  Thu Dec 29 00:13:04 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 29 Dec 2005 00:13:04 +0100
Subject: [Python-Dev] set.copy documentation string
In-Reply-To: <b348a0850512281440j561654c6s54320dc2033e0c1a@mail.gmail.com>
References: <b348a0850512281440j561654c6s54320dc2033e0c1a@mail.gmail.com>
Message-ID: <43B31C00.7060502@v.loewis.de>

Noam Raphael wrote:
> is currently "Return a shallow copy of a set."
> 
> Perhaps "shallow" should be removed, since set members are supposed to
> be immutable so there's no point in a deep copy?

That still doesn't make copy return a deep copy, right? "shallow copy"
is more precise than "copy", and correct - what is gained from
removing it?

Regards,
Martin

From martin at v.loewis.de  Thu Dec 29 00:19:29 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 29 Dec 2005 00:19:29 +0100
Subject: [Python-Dev] Python + Visual C++ 8.0?
In-Reply-To: <20051228225828.85055.qmail@web31505.mail.mud.yahoo.com>
References: <20051228225828.85055.qmail@web31505.mail.mud.yahoo.com>
Message-ID: <43B31D81.7000809@v.loewis.de>

Ralf W. Grosse-Kunstleve wrote:
> However, we don't have any home-grown
> C++ GUI code. Could it be that problems due to mixing objects from different
> compiler versions are restricted to certain areas, like GUI libraries?

Well, yes: the areas are
- memory management
- stdio
- locales
for the C library; for the C++, I'm not so sure, but I think one of the
areas is
- static members of class templates, in particular in STL containers

In all these cases, global variables exist twice if you have two copies
of the library defining them in your address space. If you pass objects
around from one library to the other, the other library may operate on
the wrong global variable.

To give some examples:
- memory management: if you malloc() with one library, and free() with
  the other, the memory will be "mostly" leak (i.e. not be reused by
  the malloc() of the first library); the consequence is not a crash,
  but a memory leak, in certain applications
- stdio: if you fprintf to a stdout of a different library, the library
  will crash (GPF)
- static members of class templates: I think it is the rb_tree template
  which will not find its data correctly (but my memory is weak here)

Regards,
Martin

From noamraph at gmail.com  Thu Dec 29 00:22:03 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Thu, 29 Dec 2005 01:22:03 +0200
Subject: [Python-Dev] When do sets shrink?
Message-ID: <b348a0850512281522r1c4f8a01v4288a5e11586b33d@mail.gmail.com>

Hello,

If I do something like this:

s = set()
for i in xrange(1000000):
    s.add(i)
while s:
    s.pop()
gc.collect()

the memory consumption of the process remains the same even after the pops.

I checked the code (that's where I started from, really), and there's
nothing in set.pop or set.remove that resizes the table. And it turns
out that it's the same with dicts.

Should something be done about it?

Noam

From martin at v.loewis.de  Thu Dec 29 00:28:38 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 29 Dec 2005 00:28:38 +0100
Subject: [Python-Dev] suggestion for smarter garbage collection
 in	function of size (gc.set_collect_mem_growth(2))
In-Reply-To: <20051228145723.GH27200@opteron.random>
References: <20051227140516.GB27200@opteron.random>	<20051228135206.GA18974@panix.com>
	<20051228145723.GH27200@opteron.random>
Message-ID: <43B31FA6.8030604@v.loewis.de>

Andrea Arcangeli wrote:
> If there's anybody willing to suggest the files to hook into (the
> location where the interpreter allocates all anonymous memory) and how
> to invoke gc.collect() from C, that would help. thanks!

It all happens in Modules/gcmodule.c:_PyObject_GC_Malloc. There are
per-generation counters; _PyObject_GC_Malloc increments the
generation 0 counter, and PyObject_GC_Del decreases it. The counters
of the higher generations are incremented when a lower collection
occurs.

One challenge is that PyObject_GC_Del doesn't know how large the memory
block is that is being released. So it is difficult to find out how
much memory is being released in the collection.

Regards,
Martin

From noamraph at gmail.com  Thu Dec 29 00:31:44 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Thu, 29 Dec 2005 01:31:44 +0200
Subject: [Python-Dev] set.copy documentation string
In-Reply-To: <43B31C00.7060502@v.loewis.de>
References: <b348a0850512281440j561654c6s54320dc2033e0c1a@mail.gmail.com>
	<43B31C00.7060502@v.loewis.de>
Message-ID: <b348a0850512281531i737fffe0ibdb587664f67abe7@mail.gmail.com>

On 12/29/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Noam Raphael wrote:
> > is currently "Return a shallow copy of a set."
> >
> > Perhaps "shallow" should be removed, since set members are supposed to
> > be immutable so there's no point in a deep copy?
>
> That still doesn't make copy return a deep copy, right? "shallow copy"
> is more precise than "copy", and correct - what is gained from
> removing it?

Perhaps it bothers the programmer with something that shouldn't bother
him. I mean that I might do help(set.copy), and then think, "Oh, it
returns a shallow copy. Wait a minute - 'shallow' means that I get a
new object, which references the same objects as the old one. Perhaps
I should use another function, which does deep copying? Let me think
about it - no. All members of a set are immutable, so it doesn't
matter." I think that in this case, the fact that the copy is shallow
is an implementation detail, since there's no sense in making a deep
copy of a set. Implementation details should probably not be a part of
the definition of what a method does.

I know it's just a word, and that it doesn't matter a lot. But why not
make things a tiny bit better?

Noam

From martin at v.loewis.de  Thu Dec 29 00:32:36 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 29 Dec 2005 00:32:36 +0100
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <b348a0850512281522r1c4f8a01v4288a5e11586b33d@mail.gmail.com>
References: <b348a0850512281522r1c4f8a01v4288a5e11586b33d@mail.gmail.com>
Message-ID: <43B32094.9040105@v.loewis.de>

Noam Raphael wrote:
> Should something be done about it?

It's very difficult to do something useful about it. Even if
you manage to determine how much memory you want to release,
it's nearly impossible to actually release the memory to the
operating system, because of the many layers of memory
management software that all need to agree that the memory
should be reused somewhere else (instead of keeping it on
that layer, just in case somebody using that layer wants
some memory).

Regards,
Martin

From martin at v.loewis.de  Thu Dec 29 00:43:38 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 29 Dec 2005 00:43:38 +0100
Subject: [Python-Dev] set.copy documentation string
In-Reply-To: <b348a0850512281531i737fffe0ibdb587664f67abe7@mail.gmail.com>
References: <b348a0850512281440j561654c6s54320dc2033e0c1a@mail.gmail.com>	<43B31C00.7060502@v.loewis.de>
	<b348a0850512281531i737fffe0ibdb587664f67abe7@mail.gmail.com>
Message-ID: <43B3232A.2000705@v.loewis.de>

Noam Raphael wrote:
> Perhaps it bothers the programmer with something that shouldn't bother
> him. I mean that I might do help(set.copy), and then think, "Oh, it
> returns a shallow copy. Wait a minute - 'shallow' means that I get a
> new object, which references the same objects as the old one. Perhaps
> I should use another function, which does deep copying? Let me think
> about it - no. All members of a set are immutable, so it doesn't
> matter." I think that in this case, the fact that the copy is shallow
> is an implementation detail, since there's no sense in making a deep
> copy of a set.

If "makes no sense" means "would not make a difference", then you
are wrong. Objects in a set are not necessarily unmodifiable;
they just have to be hashable. Watch this:

>>> class A:
...   def __hash__(self):return 0
...   def __cmp__(self, other):return 0
...
>>> a1 = A()
>>> import copy
>>> a2 = copy.deepcopy(a1)
>>> a1.x = 10
>>> a2.x = 20
>>> a1.x
10
>>> s = set([a1])
>>> a2 in s
True

So even though a1 and a2 are distinct objects with different, modifiable
state, they are treated as equal in the set. A deepcopy of the set (if
it existed) would do something different than a plain copy.

> I know it's just a word, and that it doesn't matter a lot. But why not
> make things a tiny bit better?

Things would get worse. The code would become underspecified.

Regards,
Martin

From noamraph at gmail.com  Thu Dec 29 00:51:08 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Thu, 29 Dec 2005 01:51:08 +0200
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <43B32094.9040105@v.loewis.de>
References: <b348a0850512281522r1c4f8a01v4288a5e11586b33d@mail.gmail.com>
	<43B32094.9040105@v.loewis.de>
Message-ID: <b348a0850512281551wa469085v4bf0c1c88e89dd3a@mail.gmail.com>

On 12/29/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Noam Raphael wrote:
> > Should something be done about it?
>
> It's very difficult to do something useful about it. Even if
> you manage to determine how much memory you want to release,
> it's nearly impossible to actually release the memory to the
> operating system, because of the many layers of memory
> management software that all need to agree that the memory
> should be reused somewhere else (instead of keeping it on
> that layer, just in case somebody using that layer wants
> some memory).
>
I checked - when doing the same thing with lists, all the memory was
released for use by other Python objects, and most of it was released
for use by the operating system.

Noam

From raymond.hettinger at verizon.net  Thu Dec 29 00:57:26 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Wed, 28 Dec 2005 18:57:26 -0500
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <b348a0850512281551wa469085v4bf0c1c88e89dd3a@mail.gmail.com>
Message-ID: <001701c60c0a$74f11240$78fdcc97@oemcomputer>

> > It's very difficult to do something useful about it. Even if
> > you manage to determine how much memory you want to release,
> > it's nearly impossible to actually release the memory to the
> > operating system, because of the many layers of memory
> > management software that all need to agree that the memory
> > should be reused somewhere else (instead of keeping it on
> > that layer, just in case somebody using that layer wants
> > some memory).
> >
> I checked - when doing the same thing with lists, all the memory was
> released for use by other Python objects, and most of it was released
> for use by the operating system.

Put another way, it is difficult to assure that the memory is released
back to the operating system.  We don't control that part.

What could be done is to add a test for excess dummy entries and trigger
a periodic resize operation.  That would make the memory available for
other parts of the currently running script and possibly available for
the O/S.

The downside is slowing down a fine-grained operation like pop().  For
dictionaries, this wasn't considered worth it.  For sets, I made the
same design decision.  It wasn't an accident.  I don't plan on changing
that decision unless we find a body of real world code that would be
better-off with more frequent re-sizing.


Raymond


From noamraph at gmail.com  Thu Dec 29 00:59:09 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Thu, 29 Dec 2005 01:59:09 +0200
Subject: [Python-Dev] set.copy documentation string
In-Reply-To: <43B3232A.2000705@v.loewis.de>
References: <b348a0850512281440j561654c6s54320dc2033e0c1a@mail.gmail.com>
	<43B31C00.7060502@v.loewis.de>
	<b348a0850512281531i737fffe0ibdb587664f67abe7@mail.gmail.com>
	<43B3232A.2000705@v.loewis.de>
Message-ID: <b348a0850512281559p78ac27d3o88c1eb12381f7c5e@mail.gmail.com>

On 12/29/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> If "makes no sense" means "would not make a difference", then you
> are wrong. Objects in a set are not necessarily unmodifiable;
> they just have to be hashable.
>
Oh, you are right. I thought so much about dropping default hash=id,
or more generally that only frozen objects should be hashable, that I
forgot that it's not the case yet... :)

(I used the term "frozen" instead of "immutable" since I think that
"immutable" is not defined very well, because tuples are considered
immutable even though their value can change if they reference mutable
objects.)

Thanks,
Noam

From martin at v.loewis.de  Thu Dec 29 01:03:51 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 29 Dec 2005 01:03:51 +0100
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <b348a0850512281551wa469085v4bf0c1c88e89dd3a@mail.gmail.com>
References: <b348a0850512281522r1c4f8a01v4288a5e11586b33d@mail.gmail.com>	
	<43B32094.9040105@v.loewis.de>
	<b348a0850512281551wa469085v4bf0c1c88e89dd3a@mail.gmail.com>
Message-ID: <43B327E7.5020906@v.loewis.de>

Noam Raphael wrote:
> I checked - when doing the same thing with lists, all the memory was
> released for use by other Python objects, and most of it was released
> for use by the operating system.

In this specific case, perhaps. malloc will typically return memory to
the system only if that memory is at the end of the heap. If there
is more memory after block to be released, it can't return the memory
block, because it won't punch a whole into the heap.

So as soon as you have more than one object, things become interesting.

Different systems use different, enhance strategies, of course. For
example, Linux glibc malloc will allocate "large" blocks through
mmap (instead of sbrk), such blocks then can be returned individually.

Regards,
Martin

From noamraph at gmail.com  Thu Dec 29 01:23:31 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Thu, 29 Dec 2005 02:23:31 +0200
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <001701c60c0a$74f11240$78fdcc97@oemcomputer>
References: <b348a0850512281551wa469085v4bf0c1c88e89dd3a@mail.gmail.com>
	<001701c60c0a$74f11240$78fdcc97@oemcomputer>
Message-ID: <b348a0850512281623y55c1c4dej219ebfd6e2ad928c@mail.gmail.com>

On 12/29/05, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> What could be done is to add a test for excess dummy entries and trigger
> a periodic resize operation.  That would make the memory available for
> other parts of the currently running script and possibly available for
> the O/S.
>
> The downside is slowing down a fine-grained operation like pop().  For
> dictionaries, this wasn't considered worth it.  For sets, I made the
> same design decision.  It wasn't an accident.  I don't plan on changing
> that decision unless we find a body of real world code that would be
> better-off with more frequent re-sizing.

The computer scientist in me prefers O() terms over changes in a
constant factor, but that's only me. Perhaps a note about it should be
added to the documentation, though?

Noam

From martin at v.loewis.de  Thu Dec 29 01:36:45 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 29 Dec 2005 01:36:45 +0100
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <b348a0850512281623y55c1c4dej219ebfd6e2ad928c@mail.gmail.com>
References: <b348a0850512281551wa469085v4bf0c1c88e89dd3a@mail.gmail.com>	
	<001701c60c0a$74f11240$78fdcc97@oemcomputer>
	<b348a0850512281623y55c1c4dej219ebfd6e2ad928c@mail.gmail.com>
Message-ID: <43B32F9D.8070800@v.loewis.de>

Noam Raphael wrote:
> The computer scientist in me prefers O() terms over changes in a
> constant factor, but that's only me.

That remark, I don't understand. In a hash table, most "simple"
operations are O(n) as the worst-case time, except for operations
that may cause resizing, which are O(n**2) (worst case).

So you are proposing that .pop() might trigger a resize, thus
changing from O(n) worst case to O(n**2) worst case? Why would
a computer scientist prefer that?

> Perhaps a note about it should be added to the documentation, though?

Sure. Patches (to sf.net/projects/python) are welcome.

Regards,
Martin

From adal.chiriliuc at gmail.com  Thu Dec 29 01:41:10 2005
From: adal.chiriliuc at gmail.com (Adal Chiriliuc)
Date: Thu, 29 Dec 2005 02:41:10 +0200
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <43B327E7.5020906@v.loewis.de>
References: <b348a0850512281522r1c4f8a01v4288a5e11586b33d@mail.gmail.com>
	<43B32094.9040105@v.loewis.de>
	<b348a0850512281551wa469085v4bf0c1c88e89dd3a@mail.gmail.com>
	<43B327E7.5020906@v.loewis.de>
Message-ID: <303741290.20051229024110@myrealbox.com>

On Thursday, December 29, 2005 "Martin v. L?wis" wrote:
> Noam Raphael wrote:

> In this specific case, perhaps. malloc will typically return memory to
> the system only if that memory is at the end of the heap. If there
> is more memory after block to be released, it can't return the memory
> block, because it won't punch a whole into the heap.

> So as soon as you have more than one object, things become interesting.

> Different systems use different, enhance strategies, of course. For
> example, Linux glibc malloc will allocate "large" blocks through
> mmap (instead of sbrk), such blocks then can be returned individually.

MSVC 7.1 and 8.0 malloc always uses the Windows heap functions
(HeapAlloc & friends) if running on Windows 2000 or newer
(malloc.c and heapinit.c).

So it seems that for both Linux (gcc) and Win (msvc) the memory is
released to the operating system.

Of course, fragmentation is still an issue, but now it's on the OS
side of things.


From rwgk at yahoo.com  Thu Dec 29 02:02:14 2005
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Wed, 28 Dec 2005 17:02:14 -0800 (PST)
Subject: [Python-Dev] Python + Visual C++ 8.0?
In-Reply-To: <43B31D81.7000809@v.loewis.de>
Message-ID: <20051229010214.58631.qmail@web31503.mail.mud.yahoo.com>

--- "Martin v. L?wis" <martin at v.loewis.de> wrote:

> Well, yes: the areas are
> - memory management
> - stdio
> - locales
> for the C library; for the C++, I'm not so sure, but I think one of the
> areas is
> - static members of class templates, in particular in STL containers

Thanks for the insight! For Boost.Python users the situation is actually not
bad at all:

- there is not a single malloc() in the Boost.Python sources

- there is only one free(), which is specific to gcc; i.e. this code is not
seen by Visual C++.

- there is only one FILE* that is seen by Visual C++. It appears in the
signature of a function assigned to the tp_print slot. As far as I can tell
this is the only soft spot. I'll lobby for having the print function removed
since it is optional and both tp_str and tp_repr are defined anyway.

- nobody should be mixing C++ libraries compiled with different compilers on
any platform; it doesn't get much more dangerous than this. Since Python is
pure C "static members of class templates" should therefore be a non-issue.

This leaves only "stdio" and "locales" to be considered. We are strictly
avoiding both in C++ to maximize portability. I/O is intrinsically slow and
therefore there is no point in coding it in C++, especially since Python's I/O
is very fast anyway. I am not sure about locales because this is not very
interesting at all for scientific code development.

Under the assumption that the one tp_print function in the Boost.Python sources
is removed, I am arriving at the conclusion that it is more than pure luck that
all our tests work. People writing C++ extensions view the compiled layer as
low-level number-crunching back-end. It is quite natural not to find I/O and
use of locales in such code. And even if there is I/O, chances are it is done
via std::cout rather than a raw FILE* obtained from Python via difficult to
memorize incantations (I don't even know how to do it).

Cheers,
        Ralf



		
__________________________________________ 
Yahoo! DSL ? Something to write home about. 
Just $16.99/mo. or less. 
dsl.yahoo.com 


From noamraph at gmail.com  Thu Dec 29 02:11:05 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Thu, 29 Dec 2005 03:11:05 +0200
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <43B32F9D.8070800@v.loewis.de>
References: <b348a0850512281551wa469085v4bf0c1c88e89dd3a@mail.gmail.com>
	<001701c60c0a$74f11240$78fdcc97@oemcomputer>
	<b348a0850512281623y55c1c4dej219ebfd6e2ad928c@mail.gmail.com>
	<43B32F9D.8070800@v.loewis.de>
Message-ID: <b348a0850512281711u302f588m198dc3b70ec3f76d@mail.gmail.com>

On 12/29/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Noam Raphael wrote:
> > The computer scientist in me prefers O() terms over changes in a
> > constant factor, but that's only me.
>
> That remark, I don't understand. In a hash table, most "simple"
> operations are O(n) as the worst-case time, except for operations
> that may cause resizing, which are O(n**2) (worst case).
>
> So you are proposing that .pop() might trigger a resize, thus
> changing from O(n) worst case to O(n**2) worst case? Why would
> a computer scientist prefer that?
>
Perhaps I'm not that great computer scientist, but simple operations
on hash tables are supposed to be O(1) in the average case (which is
what matters to me). This means that resizing is an O(n) operation in
the average case.

This means that just like list.append() and list.pop(), that are made
O(1) operations amortized, so can operations on hash table be made
O(1) amortized - all you need is to use the trick for setting bounds
so that resize operations will always happen after O(n) simple
operations.

> > Perhaps a note about it should be added to the documentation, though?
>
> Sure. Patches (to sf.net/projects/python) are welcome.

I will try to send one when sf becomes healthier.

Noam

From adal.chiriliuc at gmail.com  Thu Dec 29 02:23:23 2005
From: adal.chiriliuc at gmail.com (Adal Chiriliuc)
Date: Thu, 29 Dec 2005 03:23:23 +0200
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <303741290.20051229024110@myrealbox.com>
References: <b348a0850512281522r1c4f8a01v4288a5e11586b33d@mail.gmail.com>
	<43B32094.9040105@v.loewis.de>
	<b348a0850512281551wa469085v4bf0c1c88e89dd3a@mail.gmail.com>
	<43B327E7.5020906@v.loewis.de> <303741290.20051229024110@myrealbox.com>
Message-ID: <13973404.20051229032323@myrealbox.com>

I did a little test using MSVC 8.0 on WinXP.

I allocated 128 MB using 128 different buffers of 1 MB each,
freed half of them (alternatively) then freed the remaining half.

I monitored memory usage using the Task Manager and memory is really
freed as indicated by both the "Mem Usage" and "VM Size" process
counters and also by the global "Commit Charge". After exiting the
process the commit charge remains the same, suggesting that all the
memory was indeed released.

Even if the Windows heap documentation says that once memory is
allocated, it won't be released until the heap is destroyed, this
seems to not be true when allocating large chunks. There probably is a
threshold after which VirtualAlloc is used.


#include <windows.h>
#include <stdio.h>
#include <malloc.h>

#define COUNT 128

void main()
{
    char** ptr = malloc(COUNT * sizeof(char*));

    int i;
    for (i = 0; i < COUNT; ++i)
    {
        ptr[i] = malloc(1024 * 1024);
        // Touch memory to really commit it.
        SecureZeroMemory(ptr[i], 1024 * 1024);
    }

    printf("Mem allocated\n");
    Sleep(5000);

    for (i = 0; i < COUNT; i += 2)
        free(ptr[i]);
    printf("Half memory deallocated\n");
    Sleep(10000);

    for (i = 1; i < COUNT; i += 2)
        free(ptr[i]);
    printf("Full memory deallocated\n");
    Sleep(10000);
}


From martin at v.loewis.de  Thu Dec 29 02:27:36 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 29 Dec 2005 02:27:36 +0100
Subject: [Python-Dev] Python + Visual C++ 8.0?
In-Reply-To: <20051229010214.58631.qmail@web31503.mail.mud.yahoo.com>
References: <20051229010214.58631.qmail@web31503.mail.mud.yahoo.com>
Message-ID: <43B33B88.40501@v.loewis.de>

Ralf W. Grosse-Kunstleve wrote:
> - there is only one FILE* that is seen by Visual C++. It appears in the
> signature of a function assigned to the tp_print slot. As far as I can tell
> this is the only soft spot. I'll lobby for having the print function removed
> since it is optional and both tp_str and tp_repr are defined anyway.

Out of curiosity: can you please try invoking this enum_print to stdout
with your VS2005-built boost module? I see it uses fprintf, so I would
expect it to crash.

Regards,
Martin

From martin at v.loewis.de  Thu Dec 29 02:35:25 2005
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 29 Dec 2005 02:35:25 +0100
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <303741290.20051229024110@myrealbox.com>
References: <b348a0850512281522r1c4f8a01v4288a5e11586b33d@mail.gmail.com>	<43B32094.9040105@v.loewis.de>	<b348a0850512281551wa469085v4bf0c1c88e89dd3a@mail.gmail.com>	<43B327E7.5020906@v.loewis.de>
	<303741290.20051229024110@myrealbox.com>
Message-ID: <43B33D5D.9010209@v.loewis.de>

Adal Chiriliuc wrote:
> MSVC 7.1 and 8.0 malloc always uses the Windows heap functions
> (HeapAlloc & friends) if running on Windows 2000 or newer
> (malloc.c and heapinit.c).
> 
> So it seems that for both Linux (gcc) and Win (msvc) the memory is
> released to the operating system.

How so? HeapFree does not return the memory to the system. From

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/heap_functions.asp

"Once the pages are committed, they are not decommitted until the
process is terminated or until the heap is destroyed by calling the
HeapDestroy function."

Regards,
Martin

From nas at arctrix.com  Thu Dec 29 03:14:32 2005
From: nas at arctrix.com (Neil Schemenauer)
Date: Thu, 29 Dec 2005 02:14:32 +0000 (UTC)
Subject: [Python-Dev] suggestion for smarter garbage collection
	in	function of size (gc.set_collect_mem_growth(2))
References: <20051227140516.GB27200@opteron.random>
	<20051228135206.GA18974@panix.com>
	<20051228145723.GH27200@opteron.random>
	<43B31FA6.8030604@v.loewis.de>
Message-ID: <dovgq8$3g6$1@sea.gmane.org>

Martin v. L?wis <martin at v.loewis.de> wrote:
> One challenge is that PyObject_GC_Del doesn't know how large the memory
> block is that is being released. So it is difficult to find out how
> much memory is being released in the collection.

Another idea would be to add accounting to the PyMem_* interfaces.
It could be that most memory is used by objects that are not tracked
by the GC (e.g. strings).  I guess you still have the same problem
in that PyMem_Free may not know how large the memory block is.

  Neil


From ncoghlan at gmail.com  Thu Dec 29 04:18:59 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 29 Dec 2005 13:18:59 +1000
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <doupvk$oa4$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org>	<dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com><43B172C9.4070201@v.loewis.de><17329.38860.916447.765575@montanaro.dyndns.org><Pine.LNX.4.58.0512271651040.6831@server1.LFW.org><doslos$kop$1@sea.gmane.org><17330.5976.560419.584425@montanaro.dyndns.org><2m7j9p1n2w.fsf@starship.python.net><B7EE1E00-6FC7-4404-AF12-90B8444BCD6F@gmail.com>	<61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>	<doul02$8ja$1@sea.gmane.org>
	<doupvk$oa4$1@sea.gmane.org>
Message-ID: <43B355A3.5050109@gmail.com>

Reinhold Birkenfeld wrote:
> Fredrik Lundh wrote:
>> Walter D?rwald wrote:
>>
>>> We have sys.displayhook and sys.excepthook. Why not add a sys.inputhook?
>>> sys.inputhook gets passed each line entered and may return True if it has
>>> processed the line inself and False if normal handling of the input should be
>>> done. This allows special treatment of "quit", "exit", "help" /.../
>> so how would such a hook deal with the
>>
>>     >>> def exit():
>>     ...     pass
>>     >>> exit
>>
>> case ?
> 
> In the inputhook one would have to check for "exit" being defined at
> interpreter level.

Which is fairly trivial given a slight change to my proposed default input hook:

    def default_inputhook(statement):
        if statement in vars(sys.modules["__main__"]):
            return False
        try:
            aliased = sys.alias[statement]
        except KeyError:
            return False
        else:
            aliased()
            return True

That is, a real variable will always shadow an alias - you need to get rid of 
the real variable before the alias will start working again (or else change 
the name of the alias).

Or you can give the alias a different name via:

   sys.alias["exit_"] = sys.alias["exit"]

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From tim.peters at gmail.com  Thu Dec 29 04:23:53 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 28 Dec 2005 22:23:53 -0500
Subject: [Python-Dev] suggestion for smarter garbage collection in
	function of size (gc.set_collect_mem_growth(2))
In-Reply-To: <43B31FA6.8030604@v.loewis.de>
References: <20051227140516.GB27200@opteron.random>
	<20051228135206.GA18974@panix.com>
	<20051228145723.GH27200@opteron.random> <43B31FA6.8030604@v.loewis.de>
Message-ID: <1f7befae0512281923y4d143fabmbae71b0e9af09e56@mail.gmail.com>

[Martin v. L?wis]
> ...
> One challenge is that PyObject_GC_Del doesn't know how large the
> memory block is that is being released. So it is difficult to find out how
> much memory is being released in the collection.

"Impossible in some cases" is accurate.  When pymalloc isn't enabled,
all these things call the platform malloc/free directly, and there's
no portable/standard way to find out anything from those.  When
pymalloc is enabled, PyObject_GC_Del could be taught whether pymalloc
controls the block being freed, and, when so, how to suck up the
block's size index from the block's pool header; but when pymalloc
doesn't control the memory being freed, it's the same as if pymalloc
were not enabled.

[Neil Schemenauer]
> Another idea would be to add accounting to the PyMem_* interfaces.
> It could be that most memory is used by objects that are not tracked
> by the GC (e.g. strings).

I still expect this old code in pymem.h to go away for Python 2.5:

/* In order to avoid breaking old code mixing PyObject_{New, NEW} with
   PyMem_{Del, DEL} and PyMem_{Free, FREE}, the PyMem "release memory"
   functions have to be redirected to the object deallocator. */
#define PyMem_FREE           	PyObject_FREE

When goes away, PyMem_FREE will resolve directly to the platform
free(), and will no longer have even accidental relationships to any
memory involved in cyclic gc.

> I guess you still have the same problem in that PyMem_Free may not know how
> large the memory block is.

It will be more the case that we can guarantee it won't know -- but
since direct uses of malloc/free have no useful relationship to cyclic
gc behavior, the OP shouldn't care about that.

In any case, the OP's original "the overhead of this should be zero"
claim isn't credible (I checked, and there _still_ won't be free
lunches in 2006 -- unless you work at Google ;-)).

From stephen at xemacs.org  Thu Dec 29 05:35:07 2005
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 29 Dec 2005 13:35:07 +0900
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: =?iso-8859-1?q?=3C43B2A00B=2E1080001=40v=2Eloewis=2Ede=3E_?=
	=?iso-8859-1?q?=28Martin_v=2E_L=F6wis's_message_of_=22Wed=2C_28_Dec_200?=
	=?iso-8859-1?q?5_15=3A24=3A11_+0100=22=29?=
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
	<17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
	<doslos$kop$1@sea.gmane.org>
	<17330.5976.560419.584425@montanaro.dyndns.org>
	<2m7j9p1n2w.fsf@starship.python.net> <43B2A00B.1080001@v.loewis.de>
Message-ID: <87u0csv7us.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Martin" == Martin v L?wis <martin at v.loewis.de> writes:

    Martin> That would assume that the user knows that exit is a
    Martin> function: apparently, people expect it to be a statement
    Martin> (like print),

Oh, the irony of that analogy!<wink>

    Martin> or they are entirely unaware of the distinctions between
    Martin> statements, expressions, and functions.

Then there's little point to giving them access to the interpreter!

Of course, Martin also mentioned "students".  Python is not the only
language in the world; we all know that, even if most of us much
prefer programming in Python to any other environment.  If you're
teaching, why not use this as an opportunity to deliver a brief
lecture on why Python does things differently, and why one should
understand a _formal_ language in its own terms, not in terms of what
you (the user/programmer) want it to be for momentary convenience?

My feeling is that the current behavior of "exit" and "quit" is not
"you didn't say please" but "excuse me, I don't speak BASIC; would you
say that in Python or signal that the conversation is over (ie, EOF)?"

For me, that's part of the Zen of Python.  Probably I'm just missing
something given the amount of support this idea is getting, from
really respectable sources, too, but it just seems wrong to change
this.  What's wrong with having a distinctive personality?

I suppose the current value of those variables sounds a bit rude.  Why
not fix the discourtesy, rather than what's not broken (IMHO)?

exit = """\
The Python interpreter simply interprets a Python program. It provides
no special interactive commands.  The line editor is a thin front-end
to the standard input stream.  To exit your program, use one of the
functions or exceptions provided for this purpose, or simply end the
file (interactively this is signaled by Ctrl-D).

This public service message brought to you by the global variable 'exit'."""



-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From aahz at pythoncraft.com  Thu Dec 29 05:40:24 2005
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 28 Dec 2005 20:40:24 -0800
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <43B355A3.5050109@gmail.com>
References: <dor982$qrc$1@sea.gmane.org>
	<61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>
	<doul02$8ja$1@sea.gmane.org> <doupvk$oa4$1@sea.gmane.org>
	<43B355A3.5050109@gmail.com>
Message-ID: <20051229044024.GA15414@panix.com>

Here's yet a different take on this: why not simply change the startup
message?  Whether we choose "quit" or "exit", someone will get it wrong
unless there's an alias.  Changing the message is free.  Currently we
have

    Type "help", "copyright", "credits" or "license" for more information.

Let's add another line that says

    Type "quit()" to exit

Defining it as "def quit(): raise SystemExit" should be fine.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Given that C++ has pointers and typecasts, it's really hard to have a serious 
conversation about type safety with a C++ programmer and keep a straight face.
It's kind of like having a guy who juggles chainsaws wearing body armor 
arguing with a guy who juggles rubber chickens wearing a T-shirt about who's 
in more danger."  --Roy Smith

From jcarlson at uci.edu  Thu Dec 29 06:03:50 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 28 Dec 2005 21:03:50 -0800
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <b348a0850512281623y55c1c4dej219ebfd6e2ad928c@mail.gmail.com>
References: <001701c60c0a$74f11240$78fdcc97@oemcomputer>
	<b348a0850512281623y55c1c4dej219ebfd6e2ad928c@mail.gmail.com>
Message-ID: <20051228205253.BF07.JCARLSON@uci.edu>


Noam Raphael <noamraph at gmail.com> wrote:
> 
> On 12/29/05, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> > What could be done is to add a test for excess dummy entries and trigger
> > a periodic resize operation.  That would make the memory available for
> > other parts of the currently running script and possibly available for
> > the O/S.
> >
> > The downside is slowing down a fine-grained operation like pop().  For
> > dictionaries, this wasn't considered worth it.  For sets, I made the
> > same design decision.  It wasn't an accident.  I don't plan on changing
> > that decision unless we find a body of real world code that would be
> > better-off with more frequent re-sizing.
> 
> The computer scientist in me prefers O() terms over changes in a
> constant factor, but that's only me. Perhaps a note about it should be
> added to the documentation, though?

The computer scientist in me agrees with you, but the practical
application developer in me argues that every microsecond adds up.

 - Josiah


From bcannon at gmail.com  Thu Dec 29 06:06:46 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Wed, 28 Dec 2005 21:06:46 -0800
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <20051229044024.GA15414@panix.com>
References: <dor982$qrc$1@sea.gmane.org>
	<61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>
	<doul02$8ja$1@sea.gmane.org> <doupvk$oa4$1@sea.gmane.org>
	<43B355A3.5050109@gmail.com> <20051229044024.GA15414@panix.com>
Message-ID: <bbaeab100512282106k2bd8f019labfd4ba70c779360@mail.gmail.com>

On 12/28/05, Aahz <aahz at pythoncraft.com> wrote:
> Here's yet a different take on this: why not simply change the startup
> message?  Whether we choose "quit" or "exit", someone will get it wrong
> unless there's an alias.  Changing the message is free.  Currently we
> have
>
>     Type "help", "copyright", "credits" or "license" for more information.
>
> Let's add another line that says
>
>     Type "quit()" to exit
>
> Defining it as "def quit(): raise SystemExit" should be fine.
> --

Just because people don't read unless it it thrown in their face.  =) 
But that is still nice and simple and won't hurt.  I would still
suggest using a class so that the repr can give a useful message,
though.

-Brett

From aahz at pythoncraft.com  Thu Dec 29 06:10:08 2005
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 28 Dec 2005 21:10:08 -0800
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <bbaeab100512282106k2bd8f019labfd4ba70c779360@mail.gmail.com>
References: <dor982$qrc$1@sea.gmane.org>
	<61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>
	<doul02$8ja$1@sea.gmane.org> <doupvk$oa4$1@sea.gmane.org>
	<43B355A3.5050109@gmail.com> <20051229044024.GA15414@panix.com>
	<bbaeab100512282106k2bd8f019labfd4ba70c779360@mail.gmail.com>
Message-ID: <20051229051008.GA2459@panix.com>

On Wed, Dec 28, 2005, Brett Cannon wrote:
> On 12/28/05, Aahz <aahz at pythoncraft.com> wrote:
>>
>> Here's yet a different take on this: why not simply change the startup
>> message?  Whether we choose "quit" or "exit", someone will get it wrong
>> unless there's an alias.  Changing the message is free.  Currently we
>> have
>>
>>     Type "help", "copyright", "credits" or "license" for more information.
>>
>> Let's add another line that says
>>
>>     Type "quit()" to exit
>>
>> Defining it as "def quit(): raise SystemExit" should be fine.
> 
> Just because people don't read unless it it thrown in their face.  =) 
> But that is still nice and simple and won't hurt.  I would still
> suggest using a class so that the repr can give a useful message,
> though.

That's fine.  I think the primary use case for this is someone who
accidentally stumbles into the Python interpreter, in which case that
message will be a life-saver.  Anybody who is formally attempting to use
the interpreter probably has ready access to docs that explain how to
use it.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Given that C++ has pointers and typecasts, it's really hard to have a serious 
conversation about type safety with a C++ programmer and keep a straight face.
It's kind of like having a guy who juggles chainsaws wearing body armor 
arguing with a guy who juggles rubber chickens wearing a T-shirt about who's 
in more danger."  --Roy Smith

From guido at python.org  Thu Dec 29 07:27:35 2005
From: guido at python.org (Guido van Rossum)
Date: Wed, 28 Dec 2005 22:27:35 -0800
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <dorn8g$uhk$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
Message-ID: <ca471dc20512282227k5dade465g1914039c826403f9@mail.gmail.com>

On 12/27/05, Fredrik Lundh <fredrik at pythonware.com> wrote:
> but now we're back to today's situation:
>
>     >>> quit
>     'Use Ctrl-Z plus Return to exit.'
>
> which violates the basic "if you know what I mean, why the /!"&/&!//%?
> don't you do what I say" usability rule.

What nonsense. Every Python programmer knows that the right way to
exit Python is to enter a platform EOF. The quit and exit variables
are compromises for non-programmers who have accidentally entered
Python and who don't know how to get out of it (this is usually the
same category of people who don't know how or don't dare to kill a
program using heavy artillery).

Rather than improving upon the quit/exit functionality a la /F's patch
I'd get rid of the compromise.

Now, for Py3K we could try to come up with a more intelligent
interactive mode. Probably not implemented in C. Perhaps using ideas
from ipython (which I've personally never used). We might provide a
quit or exit command using a more sophisticated mechanism. But there
are always costs (e.g. the violation of the primciple that typing a
name shows its repr()).

In the mean time I'm a strong believer in "it ain't broke so don't fix it" here.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fredrik at pythonware.com  Thu Dec 29 07:55:53 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 29 Dec 2005 07:55:53 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org>
	<ca471dc20512282227k5dade465g1914039c826403f9@mail.gmail.com>
Message-ID: <dp019r$fvf$1@sea.gmane.org>

Guido van Rossum wrote:

> > but now we're back to today's situation:
> >
> >     >>> quit
> >     'Use Ctrl-Z plus Return to exit.'
> >
> > which violates the basic "if you know what I mean, why the /!"&/&!//%¤
> > don't you do what I say" usability rule.
>
> What nonsense. Every Python programmer knows that the right way to
> exit Python is to enter a platform EOF. The quit and exit variables
> are compromises for non-programmers who have accidentally entered
> Python and who don't know how to get out of it

why is it that "non-pythoneers" are always referred to as "non-programmers" ?

is the current user base so large so we no longer need to care about people
moving in from other environments ?

(for those who follow non-python forums, there's a meme going around out
there that says that python developers are arrogant, and will always defend
the existing design with "we're right, you're wrong" or a "we're right, you're
stupid", and that the current "exit" non-design is an excellent example of this.
my proposal was partially motivated by an urge to prove them wrong, but it
seems as I accidentally proved them to be right...)

</F>




From steve at holdenweb.com  Thu Dec 29 08:09:04 2005
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 29 Dec 2005 07:09:04 +0000
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <ca471dc20512282227k5dade465g1914039c826403f9@mail.gmail.com>
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>
	<ca471dc20512282227k5dade465g1914039c826403f9@mail.gmail.com>
Message-ID: <dp022m$hkm$1@sea.gmane.org>

Guido van Rossum wrote:
> On 12/27/05, Fredrik Lundh <fredrik at pythonware.com> wrote:
> 
>>but now we're back to today's situation:
>>
>>    >>> quit
>>    'Use Ctrl-Z plus Return to exit.'
>>
>>which violates the basic "if you know what I mean, why the /!"&/&!//%?
>>don't you do what I say" usability rule.
> 
> 
> What nonsense. Every Python programmer knows that the right way to
> exit Python is to enter a platform EOF. The quit and exit variables
> are compromises for non-programmers who have accidentally entered
> Python and who don't know how to get out of it (this is usually the
> same category of people who don't know how or don't dare to kill a
> program using heavy artillery).
> 
Except that if you have iPython installed on Windows you *don't* enter 
the platform EOF any more, you enter CTRL/D (which threw me for a 
while). Plus the standard Windows build requires a return after the 
CTRL/Z while the Pythonware versions (last time I looked) only required 
the CTRL/Z. So the situation is a little less black-and-white than it 
might be.

> Rather than improving upon the quit/exit functionality a la /F's patch
> I'd get rid of the compromise.
> 
I *have* been surprised by the amount on brainpower that's been expended 
on this discussion.

> Now, for Py3K we could try to come up with a more intelligent
> interactive mode. Probably not implemented in C. Perhaps using ideas
> from ipython (which I've personally never used). We might provide a
> quit or exit command using a more sophisticated mechanism. But there
> are always costs (e.g. the violation of the primciple that typing a
> name shows its repr()).
> 
> In the mean time I'm a strong believer in "it ain't broke so don't fix it" here.
> 
+1, with the proviso that we might add a description of how to exit to 
the interactive rubric (if anyone can work out exactly what that would 
be under all circumstances).

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From fperez.net at gmail.com  Thu Dec 29 08:21:31 2005
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 29 Dec 2005 00:21:31 -0700
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org>
	<ca471dc20512282227k5dade465g1914039c826403f9@mail.gmail.com>
	<dp022m$hkm$1@sea.gmane.org>
Message-ID: <dp02pr$iir$1@sea.gmane.org>

Steve Holden wrote:

> Except that if you have iPython installed on Windows you *don't* enter
> the platform EOF any more, you enter CTRL/D (which threw me for a
> while).

To be fair, that's due to the win32 readline library used by ipython, which
modifies console handling.  IPython itself doesn't do anything to the EOF
conventions, it's pure python code with no direct access to the console.

Cheers,

f


From fredrik at pythonware.com  Thu Dec 29 08:46:59 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 29 Dec 2005 08:46:59 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org><dorgc0$ca9$1@sea.gmane.org>	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>	<dorn8g$uhk$1@sea.gmane.org><ca471dc20512282227k5dade465g1914039c826403f9@mail.gmail.com>
	<dp022m$hkm$1@sea.gmane.org>
Message-ID: <dp049l$mk3$1@sea.gmane.org>

Steve Holden wrote:

> I *have* been surprised by the amount on brainpower that's been
> expended on this discussion.

number of posts != amount of brainpower

</F>




From bob at redivi.com  Thu Dec 29 10:22:35 2005
From: bob at redivi.com (Bob Ippolito)
Date: Thu, 29 Dec 2005 04:22:35 -0500
Subject: [Python-Dev] suggestion for smarter garbage collection in
	function of size (gc.set_collect_mem_growth(2))
In-Reply-To: <20051227140516.GB27200@opteron.random>
References: <20051227140516.GB27200@opteron.random>
Message-ID: <44CD8598-644E-4C5A-9C11-77DFC120BA44@redivi.com>


On Dec 27, 2005, at 9:05 AM, Andrea Arcangeli wrote:

> I run into a problem recently with a reconnectingclientfactory with
> twisted while write some spare time software, that turned out to be  
> a gc
> inefficiency.
>
> In short the protocol memory wasn't released after the reconnect  
> and the
> protocol had about 50M attached to it. So with frequent reconnects the
> rss of the task grown to >1G and it pushed the system into heavy swap.
> In reality only 50M were allocated, so 950M were wasted by python.

In this particular case, you might be better off just writing some  
Twisted code that periodically checks the size of the current process  
and does a gc.collect() when necessary.  Of course, it requires some  
platform specific code, but presumably you only care about one, maybe  
two, platforms anyway.

-bob


From walter at livinglogic.de  Thu Dec 29 11:15:16 2005
From: walter at livinglogic.de (=?iso-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu, 29 Dec 2005 11:15:16 +0100 (CET)
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <e8a0972d0512281010t4853f3a8r936f277c28d83ab9@mail.gmail.com>
References: <dor982$qrc$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
	<17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
	<doslos$kop$1@sea.gmane.org>
	<17330.5976.560419.584425@montanaro.dyndns.org>
	<2m7j9p1n2w.fsf@starship.python.net>
	<B7EE1E00-6FC7-4404-AF12-90B8444BCD6F@gmail.com>
	<61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>
	<e8a0972d0512281010t4853f3a8r936f277c28d83ab9@mail.gmail.com>
Message-ID: <61143.85.74.40.200.1135851316.squirrel@isar.livinglogic.de>

Alex Martelli wrote:

> On 12/28/05, Walter D?rwald <walter at livinglogic.de> wrote:
>   ...
>> We have sys.displayhook and sys.excepthook. Why not add a sys.inputhook?
>
> Sure, particularly with Nick's suggestion for a default input hook it would be fine.

I'd like the inputhook to be able to define the prompt. I'm not sure how this could be accomplished.

Another API would be that the inputhook returns what line or command should be executed instead, e.g.

def default_inputhook(statement):
   if statement.endswith("?"):
      return "help(%s)" % statement[:-1]
   etc.

>> > sessions in which I want to perform some action repeatedly, and currently the least typing is 4 characters (x()<enter>)
>> > while this   would reduce it to two
>>
>> What's wrong with <cursor up>, <return>?
>
> The fact that there is no upper bound to the number of cursor-up
> keystrokes needed here -- by "perform some action repeatedly" I don't mean "half a dozen times right after each other with
> nothing
> in-between" (sorry for the ambiguous phrasing), but "numerous times, repeatedly in the course of an interactive interpreter
> session".

OK, for that I've modified my .inputrc to map <page up> to incremental-search-backwards. With this it's: <x> <page up> <return>
(and it even works when the input is x("averylonginputstring")). That's not as fast as <x> <return>, but it's close.
Bye,
   Walter D?rwald




From fredrik at pythonware.com  Thu Dec 29 12:30:12 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 29 Dec 2005 12:30:12 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org><dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><ca471dc20512282227k5dade465g1914039c826403f9@mail.gmail.com>
	<dp019r$fvf$1@sea.gmane.org>
Message-ID: <dp0hc5$u51$1@sea.gmane.org>

> (for those who follow non-python forums

make that "those who don't follow"

</F>




From abo at minkirri.apana.org.au  Thu Dec 29 12:29:58 2005
From: abo at minkirri.apana.org.au (Donovan Baarda)
Date: Thu, 29 Dec 2005 11:29:58 +0000
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <001701c60c0a$74f11240$78fdcc97@oemcomputer>
References: <001701c60c0a$74f11240$78fdcc97@oemcomputer>
Message-ID: <1135855798.4203.15.camel@warna.dub.corp.google.com>

On Wed, 2005-12-28 at 18:57 -0500, Raymond Hettinger wrote:
[...]
> What could be done is to add a test for excess dummy entries and trigger
> a periodic resize operation.  That would make the memory available for
> other parts of the currently running script and possibly available for
> the O/S.
> 
> The downside is slowing down a fine-grained operation like pop().  For
> dictionaries, this wasn't considered worth it.  For sets, I made the
> same design decision.  It wasn't an accident.  I don't plan on changing
> that decision unless we find a body of real world code that would be
> better-off with more frequent re-sizing.

I don't think it will ever be worth it.

Re-allocations that grow are expensive, as they often need to move the
entire contents from the old small allocation to the new larger
allocation. Re-allocations that shrink can also be expensive, or at the
least increase heap fragmentation. So you want to avoid re-allocations
whenever possible.

The ideal size for any container is "as big as it needs to be". The best
heuristic for this is probably "as big as it's ever been, or if it just
got bigger than that, assume it's half way through growing". which is
what Python currently does.

Without some sort of fancy overkill size hinting or history tracking,
that's probably as good a heuristic as you can get.

-- 
Donovan Baarda <abo at minkirri.apana.org.au>
http://minkirri.apana.org.au/~abo/


From fredrik at pythonware.com  Thu Dec 29 12:55:02 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 29 Dec 2005 12:55:02 +0100
Subject: [Python-Dev] When do sets shrink?
References: <b348a0850512281522r1c4f8a01v4288a5e11586b33d@mail.gmail.com>	<43B32094.9040105@v.loewis.de>	<b348a0850512281551wa469085v4bf0c1c88e89dd3a@mail.gmail.com>	<43B327E7.5020906@v.loewis.de><303741290.20051229024110@myrealbox.com>
	<43B33D5D.9010209@v.loewis.de>
Message-ID: <dp0iqn$2fd$1@sea.gmane.org>

Martin v. Löwis wrote:

> Adal Chiriliuc wrote:
> > MSVC 7.1 and 8.0 malloc always uses the Windows heap functions
> > (HeapAlloc & friends) if running on Windows 2000 or newer
> > (malloc.c and heapinit.c).
> >
> > So it seems that for both Linux (gcc) and Win (msvc) the memory is
> > released to the operating system.
>
> How so? HeapFree does not return the memory to the system. From
>
> http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/heap_functions.asp
>
> "Once the pages are committed, they are not decommitted until the
> process is terminated or until the heap is destroyed by calling the
> HeapDestroy function."

http://www.python.org/sf/1389051 agrees with the microsoft
documentation.  (where imaplib runs out of memory after read-
ing 2 megabytes of a 14 megabyte message).

</F>




From noamraph at gmail.com  Thu Dec 29 13:14:27 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Thu, 29 Dec 2005 14:14:27 +0200
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <1135855798.4203.15.camel@warna.dub.corp.google.com>
References: <001701c60c0a$74f11240$78fdcc97@oemcomputer>
	<1135855798.4203.15.camel@warna.dub.corp.google.com>
Message-ID: <b348a0850512290414i3e7f307hf2a01a6dc64c27d9@mail.gmail.com>

On 12/29/05, Donovan Baarda <abo at minkirri.apana.org.au> wrote:
> Without some sort of fancy overkill size hinting or history tracking,
> that's probably as good a heuristic as you can get.

I'm sorry, but it's not correct. There's a simple resize scheduling
algorithm that is proven to take, when you sum things up, O(1) per
each simple operation, and that keeps the amount of used memory always
proportional to the number of elements in the set.

I'm not saying that practically it must be used - I'm just saying that
it can't be called a heuristic, and that it doesn't involve any "fancy
overkill size hinting or history tracking". It actually means
something like this:
1. If you want to insert and the table is full, resize the table to
twice the current size.
2. If you delete and the number of elements turns out to be less than
a quarter of the size of the table, resize the table to half of the
current size.

Noam

From abo at minkirri.apana.org.au  Thu Dec 29 13:29:00 2005
From: abo at minkirri.apana.org.au (Donovan Baarda)
Date: Thu, 29 Dec 2005 12:29:00 +0000
Subject: [Python-Dev] file() vs open(), round 7
In-Reply-To: <20051226043831.GA24197@panix.com>
References: <20051226043831.GA24197@panix.com>
Message-ID: <1135859340.4203.42.camel@warna.dub.corp.google.com>

On Sun, 2005-12-25 at 20:38 -0800, Aahz wrote:
> Guido sez in 
> http://mail.python.org/pipermail/python-dev/2004-July/045921.html
> that it is not correct to recommend using ``file()`` instead of
> ``open()``.  However, because ``open()`` currently *is* an alias to
> ``file()``, we end up with the following problem (verified in current
> HEAD) where doing ``help(open)`` brings up the docs for ``file()``:
[...]
> This is confusing.  I suggest that we make ``open()`` a factory function
> right now.  (I'll submit a bug report (and possibly a patch) after I get
> agreement.)

Not totally related but...

way back in 2001-2002, I did some work on writing a Virtual File System
interface for Python. See;

http://minkirri.apana.org.au/~abo/projects/osVFS

The idea was that you could import a module "vfs" as "os", and then any
file operations would go through the virtual file system. I had modules
for things "fakeroot", "mountable", "ftpfs" etc. The vfs module had full
os functionality so it was a "drop in replacement".

The one wart was open(), because it is the only filesystem operation
that wasn't in the os module. At the time I worked around this by adding
a vfs.file() method, and suggesting that people alias open() to
vfs.file(). Note that os.open() already exists as a low-level file open
function, and hence could not be used as a file-object-factory method.

I'm wondering if it wouldn't be a good idea to centralise all filesystem
operations into the os module, including open() or file(). Perhaps the
builtin open() and file() could call os.file()... or P3K could remove
the builtins... I dunno... it just felt ugly at the time that open() was
the one oddity.

-- 
Donovan Baarda <abo at minkirri.apana.org.au>
http://minkirri.apana.org.au/~abo/


From skip at pobox.com  Thu Dec 29 15:04:53 2005
From: skip at pobox.com (skip@pobox.com)
Date: Thu, 29 Dec 2005 08:04:53 -0600
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <dp0hc5$u51$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<ca471dc20512282227k5dade465g1914039c826403f9@mail.gmail.com>
	<dp019r$fvf$1@sea.gmane.org> <dp0hc5$u51$1@sea.gmane.org>
Message-ID: <17331.60677.125318.419572@montanaro.dyndns.org>


    >> (for those who follow non-python forums
    Fredrik> make that "those who don't follow"

What might some of those non-python forums be?

Skip

From martin at v.loewis.de  Thu Dec 29 15:04:30 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 29 Dec 2005 15:04:30 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
Message-ID: <43B3ECEE.9060300@v.loewis.de>

Please let me know what you think.

Regards,
Martin

PEP: XXX
Title: Using ssize_t as the index type
Version: $Revision$
Last-Modified: $Date$
Author: Martin v. L?wis <martin at v.loewis.de>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 18-Dec-2005
Post-History:


Abstract
========

In Python 2.4, indices of sequences are restricted to the C type
int. On 64-bit machines, sequences therefore cannot use the full
address space, and are restricted to 2**31 elements. This PEP proposes
to change this, introducing a platform-specific index type
Py_ssize_t. An implementation of the proposed change is in
http://svn.python.org/projects/python/branches/ssize_t.


Rationale
=========

64-bit machines are becoming more popular, and the size of main memory
increases beyond 4GiB. On such machines, Python currently is limited,
in that sequences (strings, unicode objects, tuples, lists,
array.arrays, ...)  cannot contain more than 2GiElements.

Today, very few machines have memory to represent larger lists: as
each pointer is 8B (in a 64-bit machine), one needs 16GiB to just hold
the pointers of such a list; with data in the list, the memory
consumption grows even more.  However, there are three container types
for which users request improvements today:

* strings (currently restricted to 2GiB)
* mmap objects (likewise; plus the system typically
  won't keep the whole object in memory concurrently)
* Numarray objects (from Numerical Python)

As the proposed change will cause incompatibilities on 64-bit
machines, it should be carried out while such machines are not in wide
use (IOW, as early as possible).


Specification
=============

A new type Py_ssize_t is introduced, which has the same size as the
compiler's size_t type, but is signed. It will be a typedef for
ssize_t where available.

The internal representation of the length fields of all container
types is changed from int to ssize_t, for all types included in the
standard distribution.  In particular, PyObject_VAR_HEAD is changed to
use Py_ssize_t, affecting all extension modules that use that macro.

All occurrences of index and length parameters and results are changed
to use Py_ssize_t, including the sequence slots in type objects.

New conversion functions PyInt_FromSsize_t, PyInt_AsSsize_t,
PyLong_AsSsize_t are introduced. PyInt_FromSsize_t will transparently
return a long int object if the value exceeds the MAX_INT.

New function pointer typedefs ssizeargfunc, ssizessizeargfunc,
ssizeobjargproc, and ssizessizeobjargproc are introduced.

A new conversion code 'n' is introduced for PyArg_ParseTuple
and Py_BuildValue, which operates on Py_ssize_t.

The conversion codes 's#' and 't#' will output Py_ssize_t
if the macro PY_SIZE_T_CLEAN is defined before Python.h
is included, and continue to output int if that macro
isn't defined.

At places where a conversion from size_t/Py_ssize_t to
int is necessary, the strategy for conversion is chosen
on a case-by-case basis (see next section).


Conversion guidelines
=====================

Module authors have the choice whether they support this PEP in their
code or not; if they support it, they have the choice of different
levels of compatibility.

If a module is not converted to support this PEP, it will continue to
work unmodified on a 32-bit system.  On a 64-bit system, compile-time
errors and warnings might be issued, and the module might crash the
interpreter if the warnings are ignored.

Conversion of a module can either attempt to continue using int
indices, or use Py_ssize_t indices throughout.

If the module should continue to use int indices, care must be taken
when calling functions that return Py_ssize_t or size_t, in
particular, for functions that return the length of an object (this
includes the strlen function and the sizeof operator). A good compiler
will warn when a Py_ssize_t/size_t value is truncated into an int.
In these cases, three strategies are available:

*  statically determine that the size can never exceed an int
   (e.g. when taking the sizeof a struct, or the strlen of
   a file pathname). In this case, add a debug assert() that
   the value is indeed smaller than INT_MAX, and cast the
   value to int.
*  statically determine that the value shouldn't overflow an
   int unless there is a bug in the C code somewhere. Test
   whether the value is smaller than INT_MAX, and raise an
   InternalError if it isn't.
*  otherwise, check whether the value fits an int, and raise
   a ValueError if it doesn't.

The same care must be taking for tp_as_sequence slots, in
addition, the signatures of these slots change, and the
slots must be explicitly recast (e.g. from intargfunc
to ssizeargfunc). Compatibility with previous Python
versions can be achieved with the test::

 #if PY_VERSION_HEX < 0x02050000
 typedef int Py_ssize_t;
 #endif

and then using Py_ssize_t in the rest of the code. For
the tp_as_sequence slots, additional typedefs might
be necessary; alternatively, by replacing::

 PyObject* foo_item(struct MyType* obj, int index)
 {
   ...
 }

with::

 PyObject* foo_item(PyObject* _obj, Py_ssize_t index)
 {
    struct MyType* obj = (struct MyType*)_obj;
    ...
 }

it becomes possible to drop the cast entirely; the type
of foo_item should then match the sq_item slot in all
Python versions.

If the module should be extended to use Py_ssize_t indices, all usages
of the type int should be reviewed, to see whether it should be
changed to Py_ssize_t. The compiler will help in finding the spots,
but a manual review is still necessary.

Particular care must be taken for PyArg_ParseTuple calls:
they need all be checked for s# and t# converters, and
PY_SIZE_T_CLEAN must be defined before including Python.h
if the calls have been updated accordingly.


Discussion
==========

Why not size_t
--------------

An initial attempt to implement this feature tried to use
size_t. It quickly turned out that this cannot work: Python
uses negative indices in many places (to indicate counting
from the end). Even in places where size_t would be usable,
to many reformulations of code where necessary, e.g. in
loops like::

  for(index = length-1; index >= 0; index--)

This loop will never terminate if index is changed from
int to size_t.

Doesn't this break much code?
-----------------------------

With the changes proposed, code breakage is fairly
minimal. On a 32-bit system, no code will break, as
Py_ssize_t is just a typedef for int.

On a 64-bit system, the compiler will warn in many
places. If these warnings are ignored, the code will
continue to work as long as the container sizes don't
exceeed 2**31, i.e. it will work nearly as good as
it does currently. There are two exceptions to this
statement: if the extension module implements the
sequence protocol, it must be updated, or the calling
conventions will be wrong. The other exception is
the places where Py_ssize_t is output through a
pointer (rather than a return value); this applies
most notably to codecs and slice objects.

If the conversion of the code is made, the same code
can continue to work on earlier Python releases.

Doesn't this consume too much memory?
-------------------------------------

One might think that using Py_ssize_t in all tuples,
strings, lists, etc. is a waste of space. This is
not true, though: on a 32-bit machine, there is no
change. On a 64-bit machine, the size of many
containers doesn't change, e.g.

* in lists and tuples, a pointer immediately follows
  the ob_size member. This means that the compiler
  currently inserts a 4 padding bytes; with the
  change, these padding bytes become part of the size.
* in strings, the ob_shash field follows ob_size.
  This field is of type long, which is a 64-bit
  type on most 64-bit systems (except Win64), so
  the compiler inserts padding before it as well.


Copyright
=========

This document has been placed in the public domain.


From fredrik at pythonware.com  Thu Dec 29 15:20:41 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 29 Dec 2005 15:20:41 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><ca471dc20512282227k5dade465g1914039c826403f9@mail.gmail.com><dp019r$fvf$1@sea.gmane.org>
	<dp0hc5$u51$1@sea.gmane.org>
	<17331.60677.125318.419572@montanaro.dyndns.org>
Message-ID: <dp0rbr$t0e$1@sea.gmane.org>

skip at pobox.com wrote:

>     Fredrik> make that "those who don't follow"
>
> What might some of those non-python forums be?

assorted corners of the blogosphere, mostly.  no time to dig up any explicit
references, since I'm preparing for a 650 km trip through a major snowstorm,
but searching backwards from bruce eckel's recent "The departure of the
hyper-enthusiasts" might help you find some:

    http://www.artima.com/weblogs/viewpost.jsp?thread=141312

</F>




From aahz at pythoncraft.com  Thu Dec 29 15:37:23 2005
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 29 Dec 2005 06:37:23 -0800
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <ca471dc20512282227k5dade465g1914039c826403f9@mail.gmail.com>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<ca471dc20512282227k5dade465g1914039c826403f9@mail.gmail.com>
Message-ID: <20051229143723.GA5154@panix.com>

On Wed, Dec 28, 2005, Guido van Rossum wrote:
>
> In the mean time I'm a strong believer in "it ain't broke so don't fix
> it" here.

Does that also include my suggestion about improving the startup message?
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Given that C++ has pointers and typecasts, it's really hard to have a serious 
conversation about type safety with a C++ programmer and keep a straight face.
It's kind of like having a guy who juggles chainsaws wearing body armor 
arguing with a guy who juggles rubber chickens wearing a T-shirt about who's 
in more danger."  --Roy Smith

From mal at egenix.com  Thu Dec 29 15:40:36 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 29 Dec 2005 15:40:36 +0100
Subject: [Python-Dev] NotImplemented reaching top-level
In-Reply-To: <20051228221430.GA2148@code1.codespeak.net>
References: <e04bdf310512240931t1049346n@mail.gmail.com>	<20051225113550.GA28497@code1.codespeak.net>
	<43B2FC0B.6070809@egenix.com>
	<20051228221430.GA2148@code1.codespeak.net>
Message-ID: <43B3F564.60604@egenix.com>

Hi Armin,

> On Wed, Dec 28, 2005 at 09:56:43PM +0100, M.-A. Lemburg wrote:
>>>>>>> d += 1.2
>>>>>>> d
>>>> NotImplemented
>> The PEP documenting the coercion logic has complete tables
>> for what should happen:
> 
> Well, '+=' does not invoke coercion at all, with new-style classes like
> Decimal.

True, it doesn't invoke coercion in the sense that a coercion
method is called, but the mechanism described in the PEP is
still used via PyNumber_InPlaceAdd().

>> Looking at the code in abstract.c the above problem appears
>> to be related to the special cases applied to += and *=
>> in case both operands cannot deal with the type combination.
>>
>> In such a case, a check is done whether the operation could
>> be interpreted as sequence operation (concat or repeat) and
>> then delegated to the appropriate handlers.
> 
> Indeed.  The bug was caused by this delegation, which (prior to my
> patch) would also return a Py_NotImplemented that would leak through
> abstract.c.  My patch is to remove this unnecessary delegation by not
> defining sq_concat/sq_repeat for user-defined classes, and restoring the
> original expectation that the sq_concat/sq_repeat slots should not
> return Py_NotImplemented.  How does this relate to coercion?

The Py_NotImplemented singleton was introduced in the coercion
proposal to mean "there is no implementation to execute the requested
operation on the given combination of types".

At the time we also considered using an exception for this, but
it turned out that this caused too much of a slow-down. Hence the use
of a special singleton which could be tested for by a simple
pointer comparison.

Originally, the singleton was only needed for mixed-type operations.
It seems that its use has spread to other areas as well and
can now also refer to missing same-type operator implementations.

>> But then again, looking in typeobject.c, the following code
>> could be the cause for leaking a NotImplemented singleton
>> reference:
>>
>> #define SLOT1BINFULL(FUNCNAME, TESTFUNC, SLOTNAME, OPSTR, ROPSTR) \
>> static PyObject * \
>> FUNCNAME(PyObject *self, PyObject *other) \
>> { \
>> 	static PyObject *cache_str, *rcache_str; \
>> 	int do_other = self->ob_type != other->ob_type && \
>> 	    other->ob_type->tp_as_number != NULL && \
>> 	    other->ob_type->tp_as_number->SLOTNAME == TESTFUNC; \
>> 	if (self->ob_type->tp_as_number != NULL && \
>> 	    self->ob_type->tp_as_number->SLOTNAME == TESTFUNC) { \
>> 		PyObject *r; \
>> 		if (do_other && \
>> 		    PyType_IsSubtype(other->ob_type, self->ob_type) && \
>> 		    method_is_overloaded(self, other, ROPSTR)) { \
>> 			r = call_maybe( \
>> 				other, ROPSTR, &rcache_str, "(O)", self); \
>> 			if (r != Py_NotImplemented) \
>> 				return r; \
>> 			Py_DECREF(r); \
>> 			do_other = 0; \
>> 		} \
>> 		r = call_maybe( \
>> 			self, OPSTR, &cache_str, "(O)", other); \
>> 		if (r != Py_NotImplemented || \
>> 		    other->ob_type == self->ob_type) \
>>                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> If both types are of the same type, then a NotImplemented returng
>> value would be returned.
> 
> Indeed, however:
> 
>> 			return r; \
>> 		Py_DECREF(r); \
>> 	} \
>> 	if (do_other) { \
>> 		return call_maybe( \
>> 			other, ROPSTR, &rcache_str, "(O)", self); \
>> 	} \
>> 	Py_INCREF(Py_NotImplemented); \
>> 	return Py_NotImplemented; \
>> }
> 
> This last statement also returns Py_NotImplemented.  So it's expected of
> this function to be able to return Py_NotImplemented, isn't it?  The
> type slots like nb_add can return Py_NotImplemented; the code that
> converts it to a TypeError is in the caller, which is abstract.c.

You're right - silly me.

Regards,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 29 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From guido at python.org  Thu Dec 29 16:35:09 2005
From: guido at python.org (Guido van Rossum)
Date: Thu, 29 Dec 2005 07:35:09 -0800
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <20051229143723.GA5154@panix.com>
References: <dor982$qrc$1@sea.gmane.org> <dorgc0$ca9$1@sea.gmane.org>
	<60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com>
	<56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com>
	<dorn8g$uhk$1@sea.gmane.org>
	<ca471dc20512282227k5dade465g1914039c826403f9@mail.gmail.com>
	<20051229143723.GA5154@panix.com>
Message-ID: <ca471dc20512290735v20033812k6ba1d0f3b445152e@mail.gmail.com>

On 12/29/05, Aahz <aahz at pythoncraft.com> wrote:
> On Wed, Dec 28, 2005, Guido van Rossum wrote:
> > In the mean time I'm a strong believer in "it ain't broke so don't fix
> > it" here.
>
> Does that also include my suggestion about improving the startup message?

Nobody reads that; plus it looks like it's excruciatingly complex to
get it right at all times; and getting it wrong is worse that not
mentioning it at all.

Regarding the meme floating about the arrogance of Pythoneers:
bloggers (pretty much by definition) are actually the most arrogant
species; don't confuse "bloggers say" with "most people think".

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Dec 29 16:37:11 2005
From: guido at python.org (Guido van Rossum)
Date: Thu, 29 Dec 2005 07:37:11 -0800
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43B3ECEE.9060300@v.loewis.de>
References: <43B3ECEE.9060300@v.loewis.de>
Message-ID: <ca471dc20512290737m252fdf4cuf7bdcb65ef0972dc@mail.gmail.com>

+1. I think this is long overdue. While I can't judge the amount of
code breakage, 2.5 is as good an opportunity as any.

--Guido

On 12/29/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Please let me know what you think.
>
> Regards,
> Martin
>
> PEP: XXX
> Title: Using ssize_t as the index type
> Version: $Revision$
> Last-Modified: $Date$
> Author: Martin v. L?wis <martin at v.loewis.de>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 18-Dec-2005
> Post-History:
>
>
> Abstract
> ========
>
> In Python 2.4, indices of sequences are restricted to the C type
> int. On 64-bit machines, sequences therefore cannot use the full
> address space, and are restricted to 2**31 elements. This PEP proposes
> to change this, introducing a platform-specific index type
> Py_ssize_t. An implementation of the proposed change is in
> http://svn.python.org/projects/python/branches/ssize_t.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From collinw at gmail.com  Thu Dec 29 16:46:28 2005
From: collinw at gmail.com (Collin Winter)
Date: Thu, 29 Dec 2005 16:46:28 +0100
Subject: [Python-Dev] Bug in Py_InitModule4
Message-ID: <43aa6ff70512290746v38d182e6n81ea3220a25ce0b6@mail.gmail.com>

Hello all,

While working with Subversion's python API bindings this morning, I
discovered a function in one of their modules illegally named "import"
(svn.client.import, for the curious). Because the extension module in
question is written in C, the interpreter doesn't flag the
otherwise-illegal identifier "import" at compile-time; if you try to
call the function at runtime, however, the interpreter raises a
SyntaxError, since svn.client.import is an illegal name.

My question is this: is there any interest in preventing situations
like this by including checks in Python/modsupport.c:Py_InitModule4 to
make sure that the PyMethodDef contains only valid identifiers, or is
this a case of "if it hurts when you do that, don't do that"? I can
see a case for both sides: on the one hand, it would be nice to
prevent people from accidentally creating inaccessible objects. On the
other hand, perhaps this is a job that should be given to tools like
SWIG, since they're the ones actually generating the bindings (in the
case of SVN).

I've already reported this to the SVN people, but if there's any
interest in a CPython-side solution, I'm more than willing to work up
a patch to modsupport.c.

Thanks,
Collin Winter

From guido at python.org  Thu Dec 29 16:59:48 2005
From: guido at python.org (Guido van Rossum)
Date: Thu, 29 Dec 2005 07:59:48 -0800
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <b348a0850512290414i3e7f307hf2a01a6dc64c27d9@mail.gmail.com>
References: <001701c60c0a$74f11240$78fdcc97@oemcomputer>
	<1135855798.4203.15.camel@warna.dub.corp.google.com>
	<b348a0850512290414i3e7f307hf2a01a6dc64c27d9@mail.gmail.com>
Message-ID: <ca471dc20512290759g4b989f2ave74c791ad367b270@mail.gmail.com>

On 12/29/05, Noam Raphael <noamraph at gmail.com> wrote:
> On 12/29/05, Donovan Baarda <abo at minkirri.apana.org.au> wrote:
> > Without some sort of fancy overkill size hinting or history tracking,
> > that's probably as good a heuristic as you can get.
>
> I'm sorry, but it's not correct. There's a simple resize scheduling
> algorithm that is proven to take, when you sum things up, O(1) per
> each simple operation, and that keeps the amount of used memory always
> proportional to the number of elements in the set.
>
> I'm not saying that practically it must be used - I'm just saying that
> it can't be called a heuristic, and that it doesn't involve any "fancy
> overkill size hinting or history tracking". It actually means
> something like this:
> 1. If you want to insert and the table is full, resize the table to
> twice the current size.
> 2. If you delete and the number of elements turns out to be less than
> a quarter of the size of the table, resize the table to half of the
> current size.

I'm neutral on doing this.

I'd like to point out that if we decide not to do this, there's an
easy way to shrink dicts and sets under user control, which means that
you only have to pay attention in the rare cases where it matters:
create a new dct or set from the old one automatically creates one of
the right size. E.g.

s = set(s)

replaces the set s (which may once have had 1M items and now has
nearly 1M empty slots) by one with the "optimal" number of slots.
Ditto for dicts:

d = dict(d)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fredrik at pythonware.com  Thu Dec 29 17:13:01 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 29 Dec 2005 17:13:01 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><ca471dc20512282227k5dade465g1914039c826403f9@mail.gmail.com>
	<20051229143723.GA5154@panix.com>
Message-ID: <dp11ue$lpa$1@sea.gmane.org>

Aahz wrote:

> Does that also include my suggestion about improving the startup message?

when newbies get to the point that they want to quit, chances are
that the message have scrolled out of sight.  and if they only skim
the instructions, they'll probably get confused anyway...  e.g.

Python version gobbledidok gobbledidok gobbledidok gobbledidok
Type "help", "copyright", "credits" or "license" for more information.
>>> help
Type help() for interactive help, or help(object) for help about object.
>>> help()

Welcome to Python 2.4!  This is the online help utility.

If this is your first time using Python, you should definitely check out
the tutorial on the Internet at http://www.python.org/doc/tut/.

Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules.  To quit this help utility and
return to the interpreter, just type "quit".

To get a list of available modules, keywords, or topics, type "modules",
"keywords", or "topics".  Each module also comes with a one-line summary
of what it does; to list the modules whose summaries contain a given word
such as "spam", type "modules spam".

help> quit()
no Python documentation found for 'quit()'

help> quit

You are now leaving help and returning to the Python interpreter.
If you want to ask for help on a particular object directly from the
interpreter, you can type "help(object)".  Executing "help('string')"
has the same effect as typing a particular string at the help> prompt.

>>> quit
'Use Ctrl-Z plus Return to exit.'
>>> quit()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: 'str' object is not callable
>>> help(quit)
no Python documentation found for 'Use Ctrl-Z plus Return to exit.'
>>> help("quit")
Help on str object:

quit = class str(basestring)
 |  str(object) -> string
 |
 |  Return a nice string representation of the object.
 |  If the argument is a string, the return value is the same object.
 |
 |  Method resolution order:
 |      str
 |      basestring
 |      object
 |
 |  Methods defined here:
 |
 |  __add__(...)
 |      x.__add__(y) <==> x+y
 |
 |  __contains__(...)
 |      x.__contains__(y) <==> y in x
 |
 |  __eq__(...)
 |      x.__eq__(y) <==> x==y
 |
 |  __ge__(...)
 |      x.__ge__(y) <==> x>=y
 |
 |  __getattribute__(...)
 |      x.__getattribute__('name') <==> x.name
 |
 |  __getitem__(...)
 |      x.__getitem__(y) <==> x[y]
 |
 |  __getnewargs__(...)
 |
 |  __getslice__(...)
 |      x.__getslice__(i, j) <==> x[i:j]
 |
 |      Use of negative indices is not supported.
 |
 |  __gt__(...)
 |      x.__gt__(y) <==> x>y
 |
 |  __hash__(...)
 |      x.__hash__() <==> hash(x)
 |
 |  __le__(...)
 |      x.__le__(y) <==> x<=y
 |
 |  __len__(...)
 |      x.__len__() <==> len(x)
 |
 |  __lt__(...)
 |      x.__lt__(y) <==> x<y
 |
 |  __mod__(...)
 |      x.__mod__(y) <==> x%y
 |
 |  __mul__(...)
 |      x.__mul__(n) <==> x*n
 |
 |  __ne__(...)
 |      x.__ne__(y) <==> x!=y
 |
 |  __repr__(...)
 |      x.__repr__() <==> repr(x)
 |
 |  __rmod__(...)
 |      x.__rmod__(y) <==> y%x
 |
 |  __rmul__(...)
 |      x.__rmul__(n) <==> n*x
 |
 |  __str__(...)
-- More  --

(and so on)




From fperez.net at gmail.com  Thu Dec 29 17:13:44 2005
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 29 Dec 2005 09:13:44 -0700
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>
	<F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com>
	<43B172C9.4070201@v.loewis.de>
	<17329.38860.916447.765575@montanaro.dyndns.org>
	<Pine.LNX.4.58.0512271651040.6831@server1.LFW.org>
	<doslos$kop$1@sea.gmane.org>
	<17330.5976.560419.584425@montanaro.dyndns.org>
	<2m7j9p1n2w.fsf@starship.python.net>
	<B7EE1E00-6FC7-4404-AF12-90B8444BCD6F@gmail.com>
	<61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>
	<e8a0972d0512281010t4853f3a8r936f277c28d83ab9@mail.gmail.com>
	<61143.85.74.40.200.1135851316.squirrel@isar.livinglogic.de>
Message-ID: <dp11vp$lv8$1@sea.gmane.org>

Walter D?rwald wrote:

> Alex Martelli wrote:
> 
>> On 12/28/05, Walter D?rwald <walter at livinglogic.de> wrote:
>>   ...
>>> We have sys.displayhook and sys.excepthook. Why not add a sys.inputhook?
>>
>> Sure, particularly with Nick's suggestion for a default input hook it would
>> be fine.
> 
> I'd like the inputhook to be able to define the prompt. I'm not sure how this
> could be accomplished.
> 
> Another API would be that the inputhook returns what line or command should be
> executed instead, e.g.
> 
> def default_inputhook(statement):
>    if statement.endswith("?"):
>       return "help(%s)" % statement[:-1]
>    etc.

And you're on your way to re-writing ipython:

In [1]: x='hello'

In [2]: x?
Type:           str
Base Class:     <type 'str'>
String Form:    hello
Namespace:      Interactive
Length:         5
Docstring:
    str(object) -> string

    Return a nice string representation of the object.
    If the argument is a string, the return value is the same object.

I also started it with "let's add a cool hack to sys.ps1 and sys.displayhook in
10 minutes".  Now we're at 18000 lines of python, a 70 page manual, and growing
suport for remote distributed interactive computing, a new scientific computing
GUI, and more  :)   If you like this kind of thing, by all means join in: I can
use all the helping hands I can get.

Cheers,

f


From fredrik at pythonware.com  Thu Dec 29 17:17:46 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 29 Dec 2005 17:17:46 +0100
Subject: [Python-Dev] When do sets shrink?
References: <001701c60c0a$74f11240$78fdcc97@oemcomputer><1135855798.4203.15.camel@warna.dub.corp.google.com>
	<b348a0850512290414i3e7f307hf2a01a6dc64c27d9@mail.gmail.com>
Message-ID: <dp127b$mt2$1@sea.gmane.org>

Noam Raphael wrote:

> I'm not saying that practically it must be used - I'm just saying that
> it can't be called a heuristic, and that it doesn't involve any "fancy
> overkill size hinting or history tracking". It actually means
> something like this:
> 1. If you want to insert and the table is full, resize the table to
> twice the current size.
> 2. If you delete and the number of elements turns out to be less than
> a quarter of the size of the table, resize the table to half of the
> current size.

sure sounds like a heuristic algorithm to me... (as in "not guaranteed to
be optimal under all circumstances, even if it's probably quite good in all
practical cases")

</F>




From aahz at pythoncraft.com  Thu Dec 29 17:26:08 2005
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 29 Dec 2005 08:26:08 -0800
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43B3ECEE.9060300@v.loewis.de>
References: <43B3ECEE.9060300@v.loewis.de>
Message-ID: <20051229162608.GA923@panix.com>

Not sure what I think of the proposal (though I guess I'm overall +0 --
needs to be done sometime and no time like the present).

However, I think this PEP should be held up as an example of how to write
a good PEP.  Aside from my inability to follow some of the arcane points
due to lack of C programming skill, this PEP was extremely readable and
well-organized.  Nice job!
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Given that C++ has pointers and typecasts, it's really hard to have a
serious conversation about type safety with a C++ programmer and keep a
straight face.  It's kind of like having a guy who juggles chainsaws
wearing body armor arguing with a guy who juggles rubber chickens wearing
a T-shirt about who's in more danger."  --Roy Smith

From martin at v.loewis.de  Thu Dec 29 17:36:59 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 29 Dec 2005 17:36:59 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <20051229162608.GA923@panix.com>
References: <43B3ECEE.9060300@v.loewis.de> <20051229162608.GA923@panix.com>
Message-ID: <43B410AB.2070506@v.loewis.de>

Aahz wrote:
> However, I think this PEP should be held up as an example of how to write
> a good PEP.  Aside from my inability to follow some of the arcane points
> due to lack of C programming skill, this PEP was extremely readable and
> well-organized.  Nice job!

Thanks! Part of it probably stems from thinking about this stuff a lot.

Regards,
Martin

From fredrik at pythonware.com  Thu Dec 29 17:20:22 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 29 Dec 2005 17:20:22 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org><F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com><43B172C9.4070201@v.loewis.de><17329.38860.916447.765575@montanaro.dyndns.org><Pine.LNX.4.58.0512271651040.6831@server1.LFW.org><doslos$kop$1@sea.gmane.org><17330.5976.560419.584425@montanaro.dyndns.org><2m7j9p1n2w.fsf@starship.python.net><B7EE1E00-6FC7-4404-AF12-90B8444BCD6F@gmail.com><61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de><e8a0972d0512281010t4853f3a8r936f277c28d83ab9@mail.gmail.com><61143.85.74.40.200.1135851316.squirrel@isar.livinglogic.de>
	<dp11vp$lv8$1@sea.gmane.org>
Message-ID: <dp12c7$ngv$1@sea.gmane.org>

Fernando Perez wrote:

> In [1]: x='hello'
>
> In [2]: x?
/.../
> Docstring:
>     str(object) -> string
>
>     Return a nice string representation of the object.
>     If the argument is a string, the return value is the same object.

I'm not sure what I find more confusing: a help system that claims that
the variable x returns a nice string representation of an object, or that
there's no help to be found for "hello".

>>> x = "hello"
>>> help(x)
no Python documentation found for 'hello'

</F>




From martin at v.loewis.de  Thu Dec 29 17:47:56 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 29 Dec 2005 17:47:56 +0100
Subject: [Python-Dev] Bug in Py_InitModule4
In-Reply-To: <43aa6ff70512290746v38d182e6n81ea3220a25ce0b6@mail.gmail.com>
References: <43aa6ff70512290746v38d182e6n81ea3220a25ce0b6@mail.gmail.com>
Message-ID: <43B4133C.60003@v.loewis.de>

Collin Winter wrote:
> While working with Subversion's python API bindings this morning, I
> discovered a function in one of their modules illegally named "import"
> (svn.client.import, for the curious). Because the extension module in
> question is written in C, the interpreter doesn't flag the
> otherwise-illegal identifier "import" at compile-time; if you try to
> call the function at runtime, however, the interpreter raises a
> SyntaxError, since svn.client.import is an illegal name.

There is still a way to call the function, though, through

import_ = svn.client.__dict__['import']
import_()

> My question is this: is there any interest in preventing situations
> like this by including checks in Python/modsupport.c:Py_InitModule4 to
> make sure that the PyMethodDef contains only valid identifiers, or is
> this a case of "if it hurts when you do that, don't do that"?

I cannot see a "complete" solution, i.e. one that works for all
types, all methods and attribute names of all types, and so on.
There might not even be a way to determine all identifiers that
some module wants to support, other than by inspecting the source
code.

Also, this is not a frequently-reported problem: if people find they
cannot call a certain method, they just rename the function and
be done. If nobody detects that a function is not callable, then
the function is possible just useless.

If this is a SWIG-generated module, I would see a bug in SWIG here:
SWIG should know what the Python keywords are, and rename all functions
following some pattern (traditionally, suffixing an underscore).

Regards,
Martin

From fperez.net at gmail.com  Thu Dec 29 17:50:19 2005
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 29 Dec 2005 09:50:19 -0700
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org><F00AC3F9-A81D-420F-8984-DF4D3D465577@mac.com><43B172C9.4070201@v.loewis.de><17329.38860.916447.765575@montanaro.dyndns.org><Pine.LNX.4.58.0512271651040.6831@server1.LFW.org><doslos$kop$1@sea.gmane.org><17330.5976.560419.584425@montanaro.dyndns.org><2m7j9p1n2w.fsf@starship.python.net><B7EE1E00-6FC7-4404-AF12-90B8444BCD6F@gmail.com><61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de><e8a0972d0512281010t4853f3a8r936f277c28d83ab9@mail.gmail.com><61143.85.74.40.200.1135851316.squirrel@isar.livinglogic.de>
	<dp11vp$lv8$1@sea.gmane.org> <dp12c7$ngv$1@sea.gmane.org>
Message-ID: <dp144c$tpj$1@sea.gmane.org>

Fredrik Lundh wrote:

> Fernando Perez wrote:
> 
>> In [1]: x='hello'
>>
>> In [2]: x?
> /.../
>> Docstring:
>>     str(object) -> string
>>
>>     Return a nice string representation of the object.
>>     If the argument is a string, the return value is the same object.
> 
> I'm not sure what I find more confusing: a help system that claims that
> the variable x returns a nice string representation of an object, or that
> there's no help to be found for "hello".

Then, complain about docstrings:

Python 2.3.4 (#1, Feb  2 2005, 12:11:53)
[GCC 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> x='hello'
>>> print x.__doc__
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.

====

In ipython, '?' does the best it can to collect information about an object,
making heavy use of python's introspection capabilities.  It provides class and
constructor docstrings, function call signatures (built from the function code
object), and more.  Using ?? gives even more details, including
syntax-highlighted source when available.  For example:

In [5]: pydoc.ErrorDuringImport??
Type:           classobj
String Form:    pydoc.ErrorDuringImport
Namespace:      Interactive
File:           /usr/lib/python2.3/pydoc.py
Source:
class ErrorDuringImport(Exception):
    """Errors that occurred while trying to import something to document it."""
    def __init__(self, filename, (exc, value, tb)):
        self.filename = filename
        self.exc = exc
        self.value = value
        self.tb = tb

    def __str__(self):
        exc = self.exc
        if type(exc) is types.ClassType:
            exc = exc.__name__
        return 'problem in %s - %s: %s' % (self.filename, exc, self.value)
Constructor information:
Definition:     pydoc.ErrorDuringImport(self, filename, (exc, value, tb))


I'm sorry it can't provide the information you'd like to see.  It is still found
to be useful by many people, including myself.  You are welcome to use it, and
patches to improve it will be well received.

Best,

f


From abo at minkirri.apana.org.au  Thu Dec 29 17:52:28 2005
From: abo at minkirri.apana.org.au (Donovan Baarda)
Date: Thu, 29 Dec 2005 16:52:28 +0000
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <dp127b$mt2$1@sea.gmane.org>
References: <001701c60c0a$74f11240$78fdcc97@oemcomputer>
	<1135855798.4203.15.camel@warna.dub.corp.google.com>
	<b348a0850512290414i3e7f307hf2a01a6dc64c27d9@mail.gmail.com>
	<dp127b$mt2$1@sea.gmane.org>
Message-ID: <1135875148.6089.35.camel@warna.dub.corp.google.com>

On Thu, 2005-12-29 at 17:17 +0100, Fredrik Lundh wrote:
> Noam Raphael wrote:
> 
> > I'm not saying that practically it must be used - I'm just saying that
> > it can't be called a heuristic, and that it doesn't involve any "fancy
> > overkill size hinting or history tracking". It actually means
> > something like this:
> > 1. If you want to insert and the table is full, resize the table to
> > twice the current size.
> > 2. If you delete and the number of elements turns out to be less than
> > a quarter of the size of the table, resize the table to half of the
> > current size.
> 
> sure sounds like a heuristic algorithm to me... (as in "not guaranteed to
> be optimal under all circumstances, even if it's probably quite good in all
> practical cases")
> 
> </F>

My problem with this heuristic is it doesn't work well for the
probably-fairly-common use-case of; fill it, empty it, fill it, empty
it, fill it...etc.

As Guido pointed out, if you do have a use-case where a container gets
very big, then shrinks and doesn't grow again, you can manually cleanup
by creating a new container and del'ing the old one. If the
implementation is changed to use this heuristic, there is no simple way
to avoid the re-allocs for this use-case... (don't empty, but fill with
None... ugh!).

My gut feeling is this heuristic will cause more pain than it would
gain...

-- 
Donovan Baarda <abo at minkirri.apana.org.au>
http://minkirri.apana.org.au/~abo/


From fredrik at pythonware.com  Thu Dec 29 17:57:40 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 29 Dec 2005 17:57:40 +0100
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>
	<dorgc0$ca9$1@sea.gmane.org><60ed19d40512270555q3773fd89xea1a3c5ffe3895a3@mail.gmail.com><56D816B1-E5D2-46D9-A9B7-C4D5FB8A58A9@mac.com><dorn8g$uhk$1@sea.gmane.org><ca471dc20512282227k5dade465g1914039c826403f9@mail.gmail.com><20051229143723.GA5154@panix.com>
	<ca471dc20512290735v20033812k6ba1d0f3b445152e@mail.gmail.com>
Message-ID: <dp14i5$ve8$1@sea.gmane.org>

Guido van Rossum wrote:

> Regarding the meme floating about the arrogance of Pythoneers:
> bloggers (pretty much by definition) are actually the most arrogant
> species; don't confuse "bloggers say" with "most people think".

Sure, but I'm not only talking about the mindless ranters here; it's also
people that back their opinions with at least a few examples -- including
exit (and its "it's good for you that you have to learn more than you want
at a time when you're not interested" defenders), replies to "what's wrong
with Python's floating point type" along the lines of "that you don't under-
stand how floating point works", and so on.

(fwiw, this meme has also appeared on comp.lang.python quite a few times
lately -- and no, I'm not confusing "comp.lang.python" with "most people"
either)

I do think it's a problem that Python advocates suffer from a "everything's
perfect all the time" attitude.  "it ain't broke because we say so".

I think it's a larger problem that Python developers suffer from the same
attitude; in reality, some things are carefully thought out and craftily im-
plemented, some things are engineering tradeoffs made at a certain time,
and some things are just accidents -- but python-dev will happily defend
the current solution with the same energy, no matter what it is.

</F>




From noamraph at gmail.com  Thu Dec 29 18:03:08 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Thu, 29 Dec 2005 19:03:08 +0200
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <dp127b$mt2$1@sea.gmane.org>
References: <001701c60c0a$74f11240$78fdcc97@oemcomputer>
	<1135855798.4203.15.camel@warna.dub.corp.google.com>
	<b348a0850512290414i3e7f307hf2a01a6dc64c27d9@mail.gmail.com>
	<dp127b$mt2$1@sea.gmane.org>
Message-ID: <b348a0850512290903s533e2ad9k8d5406f63a98f319@mail.gmail.com>

On 12/29/05, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Noam Raphael wrote:
>
> > I'm not saying that practically it must be used - I'm just saying that
> > it can't be called a heuristic, and that it doesn't involve any "fancy
> > overkill size hinting or history tracking". It actually means
> > something like this:
> > 1. If you want to insert and the table is full, resize the table to
> > twice the current size.
> > 2. If you delete and the number of elements turns out to be less than
> > a quarter of the size of the table, resize the table to half of the
> > current size.
>
> sure sounds like a heuristic algorithm to me... (as in "not guaranteed to
> be optimal under all circumstances, even if it's probably quite good in all
> practical cases")

I'm not saying it's optimal, but it is really amortized O(1) per
insert/delete. I looked up in "Introduction to Algorithms" for this,
and it has a complicated explanation. A simple explanation is that
after every resize the table is exactly half-full. Let's say it has n
elements and the table size is 2*n. To get to the next resize, you
have to do at least n/2 removals of elements, or n insertion of
elements. After that, you do a resize operation. In either case, you
do an O(n) resize operation after at least O(n) insertions/removals
which are O(1) operations. This means that the toal cost remains O(n)
per n simple operations, which you can say is O(1) per simple
operation.

I hope that if you read this slowly it makes sense...

Noam

From andrea at suse.de  Thu Dec 29 13:02:12 2005
From: andrea at suse.de (Andrea Arcangeli)
Date: Thu, 29 Dec 2005 13:02:12 +0100
Subject: [Python-Dev] suggestion for smarter garbage collection in
	function of size (gc.set_collect_mem_growth(2))
In-Reply-To: <44CD8598-644E-4C5A-9C11-77DFC120BA44@redivi.com>
References: <20051227140516.GB27200@opteron.random>
	<44CD8598-644E-4C5A-9C11-77DFC120BA44@redivi.com>
Message-ID: <20051229120212.GP27200@opteron.random>

On Thu, Dec 29, 2005 at 04:22:35AM -0500, Bob Ippolito wrote:
> In this particular case, you might be better off just writing some  
> Twisted code that periodically checks the size of the current process  
> and does a gc.collect() when necessary.  Of course, it requires some  
> platform specific code, but presumably you only care about one, maybe  
> two, platforms anyway.

In function of time != in function of size. The timer may trigger too
late. And anyway the point was to do it in autopilot mode, I already
fixed my app with a gc.collect() after releasing the huge piece of
memory. I'll try to write a testcase for it, that if python would be
doing what I suggest, wouldn't push a system into heavy swap.

From andrea at suse.de  Thu Dec 29 03:36:58 2005
From: andrea at suse.de (Andrea Arcangeli)
Date: Thu, 29 Dec 2005 03:36:58 +0100
Subject: [Python-Dev] suggestion for smarter garbage collection
	in	function of size (gc.set_collect_mem_growth(2))
In-Reply-To: <E1ErnJJ-0001zP-5p@cranky.arctrix.com>
References: <20051227140516.GB27200@opteron.random>
	<20051228135206.GA18974@panix.com>
	<20051228145723.GH27200@opteron.random>
	<43B31FA6.8030604@v.loewis.de>
	<E1ErnJJ-0001zP-5p@cranky.arctrix.com>
Message-ID: <20051229023657.GO27200@opteron.random>

On Wed, Dec 28, 2005 at 07:14:32PM -0700, Neil Schemenauer wrote:
> [This message has also been posted.]
> Martin v. L?wis <martin at v.loewis.de> wrote:
> > One challenge is that PyObject_GC_Del doesn't know how large the memory
> > block is that is being released. So it is difficult to find out how
> > much memory is being released in the collection.
> 
> Another idea would be to add accounting to the PyMem_* interfaces.
> It could be that most memory is used by objects that are not tracked
> by the GC (e.g. strings).  I guess you still have the same problem
> in that PyMem_Free may not know how large the memory block is.

In ram_size (per my pseudocode) we have to account everything that can
be possibly released by the "gc" by an inovcation of a deep gc.collect().
So if strings can't be freed by the gc (as a side effect of releasing
other objects), then we don't necessairly need to account for them in
the algorithm, otherwise we have to. I guess some strings can be freed
by the gc too so it sounds like PyMem_ may be a more correct hooking
point.

We definitely must know the size of the free operation. The sad thing is
that glibc knows it.

	size_t free_size(void * ptr) /* free and return size of freed object */

An API like the above would be able to answer our question at very
little cost, but it requires changing glibc, and we'd need to make sure
it's really the more efficient way of doing it before considering it,
because I've some doubt at the moment (otherwise I wonder why something
like the above doesn't already exist in glibc?!?). OTOH I guess not many
apps are doing their own garbage collection, and the ones that do it,
may be using their own allocators instead of the glibc ones. This
reminds me about the pymalloc thing I heard about over time. That should
be able to provide a pymalloc_free_size kind of thing returning the size
of the object freed, we could start with that assuming it's more
efficient than doing the accounting in the upper layer.

PS. your mail client looks broken the way it handles CC ;)

From arigo at tunes.org  Thu Dec 29 18:30:41 2005
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 29 Dec 2005 18:30:41 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43B3ECEE.9060300@v.loewis.de>
References: <43B3ECEE.9060300@v.loewis.de>
Message-ID: <20051229173041.GA19575@code1.codespeak.net>

Hi Martin,

On Thu, Dec 29, 2005 at 03:04:30PM +0100, "Martin v. L?wis" wrote:
> New conversion functions PyInt_FromSsize_t, PyInt_AsSsize_t,
> PyLong_AsSsize_t are introduced. PyInt_FromSsize_t will transparently
> return a long int object if the value exceeds the MAX_INT.

I guess you mean LONG_MAX instead of MAX_INT, in the event that ssize_t
is larger than a long.  Also, distinguishing between PyInt_AsSsize_t()
and PyLong_AsSsize_t() doesn't seem to be useful (a quick look in your
branch makes me guess that they both accept an int or a long object
anyway).

> The conversion codes 's#' and 't#' will output Py_ssize_t
> if the macro PY_SIZE_T_CLEAN is defined before Python.h
> is included, and continue to output int if that macro
> isn't defined.

Hum.  It would be much cleaner to introduce a new format character to
replace '#' and deprecate '#'...

> Compatibility with previous Python
> versions can be achieved with the test::
> 
>  #if PY_VERSION_HEX < 0x02050000
>  typedef int Py_ssize_t;
>  #endif
> 
> and then using Py_ssize_t in the rest of the code.

Nice trick :-)

As far as I can tell you have done as much as possible to ensure
compatibility, short of adding new slots duplicating the existing ones
with the new signature -- which would make abstract.c/typeobject.c a
complete nightmare.  I'm +1 on doing this in 2.5.


A bientot,

Armin

From jcarlson at uci.edu  Thu Dec 29 18:52:41 2005
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu, 29 Dec 2005 09:52:41 -0800
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <b348a0850512290903s533e2ad9k8d5406f63a98f319@mail.gmail.com>
References: <dp127b$mt2$1@sea.gmane.org>
	<b348a0850512290903s533e2ad9k8d5406f63a98f319@mail.gmail.com>
Message-ID: <20051229093449.BF0A.JCARLSON@uci.edu>


Noam Raphael <noamraph at gmail.com> wrote:
> On 12/29/05, Fredrik Lundh <fredrik at pythonware.com> wrote:
> > Noam Raphael wrote:
> >
> > > I'm not saying that practically it must be used - I'm just saying that
> > > it can't be called a heuristic, and that it doesn't involve any "fancy
> > > overkill size hinting or history tracking". It actually means
> > > something like this:
> > > 1. If you want to insert and the table is full, resize the table to
> > > twice the current size.
> > > 2. If you delete and the number of elements turns out to be less than
> > > a quarter of the size of the table, resize the table to half of the
> > > current size.
> >
> > sure sounds like a heuristic algorithm to me... (as in "not guaranteed to
> > be optimal under all circumstances, even if it's probably quite good in all
> > practical cases")
> 
> I'm not saying it's optimal, but it is really amortized O(1) per
> insert/delete. I looked up in "Introduction to Algorithms" for this,
> and it has a complicated explanation. A simple explanation is that
> after every resize the table is exactly half-full. Let's say it has n
> elements and the table size is 2*n. To get to the next resize, you
> have to do at least n/2 removals of elements, or n insertion of
> elements. After that, you do a resize operation. In either case, you
> do an O(n) resize operation after at least O(n) insertions/removals
> which are O(1) operations. This means that the toal cost remains O(n)
> per n simple operations, which you can say is O(1) per simple
> operation.
> 
> I hope that if you read this slowly it makes sense...

This is understood by (I would expect) most people here (hash-tables
are (theoretically and practically) average as you state, but
(theoretically) worst-case as Martin states).

For resizing, a quick-and-dirty rule of thumb is that if you are
overallocating by a factor of f(n), the amount of work you will be
performing per insertion is ~cn/f(n) (1 <= c <= 2).  As per recent
discussions on lists, Python chooses f(n) to be n/8. (at least in the
list case) This says that every insertion is taking around ~8 memory
copies if/when the list is resized up, but practical experience has
shown that it also tends to minimize memory usage as the list grows.

It's all about tradeoffs.  Increasing general memory usage for the sake
of a lower constant or not is a tradeoff.  As is resizing or not
resizing as a list gets smaller.

Would changing the overallocation strategy change Python's performance? 
Likely, but possibly not noticable.  Would it change Python memory usage?
Yes. A vast majority of list use would cause larger allocations than
currently performed by the list allocator.  Want to test it?  Create a
micro benchmark which tests repeated append performance with the two
list allocation strategies and remember to note the memory usage of
Python after each test.

 - Josiah


From guido at python.org  Thu Dec 29 19:48:38 2005
From: guido at python.org (Guido van Rossum)
Date: Thu, 29 Dec 2005 10:48:38 -0800
Subject: [Python-Dev] floating point literals don't work in non-US
	locale in 2.5
In-Reply-To: <dotrpj$o9o$1@sea.gmane.org>
References: <dotrpj$o9o$1@sea.gmane.org>
Message-ID: <ca471dc20512291048j73313375wc72b1c1dcc3cc5b3@mail.gmail.com>

Not the first time this happened. :-(

Could someone add a unit test for this please?

--Guido

On 12/28/05, Fredrik Lundh <fredrik at pythonware.com> wrote:
> someone recently broke floating point literals in a rather spectacular
> way:
>
> $ export LANG=sv_SE.utf8
> $ ./python
> Python 2.5a0 (41806M, Dec 25 2005, 12:12:29)
> Type "help", "copyright", "credits" or "license" for more information.
> >>> 3.14
> 3.1400000000000001
> >>> import locale
> >>> locale.setlocale(locale.LC_ALL, "")
> 'sv_SE.utf8'
> >>> 3.14
> 3.0
>
> more here:
>
> http://www.python.org/sf/1391872
>
> </F>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fredrik at pythonware.com  Thu Dec 29 19:54:11 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 29 Dec 2005 19:54:11 +0100
Subject: [Python-Dev] floating point literals don't work in non-USlocale
	in 2.5
References: <dotrpj$o9o$1@sea.gmane.org>
	<ca471dc20512291048j73313375wc72b1c1dcc3cc5b3@mail.gmail.com>
Message-ID: <dp1bcl$sl3$1@sea.gmane.org>

Guido wrote:

> Not the first time this happened. :-(
>
> Could someone add a unit test for this please?

Hye-Shik Chang just added the necessary tests to his bugfix patch.  I'll
check this in later tonight.

</F>




From mcherm at mcherm.com  Thu Dec 29 20:18:20 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu, 29 Dec 2005 11:18:20 -0800
Subject: [Python-Dev] a quit that actually quits
Message-ID: <20051229111820.pncqj7pozj7oks84@login.werra.lunarpages.com>

The F-bot writes:
> in reality, some things are carefully thought out and craftily im-
> plemented, some things are engineering tradeoffs made at a certain time,
> and some things are just accidents -- but python-dev will happily defend
> the current solution with the same energy, no matter what it is.

+1 QOTF.

Seriously... I've seen this behavior also, but I haven't ever thought
about it as clearly as Fredrik does here. When we go to answer questions
we ought to pause briefly first and decide which of these categories
applies to a given piece of behavior. I think users will be understanding
if we're honest about what are the accidents -- every language or
software package has some, and just because it's an accident does NOT
mean we should "fix" it.

-- Michael Chermside

From martin at v.loewis.de  Thu Dec 29 20:25:06 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 29 Dec 2005 20:25:06 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <20051229111820.pncqj7pozj7oks84@login.werra.lunarpages.com>
References: <20051229111820.pncqj7pozj7oks84@login.werra.lunarpages.com>
Message-ID: <43B43812.9030505@v.loewis.de>

Michael Chermside wrote:
> Seriously... I've seen this behavior also, but I haven't ever thought
> about it as clearly as Fredrik does here. When we go to answer questions
> we ought to pause briefly first and decide which of these categories
> applies to a given piece of behavior. I think users will be understanding
> if we're honest about what are the accidents -- every language or
> software package has some, and just because it's an accident does NOT
> mean we should "fix" it.

But we do that (atleast I do): when I believe something is wrong,
I don't argue it is right; instead, I ask for a contribution of fixes.
I do argue it is right when I believe it is right.

Regards,
Martin

From pedronis at strakt.com  Thu Dec 29 20:45:19 2005
From: pedronis at strakt.com (Samuele Pedroni)
Date: Thu, 29 Dec 2005 20:45:19 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <20051229111820.pncqj7pozj7oks84@login.werra.lunarpages.com>
References: <20051229111820.pncqj7pozj7oks84@login.werra.lunarpages.com>
Message-ID: <43B43CCF.8050807@strakt.com>

Michael Chermside wrote:
> The F-bot writes:
> 
>>in reality, some things are carefully thought out and craftily im-
>>plemented, some things are engineering tradeoffs made at a certain time,
>>and some things are just accidents -- but python-dev will happily defend
>>the current solution with the same energy, no matter what it is.
> 
> 
> +1 QOTF.
> 
> Seriously... I've seen this behavior also, but I haven't ever thought
> about it as clearly as Fredrik does here. When we go to answer questions
> we ought to pause briefly first and decide which of these categories
> applies to a given piece of behavior. I think users will be understanding
> if we're honest about what are the accidents -- every language or
> software package has some, and just because it's an accident does NOT
> mean we should "fix" it.
> 

it's indeed a matter of trade-offs. Converting NameErrors into commands
doesn't look like a good trade off in terms of expectations management
and understandable behavior. Ka-Ping Yee ExitHatch still seem a 
reasonable improvement. Fernando Perez considerations about Python
vs. "commands" and prefixing and extra-linguistic extensions seem
also on spot. It's not a matter of defending the status quo, more about
what kind of price is reasonable for DWIM.

From goodger at python.org  Thu Dec 29 21:38:15 2005
From: goodger at python.org (David Goodger)
Date: Thu, 29 Dec 2005 15:38:15 -0500
Subject: [Python-Dev] [Doc-SIG] that library reference, again
In-Reply-To: <dp0qrh$r78$1@sea.gmane.org>
References: <dope79$flv$2@sea.gmane.org> <dp0qrh$r78$1@sea.gmane.org>
Message-ID: <4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com>

On 12/29/05, Fredrik Lundh <fredrik at pythonware.com> wrote:
> however, given that the discussion that led up to this has been dead for
> almost a week,

You mean since Christmas?

> I'm beginning to fear that I've wasted my time on a project
> that nobody's interested in.

Could be. I don't see HTML+PythonDoc as a significant improvement
over LaTeX.

Yes, I'm biased. So are you.

> are we stuck with latex for the foreseeable future?

Until we have something clearly and significantly better, yes.

--
David Goodger <http://python.net/~goodger>

From fredrik at pythonware.com  Thu Dec 29 22:11:02 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 29 Dec 2005 22:11:02 +0100
Subject: [Python-Dev] [Doc-SIG] that library reference, again
References: <dope79$flv$2@sea.gmane.org> <dp0qrh$r78$1@sea.gmane.org>
	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com>
Message-ID: <dp1jd8$p4a$1@sea.gmane.org>

David Goodger wrote:

> > however, given that the discussion that led up to this has been dead for
> > almost a week,
>
> You mean since Christmas?
>
> > I'm beginning to fear that I've wasted my time on a project
> > that nobody's interested in.
>
> Could be. I don't see HTML+PythonDoc as a significant improvement
> over LaTeX.

Really?  Have you read my list of goals?  Does LaTeX address all of them?
Does ReST?  If not, can you explain why they're not important.

> Yes, I'm biased. So are you.

I don't think you understand the problem, so your bias is irrelevant.  This is
all about semantics, not presentation markup.  All I've seen from the ReST
camp is handwaving, there's nothing in the documentation that indicates that
semantic markup has ever been a design goal in ReST, and nobody I've talked
to who've tried using ReST for rich semantic markup considers it to be an
alternative.  This isn't about bias, this is about technical suitability.

If you think otherwise, I suggest you put pick a couple of typical document
pages and show how they would look in ReST, how the corresponding semantic
model will look (and when I say semantic, I mean in Python terms, not in ReST
presentation terms), and how to get from LaTeX to ReST+semantics and HTML+
semantics without having to rewrite everything from scratch.

We know that you have big hats over in ReST-land; now show us some cattle.

</F>




From Scott.Daniels at Acm.Org  Thu Dec 29 22:12:11 2005
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Thu, 29 Dec 2005 13:12:11 -0800
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <20051229051008.GA2459@panix.com>
References: <dor982$qrc$1@sea.gmane.org>	<61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>	<doul02$8ja$1@sea.gmane.org>
	<doupvk$oa4$1@sea.gmane.org>	<43B355A3.5050109@gmail.com>
	<20051229044024.GA15414@panix.com>	<bbaeab100512282106k2bd8f019labfd4ba70c779360@mail.gmail.com>
	<20051229051008.GA2459@panix.com>
Message-ID: <dp1jet$p9m$1@sea.gmane.org>

Aahz wrote:
> On Wed, Dec 28, 2005, Brett Cannon wrote:
>> On 12/28/05, Aahz <aahz at pythoncraft.com> wrote:
>>> Here's yet a different take on this: .. change the startup message...
>>>     Type "help", "copyright", "credits" or "license" for more information.
>>> Let's add another line that says
>>>     Type "quit()" to exit
>>> ...

Or, perhaps:
     class _Quitter(str):
         def __call__(self): raise SystemExit
     quit = _Quitter('The quit command.  Type "quit()" to exit')
     exit = _Quitter('The exit command.  Type "exit()" to exit')

--Scott David Daniels
Scott.Daniels at Acm.Org


From tim.peters at gmail.com  Thu Dec 29 22:34:45 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 29 Dec 2005 16:34:45 -0500
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43B3ECEE.9060300@v.loewis.de>
References: <43B3ECEE.9060300@v.loewis.de>
Message-ID: <1f7befae0512291334r77fa03cx62e1e43dce757213@mail.gmail.com>

[Martin v. L?wis]
...
> PEP: XXX
> Title: Using ssize_t as the index type

+1, and for Python 2.5, and the sooner done the less painful for all. 
Same concerns as Armin, where this is especially unattractive:

> The conversion codes 's#' and 't#' will output Py_ssize_t
> if the macro PY_SIZE_T_CLEAN is defined before Python.h
> is included, and continue to output int if that macro
> isn't defined.

Better to use a new gibberish character and deprecate the old one? 
Otherwise I'm afraid we'll be stuck supporting PY_SIZE_T_CLEAN
forever, and it's not good to have the meaning of a format string
depend on the presence or absence of a #define far away in the file.

A suggestion:

...

> In these cases, three strategies are available:
>
> *  statically determine that the size can never exceed an int
>    (e.g. when taking the sizeof a struct, or the strlen of
>    a file pathname). In this case, add a debug assert() that
>    the value is indeed smaller than INT_MAX, and cast the
>    value to int.

That can be done in one gulp via:

    some_int = Py_SAFE_DOWNCAST(some_value, Py_ssize_t, int);

In a debug build that will assert-fail if some_value loses info when
cast from Py_ssize_t to int.  If people had been careful, this would
already be in use when casting from size_t to int;  there's even one
place in unicodeobject.c that does so ;-).

From bcannon at gmail.com  Thu Dec 29 22:59:23 2005
From: bcannon at gmail.com (Brett Cannon)
Date: Thu, 29 Dec 2005 13:59:23 -0800
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43B3ECEE.9060300@v.loewis.de>
References: <43B3ECEE.9060300@v.loewis.de>
Message-ID: <bbaeab100512291359u4f791b0cx974d2d826a4fd330@mail.gmail.com>

On 12/29/05, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Please let me know what you think.
>
> Regards,
> Martin
>
> PEP: XXX
> Title: Using ssize_t as the index type
[SNIP]

+1 from me.  As everyone else is saying, this has to happen at some
point and 2.5 is as good as any.

-Brett

From fredrik at pythonware.com  Thu Dec 29 23:22:32 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 29 Dec 2005 23:22:32 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
References: <43B3ECEE.9060300@v.loewis.de>
Message-ID: <dp1nja$6gk$1@sea.gmane.org>

Martin v. Löwis wrote:

> Please let me know what you think.

+1.  in honor of the Bike Shed Effect, I'm going to assume that you've
thought of everything.

:::

well, one thing seems to missing from your PEP: in several modules, you've
changed the cast used in the type table. e.g.

--- python/branches/ssize_t/Modules/_elementtree.c (original)
+++ python/branches/ssize_t/Modules/_elementtree.c Tue Dec 20 09:52:16 2005
@@ -1228,7 +1228,7 @@
 }

 static int
-element_setitem(ElementObject* self, size_t index, PyObject* item)
+element_setitem(ElementObject* self, Py_ssize_t index, PyObject* item)
 {
     int i;
     PyObject* old;
@@ -1373,7 +1373,7 @@
     0, /* sq_repeat */
     (ssizeargfunc) element_getitem,
     (ssizessizeargfunc) element_getslice,
-    (sizeobjargproc) element_setitem,
+    (ssizeobjargproc) element_setitem,
     (ssizessizeobjargproc) element_setslice,
 };

is this change backwards compatible ?

</F>




From nas at arctrix.com  Fri Dec 30 02:38:57 2005
From: nas at arctrix.com (Neil Schemenauer)
Date: Fri, 30 Dec 2005 01:38:57 +0000 (UTC)
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>
	<61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>
	<doul02$8ja$1@sea.gmane.org> <doupvk$oa4$1@sea.gmane.org>
	<43B355A3.5050109@gmail.com> <20051229044024.GA15414@panix.com>
	<bbaeab100512282106k2bd8f019labfd4ba70c779360@mail.gmail.com>
	<20051229051008.GA2459@panix.com> <dp1jet$p9m$1@sea.gmane.org>
Message-ID: <dp233h$4cr$1@sea.gmane.org>

Scott David Daniels <Scott.Daniels at Acm.Org> wrote:
> Or, perhaps:
>      class _Quitter(str):
>          def __call__(self): raise SystemExit
>      quit = _Quitter('The quit command.  Type "quit()" to exit')
>      exit = _Quitter('The exit command.  Type "exit()" to exit')

FWIW, I like this kind of solution best.  Something magical would be
a mistake.  I don't like the status quo because there is no
cross-plaform way to indicate EOF (or more pedantically "push
current line").  Maybe we can make everyone happy by making the
'quit' and 'exit' objects callable and changing the message to
something like:

    Use quit() or Ctrl-D (i.e. EOF) to exit.

Cheers,

  Neil


From ncoghlan at gmail.com  Fri Dec 30 03:45:03 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 30 Dec 2005 12:45:03 +1000
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <dp233h$4cr$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org>	<61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>	<doul02$8ja$1@sea.gmane.org>
	<doupvk$oa4$1@sea.gmane.org>	<43B355A3.5050109@gmail.com>
	<20051229044024.GA15414@panix.com>	<bbaeab100512282106k2bd8f019labfd4ba70c779360@mail.gmail.com>	<20051229051008.GA2459@panix.com>
	<dp1jet$p9m$1@sea.gmane.org> <dp233h$4cr$1@sea.gmane.org>
Message-ID: <43B49F2F.5020906@gmail.com>

Neil Schemenauer wrote:
> Scott David Daniels <Scott.Daniels at Acm.Org> wrote:
>> Or, perhaps:
>>      class _Quitter(str):
>>          def __call__(self): raise SystemExit
>>      quit = _Quitter('The quit command.  Type "quit()" to exit')
>>      exit = _Quitter('The exit command.  Type "exit()" to exit')
> 
> FWIW, I like this kind of solution best.  Something magical would be
> a mistake.  I don't like the status quo because there is no
> cross-plaform way to indicate EOF (or more pedantically "push
> current line").  Maybe we can make everyone happy by making the
> 'quit' and 'exit' objects callable and changing the message to
> something like:
> 
>     Use quit() or Ctrl-D (i.e. EOF) to exit.
> 
> Cheers,

This sounds pretty good actually (especially combined with the modifed startup 
banner which tells you how to exit).

As Fernando pointed out, anything else means we'd be well on our way to 
re-inventing IPython (although I'd be interested to know if sys.inputhook 
would have made IPython easier to write).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From goodger at python.org  Fri Dec 30 03:58:00 2005
From: goodger at python.org (David Goodger)
Date: Thu, 29 Dec 2005 21:58:00 -0500
Subject: [Python-Dev] [Doc-SIG] that library reference, again
In-Reply-To: <dp1jd8$p4a$1@sea.gmane.org>
References: <dope79$flv$2@sea.gmane.org>
	<dp0qrh$r78$1@sea.gmane.org>	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com>
	<dp1jd8$p4a$1@sea.gmane.org>
Message-ID: <43B4A238.2030101@python.org>

[Fredrik Lundh]
>>> I'm beginning to fear that I've wasted my time on a project
>>> that nobody's interested in.

[David Goodger]
>> Could be. I don't see HTML+PythonDoc as a significant improvement
>> over LaTeX.

[Fredrik Lundh]
> Really?

Yes, really.

> Have you read my list of goals?

Yes, and I mostly agree with most of them.  One is worded in a very
slanted way:

    * Build on existing non-Python-specific documentation standards
      and tools, where possible (basic html, javadoc, xhtml modules,
      etc).

It seems that this goal is specifically written to exclude ReST.
Javadoc is only a standard in the Java world.  LaTeX *is* an
existing language-independent standard, and has a much longer
history than javadoc.

I'd re-write the above goal as:

    * Build on existing documentation standards and tools, where
      possible.

Another goal is highly biased toward XML-style markup:

    * Make it trivial to do basic rendering to HTML (a few re.sub
      calls should suffice), to enable simple tools (CGI scripts, etc)
      to be able to render reference documentation.

This is the tail wagging the dog.

This item is under-specified:

    * Make is easy to do advanced rendering to HTML, XHTML or XML
      models (e.g. typographic issues, navigation, dynamic styling,
      etc.). No more -- dashes and silly ``quotes''.

The second sentence lacks a rationale.  What's wrong with "-- dashes"?
What's "silly" about "``quotes''"?

Your list is missing an important goal:

    * Easy to read.

The final paragraph of the "Project Goals" section lacks context and
contains a false statement:

    The semantic model should be simple, python-oriented, and more
    detailed than today.  For example, while a ReST-based solution
    knows that open in a given context should be rendered in bold, and
    a LaTeX-based solution knows that the given open is a function,
    the PythonDoc model also knows that it refers to the os.open
    function, rather than the built-in open function.

The reference to ReST is wrong here; ReST certainly can know the
semantics of a given identifier.  I'd like to see an example of how
the HTML+PythonDoc markup indicates that a particular "open" is
os.open and not __builtins__.open.

> Does LaTeX address all of them?

Perhaps not.  So?  It works well enough.

> Does ReST?

Not currently, but so what?  I didn't propose ReST as an alternative
to LaTeX for Python's documentation.  All I'm saying is that the
proposed HTML+PythonDoc markup is not significantly better than the
status quo.  I don't think there's enough benefit to make leaving
LaTeX worthwhile.

IOW: -1 on replacing LaTeX with HTML+PythonDoc.

> If not, can you explain why they're not important.
>
>> Yes, I'm biased. So are you.
>
> I don't think you understand the problem, so your bias is
> irrelevant.  This is all about semantics, not presentation markup.

I don't think you understand ReST except superficially.  In any case,
ReST is irrelevant to this discussion.  I do not consider ReST a
suitable replacement for LaTeX in Python's docs at present.

My bias is simple: I am against wasting the time and effort required
to replace one form of verbose markup with a different but equivalent
form.  The proposed solution is no better than the status quo.  I
agree with the people who have stated that they find the direct
writing of HTML painful.

IMO, the markup itself is almost irrelevant.  As others have stated,
writing good documentation is hard.  People will latch on to any
excuse to avoid it, and markup is convenient.  But even if the docs
had no markup whatsoever, if they were written in plain text like
RFCs, I doubt there would be significantly more patch contributions.
Plain text patches are already accepted; perhaps this fact needs more
emphasis, but that's all.

> We know that you have big hats over in ReST-land; now show us some
> cattle.

Moo.  Or would you prefer the Buddhist "mu"?
What *are* you talking about?

-- 
David Goodger <http://python.net/~goodger>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 249 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20051229/6adae0ba/signature-0001.pgp

From ncoghlan at gmail.com  Fri Dec 30 04:05:26 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 30 Dec 2005 13:05:26 +1000
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <43B43CCF.8050807@strakt.com>
References: <20051229111820.pncqj7pozj7oks84@login.werra.lunarpages.com>
	<43B43CCF.8050807@strakt.com>
Message-ID: <43B4A3F6.5040602@gmail.com>

Samuele Pedroni wrote:
> Michael Chermside wrote:
>> The F-bot writes:
>>
>>> in reality, some things are carefully thought out and craftily im-
>>> plemented, some things are engineering tradeoffs made at a certain time,
>>> and some things are just accidents -- but python-dev will happily defend
>>> the current solution with the same energy, no matter what it is.
>>
>> +1 QOTF.
>>
>> Seriously... I've seen this behavior also, but I haven't ever thought
>> about it as clearly as Fredrik does here. When we go to answer questions
>> we ought to pause briefly first and decide which of these categories
>> applies to a given piece of behavior. I think users will be understanding
>> if we're honest about what are the accidents -- every language or
>> software package has some, and just because it's an accident does NOT
>> mean we should "fix" it.

Most of the times I've asked questions along these lines I've gotten 
well-considered answers (usually because something I thought was a deliberate 
design decision on Guido's part turned out to simply be an accident of the way 
it happened to be implemented in CPython).

> it's indeed a matter of trade-offs. Converting NameErrors into commands
> doesn't look like a good trade off in terms of expectations management
> and understandable behavior. Ka-Ping Yee ExitHatch still seem a 
> reasonable improvement. Fernando Perez considerations about Python
> vs. "commands" and prefixing and extra-linguistic extensions seem
> also on spot. It's not a matter of defending the status quo, more about
> what kind of price is reasonable for DWIM.

I think Fredrik has made an excellent case for promoting the quit & exit 
interpreter-only builtins to be proper callables.

Hell, we even have that capability for the callable that displays the 
*license* text. . . surely quitting the interpreter is far more important than 
being able to display the license text, but the support for the latter is 
significantly better:


Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Py> exit
'Use Ctrl-Z plus Return to exit.'
Py> quit
'Use Ctrl-Z plus Return to exit.'
Py> license
Type license() to see the full license text
Py> type(license)
<class 'site._Printer'>

Counting blank lines, 60 lines of site.py are devoted to getting copyright, 
credit and license to work right, 16 to getting help to work, and only 12 to 
setting quit and exit to the 'right' strings - and due to modules like 
readline for Windows and differences in the way interpreters buffer the line 
input, the exit string for Windows is not always correct.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From python-dev at zesty.ca  Fri Dec 30 05:29:21 2005
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Thu, 29 Dec 2005 22:29:21 -0600 (CST)
Subject: [Python-Dev] Naming conventions in Py3K
Message-ID: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>

In a fair number of cases, Python doesn't follow its own recommended
naming conventions.  Changing these things would break backward
compatibility, so they are out of the question for Python 2.*, but
it would be nice to keep these in mind for Python 3K.

    Constants in all caps:
        NONE, TRUE, FALSE, ELLIPSIS

    Classes in initial-caps:
        Object, Int, Float, Str, Unicode, Set, List, Tuple, Dict,
        and lots of classes in the standard library, e.g.
        anydbm.error, csv.excel, imaplib.error, mutex.mutex...

I know these probably look a little funny now to most of us, as
we're used to looking at today's Python (they even look a little
funny to me) but i'm pretty convinced that consistency will be
better in the long run.


-- ?!ng

From stephen at xemacs.org  Fri Dec 30 06:14:31 2005
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 30 Dec 2005 14:14:31 +0900
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <43B4A3F6.5040602@gmail.com> (Nick Coghlan's message of "Fri,
	30 Dec 2005 13:05:26 +1000")
References: <20051229111820.pncqj7pozj7oks84@login.werra.lunarpages.com>
	<43B43CCF.8050807@strakt.com> <43B4A3F6.5040602@gmail.com>
Message-ID: <87wthntbd4.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Nick" == Nick Coghlan <ncoghlan at gmail.com> writes:

    Nick> Samuele Pedroni wrote:

    >> It's not a matter of defending the status quo, more about what
    >> kind of price is reasonable for DWIM.

IMHO, +N*10^6 for simplicity, regularity, and discoverability, -1 for
DWIM in the interpreter.  DWIM is for wrappers, interactive tutorials,
and IDEs.

    Nick> I think Fredrik has made an excellent case for promoting the
    Nick> quit & exit interpreter-only builtins to be proper
    Nick> callables.

No, Fredrik has made a good (but not good enough!) case for making
them syntax (or for adding the concept of "interpreter command" to the
specification), but your own example of "license" works against you
for callables:

    Nick> Hell, we even have that capability for the callable that
    Nick> displays the *license* text. . . surely quitting the
    Nick> interpreter is far more important than being able to display
    Nick> the license text, but the support for the latter is
    Nick> significantly better:

  Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32
  Type "help", "copyright", "credits" or "license" for more information.
  Py> license
  Type license() to see the full license text
  Py>

Now, unlike the case of quit, where the user did something
undocumented (albeit "natural") that really is rude (and its semantics
are no "better" than the support for quit).  (Example edited to
enhance the effect.)  I do think it's reasonable that those callables
be somewhat python-newbie-friendly.  What I would want to see is

papa% python
Python 2.4.2 (#1, Dec 23 2005, 01:55:50) 
[GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin
Type "help()", "copyright()", "credits()" or "license()" for more information.
Type EOF (Ctrl-D) to end the interpreter session.
>>> help
Type "help()".
In Python, work is done by function calls, not statements or commands.
This message is the printable representation of the help object.
>>> help()
You may exit the interpreter with an EOF (Ctrl-D), or by calling a system
function or raising an appropriate exception.
[ ... etc, etc, ... ]
>>> 

copyright, credits, license, quit, and exit would be treated similarly
(except maybe they should not be quite so "educational," just the
brief reminder 'Type "<callable>()".  Type "help()" for help.' should
be enough).  I definitely don't think help() should advertise exit(),
as it is too similar to sys.exit().  I'm -1 on quit() being
advertised because I'm -1 on DWIM in principle, but if DWIM is
accepted for this purpose, I don't see practical harm in advertising it.

IMO, in the end making "quit" callable really isn't responsive to
Fredrik's point.  Which is AIUI that most interactive shells do have a
quit command, and newbies are going to expect Python to have one, too.

-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From fperez.net at gmail.com  Fri Dec 30 06:53:50 2005
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 29 Dec 2005 22:53:50 -0700
Subject: [Python-Dev] a quit that actually quits
References: <dor982$qrc$1@sea.gmane.org>	<61328.85.74.5.147.1135789495.squirrel@isar.livinglogic.de>	<doul02$8ja$1@sea.gmane.org>
	<doupvk$oa4$1@sea.gmane.org>	<43B355A3.5050109@gmail.com>
	<20051229044024.GA15414@panix.com>	<bbaeab100512282106k2bd8f019labfd4ba70c779360@mail.gmail.com>	<20051229051008.GA2459@panix.com>
	<dp1jet$p9m$1@sea.gmane.org> <dp233h$4cr$1@sea.gmane.org>
	<43B49F2F.5020906@gmail.com>
Message-ID: <dp2i1f$klj$1@sea.gmane.org>

Nick Coghlan wrote:

> As Fernando pointed out, anything else means we'd be well on our way to
> re-inventing IPython (although I'd be interested to know if sys.inputhook
> would have made IPython easier to write).

[sorry if this drifts off-topic for python-dev.  I'll try to provide useful info
on interactive computing in python here, and will gladly answer further
detailed discussion about ipython on the ipython-dev/user lists ]

In my case, I don't think it would have made that much of a difference in the
end, though initially it might have been tempting to use it.  IPython started
as my private collection of sys.ps{1,2} + sys.displayhook hacks in
$PYTHONSTARTUP.  I then discovered LazyPython, which had a great
sys.excepthook, and IPP, which was a full-blown derivative of
code.InteractiveConsole.  I decided to join all three projects, and thus was
ipython born.  Given that IPP was the 'architecture', from the moment we had
what is still today's ipython, it was based on code.InteractiveConsole, and at
that point I doubt that having sys.inputhook would have mattered.

Incidentally, just two days ago I removed the last connection to code.py: at
this point I had overridden so many methods, that there was no point in keeping
the inheritance relationship.  All I had to do was copy _two_ remaining
methods, and the main ipython class became standalone (this frees me for
ongoing redesign work, so it was worth doing it).

So in summary, while sys.inputhook would make it easy to do _lightweight_
interactive customizations, if you really want a more sophisticated and
featureful system, it probably won't matter.

Note that this is not an argument against sys.inputhook: exposing
customizability here may indeed be useful.  This will allow people to write,
with minimal effort, systems which pre-process special syntaxes and ride on top
of the python engine.  IPython exposes the exact same thing as a customizable
hook (called prefilter), and people have made some excellent use of this
capability.  The most notable one is SAGE:

http://modular.ucsd.edu/sage

a system for interactive mathematical computing (NSF funded).  If anyone is in
the San Diego area, the first SAGE meeting is in February:

http://modular.ucsd.edu/sage/days1/

and I'll be giving a talk there about ipython, including some of its design and
my plans for a more ambitious system for interactive computing (including
distributed computing) based on Python.  The prototypes of what we've done so
far are briefly described here (the first was funded by Google as a summer of
code project):

http://ipython.scipy.org/misc/ipython-notebooks-scipy05.pdf
http://ipython.scipy.org/misc/scipy05-parallel.pdf

I hope this is of some use and interest.

Regards,

f


From brett at python.org  Fri Dec 30 07:33:49 2005
From: brett at python.org (Brett Cannon)
Date: Thu, 29 Dec 2005 22:33:49 -0800
Subject: [Python-Dev] Naming conventions in Py3K
In-Reply-To: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
References: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
Message-ID: <bbaeab100512292233p4afaec3bme4350042a9ee5c48@mail.gmail.com>

On 12/29/05, Ka-Ping Yee <python-dev at zesty.ca> wrote:
> In a fair number of cases, Python doesn't follow its own recommended
> naming conventions.  Changing these things would break backward
> compatibility, so they are out of the question for Python 2.*, but
> it would be nice to keep these in mind for Python 3K.
>
>     Constants in all caps:
>         NONE, TRUE, FALSE, ELLIPSIS
>
>     Classes in initial-caps:
>         Object, Int, Float, Str, Unicode, Set, List, Tuple, Dict,
>         and lots of classes in the standard library, e.g.
>         anydbm.error, csv.excel, imaplib.error, mutex.mutex...
>
> I know these probably look a little funny now to most of us,

Oh yeah.  =)

> as
> we're used to looking at today's Python (they even look a little
> funny to me) but i'm pretty convinced that consistency will be
> better in the long run.
>

Well, the problem is obviously requiring existing Python coders to
have to re-educate themselves and the amount of code breakage. 
Luckily stuff like this should be changeable by some script that can
try to convert 2.x code to 3.0 code.

I am fine with changing the built-in types, but changing the built-in
singletons just looks *really* odd to me.  So odd that I just don't
want to see them changed.  I mean when I think of constants, I think
of a variable that references an object and that reference never
changes.  The built-ins you are referencing, though, are singletons:
named objects that are not variables.  So keeping their naming scheme
as-is does not feel like a breaking of the rules to me since they are
not treated the same (especially at the C level).

-Brett

From robey at lag.net  Fri Dec 30 07:36:40 2005
From: robey at lag.net (Robey Pointer)
Date: Thu, 29 Dec 2005 22:36:40 -0800
Subject: [Python-Dev] [Doc-SIG] that library reference, again
In-Reply-To: <43B4A238.2030101@python.org>
References: <dope79$flv$2@sea.gmane.org>
	<dp0qrh$r78$1@sea.gmane.org>	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com>
	<dp1jd8$p4a$1@sea.gmane.org> <43B4A238.2030101@python.org>
Message-ID: <51A7717B-3832-477E-BED5-3C36DCA20336@lag.net>


On 29 Dec 2005, at 18:58, David Goodger wrote:

> [Fredrik Lundh]
>>>> I'm beginning to fear that I've wasted my time on a project
>>>> that nobody's interested in.
>
> [David Goodger]
>>> Could be. I don't see HTML+PythonDoc as a significant improvement
>>> over LaTeX.
>
> [Fredrik Lundh]
>> Really?
>
> Yes, really.

Just out of curiosity (really -- not trying to jump into the flames)  
why not just use epydoc?  If it's good enough for 3rd-party python  
libraries, isn't that just a small step from being good enough for  
the builtin libraries?

robey


From fredrik at pythonware.com  Fri Dec 30 08:13:18 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri, 30 Dec 2005 08:13:18 +0100
Subject: [Python-Dev] [Doc-SIG] that library reference, again
References: <dope79$flv$2@sea.gmane.org><dp0qrh$r78$1@sea.gmane.org>	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com><dp1jd8$p4a$1@sea.gmane.org>
	<43B4A238.2030101@python.org>
	<51A7717B-3832-477E-BED5-3C36DCA20336@lag.net>
Message-ID: <dp2mmh$45b$1@sea.gmane.org>

Robey Pointer wrote:
> > [Fredrik Lundh]
> >> Really?
> >
> > Yes, really.
>
> Just out of curiosity (really -- not trying to jump into the flames)
> why not just use epydoc?  If it's good enough for 3rd-party python
> libraries, isn't that just a small step from being good enough for
> the builtin libraries?

but epydoc is a docstring-based format, right?

I'm trying to put together a light-weight alternative to the markup
used for, primarily, the current library reference.

moving all of (or parts of) the reference documentation in to the
library source code would be an alternative, of course [1], but that
would basically mean starting over from scratch.

(but tighter integration is on my list; with a better semantic markup,
you can "import" reference documentation into a module's docstrings
at runtime, the builtin help can point you directly to the documentation
for a given class, etc.).

</F>

1) I'm using a mix of wiki-based introductions and machine generated
markup for e.g. the Tkinter manual; an example:

         http://effbot.org/tkinterbook/listbox.htm




From rwgk at yahoo.com  Fri Dec 30 08:29:02 2005
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Thu, 29 Dec 2005 23:29:02 -0800 (PST)
Subject: [Python-Dev] Python + Visual C++ 8.0?
In-Reply-To: <43B33B88.40501@v.loewis.de>
Message-ID: <20051230072902.20248.qmail@web31513.mail.mud.yahoo.com>

--- "Martin v. L?wis" <martin at v.loewis.de> wrote:

> Out of curiosity: can you please try invoking this enum_print to stdout
> with your VS2005-built boost module? I see it uses fprintf, so I would
> expect it to crash.

After beating on this for half an hour I am coming to the conclusion that there
is no way the enum_print function is invoked. No matter what I try, Python
always calls enum_str if available. If I set the enum_str slot to 0, it calls
enum_repr. If I set that slot to 0 as well it calls the tp_str method of
PyInt_Type, since the Boost.Python enum type inherits from it.

I have no clue how I can crash the VC7/VC8 mix doing "normal" things.

Cheers,
        Ralf



		
__________________________________________ 
Yahoo! DSL ? Something to write home about. 
Just $16.99/mo. or less. 
dsl.yahoo.com 


From fredrik at pythonware.com  Fri Dec 30 09:25:09 2005
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri, 30 Dec 2005 09:25:09 +0100
Subject: [Python-Dev] [Doc-SIG] that library reference, again
References: <dope79$flv$2@sea.gmane.org><dp0qrh$r78$1@sea.gmane.org>	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com><dp1jd8$p4a$1@sea.gmane.org>
	<43B4A238.2030101@python.org>
Message-ID: <dp2qt7$dib$1@sea.gmane.org>

> [David Goodger]
> >> Could be. I don't see HTML+PythonDoc as a significant improvement
> >> over LaTeX.
>
> [Fredrik Lundh]
> > Really?
>
> Yes, really.

Your reply makes it obvious that you don't understand the issues involved
here, nor how the goals address them.

(Snipping heavily below due to lack of time; if you want some other argument
refuted, you have to repost and wait until next week.)

> Another goal is highly biased toward XML-style markup:
>
>     * Make it trivial to do basic rendering to HTML (a few re.sub
>       calls should suffice), to enable simple tools (CGI scripts, etc)
>       to be able to render reference documentation.
>
> This is the tail wagging the dog.

No, it's a fundamental goal: to support light-weight generation of rendered
markup directly from source code, to enable simple tools (CGI scripts, etc)
to be able to render reference documentation.

The issue this is addressing is that the current workflow is way too heavy;
when I made my first post, it typically took 3-6 months for a contribution to
the documentation to appear on python.org.  Thanks to Trent and Neal, the
situation is a bit better today, but it's far from ideal.  (More on this below).

> The second sentence lacks a rationale.  What's wrong with "-- dashes"?
> What's "silly" about "``quotes''"?

I'd say that the fact that you're asking this should disqualify you from ever
working on documentation tools again...  don't you know *anything* about
typography ?

> The reference to ReST is wrong here; ReST certainly can know the
> semantics of a given identifier.

> I don't think you understand ReST except superficially.

That's why I'm listening to people who've tried to use ReST for this pur-
pose.  They've all failed.  Maybe they also only understood ReST super-
ficially.  Or maybe it's because ReST is created by people who have a
very shallow understanding of the issues involved in writing structured
reference documentation.  Your guess is as good as mine.

> My bias is simple: I am against wasting the time and effort required
> to replace one form of verbose markup with a different but equivalent
> form.  The proposed solution is no better than the status quo.

Support for quick turnaround, edit via the web, richer semantic information,
and a larger existing toolbase isn't better than the status quo ?

Sounds more like a ReSTian sour-grapes (or scorched-earth) approach
than a serious argument from someone who's interested in improving the
Python documentation.

> IMO, the markup itself is almost irrelevant.  As others have stated,
> writing good documentation is hard.  People will latch on to any
> excuse to avoid it, and markup is convenient.  But even if the docs
> had no markup whatsoever, if they were written in plain text like
> RFCs, I doubt there would be significantly more patch contributions.

The problem is that the WORKFLOW doesn't work.  Quoting from an earlier
post (you have read the earlier discussion, right?):

    If an ordinary user finds a minor issue, a typo, or an error in the
    documentation, the current user workflow is:

    1) (optionally) cut and paste the text to an editor, edit, and save to disk
    2) go to the sourceforge site, and locate the python project
    3) (optionally) sign up for a sourceforge account
    4) log in to your sourceforge account
    5) open a new bug or patch issue, and attach your suggestion
    6) wait 3-6 months for someone to pick up your patch, and for the
       next documentation release to appear on the site

to which can be added:

    If a developer finds a minor issue, a typo, or an error in the documentation,
    the current developer workflow is:

    If logged in to a machine with a full Python development environment:

    1) sync the project
    2) locate the right source file
    3) edit, and save to disk
    4) (optionally) regenerate the documentation and inspect it (3-6 minutes
        if you have the toolchain installed)
    5) commit the change (alternatively, generated a patch; see 2-5 above)
    6) wait up to 12 hours for the updated text to appear on the developer
       staging area on python.org, and 3-6 months for the next documentation
       release to appear on the site

    If no access to a Python development environment:

    1) (optionally) cut and paste the text to an editor, edit, and save to disk
    2) write note or mail to self (or generated a patch; see user instructions
       above)
    3) at an unspecified later time, try to remember what you did, and follow
       the developer instructions above.  wait 3-6 months for the next doc
       release to appear on the site.

I'm not arguing that a change of markup will solve this; I'm saying that a
a light-weight toolchain will make it possible to cut this down to 3-6 seconds
(plus optional moderation overhead for ordinary users).  If people get feedback,
they contribute.

The need for a light-weight markup language follows from this; to eliminate
as much overhead as possible,

 - the semantic model should match the reader's understanding of the
   target domain (in this case, the contents of Python library namespace)
 - the markup closely reflect the underlying semantic model
 - it should be *understandable* and *modifiable* (so you can tweak and
   copy constructs you see in the source, rather than having to look them
   up in a huge reference manual)
 - and it should be easy to parse for both machines and humans.

It's the workflow that's the real problem here, but you cannot fix the workflow
without fixing the markup.

> > We know that you have big hats over in ReST-land; now show us some
> > cattle.
>
> Moo.  Or would you prefer the Buddhist "mu"?
> What *are* you talking about?

I assume this means that we're going to keep getting more "ReST can certainly
do this but we're not going to show anyone how" posts from the ReST camp ?

</F>




From oliphant.travis at ieee.org  Fri Dec 30 10:13:00 2005
From: oliphant.travis at ieee.org (Travis E. Oliphant)
Date: Fri, 30 Dec 2005 02:13:00 -0700
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43B3ECEE.9060300@v.loewis.de>
References: <43B3ECEE.9060300@v.loewis.de>
Message-ID: <dp2tmv$l4j$1@sea.gmane.org>

Martin v. L?wis wrote:
> Please let me know what you think.
> 
> Regards,
> Martin
> 
> PEP: XXX
> Title: Using ssize_t as the index type
> Version: $Revision$
> Last-Modified: $Date$
> Author: Martin v. L?wis <martin at v.loewis.de>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 18-Dec-2005
> Post-History:
> 
> 
> Abstract
> ========
> 
> In Python 2.4, indices of sequences are restricted to the C type
> int. On 64-bit machines, sequences therefore cannot use the full
> address space, and are restricted to 2**31 elements. This PEP proposes
> to change this, introducing a platform-specific index type
> Py_ssize_t. An implementation of the proposed change is in
> http://svn.python.org/projects/python/branches/ssize_t.

Sounds wonderful.   Would love to see this in Python 2.5.  This will fix 
important 64-bit issues.  Perhaps someone could explain to me what the 
difference between Py_ssize_t and Py_intptr_t would be? Is this not a 
satisfactory Py_ssize_t already?


> 
> 
> Rationale
> =========
> 
> 64-bit machines are becoming more popular, and the size of main memory
> increases beyond 4GiB. On such machines, Python currently is limited,
> in that sequences (strings, unicode objects, tuples, lists,
> array.arrays, ...)  cannot contain more than 2GiElements.
> 
> Today, very few machines have memory to represent larger lists: as
> each pointer is 8B (in a 64-bit machine), one needs 16GiB to just hold
> the pointers of such a list; with data in the list, the memory
> consumption grows even more.  However, there are three container types
> for which users request improvements today:
> 
> * strings (currently restricted to 2GiB)
> * mmap objects (likewise; plus the system typically
>   won't keep the whole object in memory concurrently)
> * Numarray objects (from Numerical Python)

scipy_core objects are the replacement for both Numarray and Numerical 
Python and support 64-bit clean objects *except* for the sequence 
protocol and the buffer protocol.  Fixing this problem will clean up a 
lot of unnecessary code.

Looking forward to it...

-Travis




From reinhold-birkenfeld-nospam at wolke7.net  Fri Dec 30 10:26:10 2005
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Fri, 30 Dec 2005 10:26:10 +0100
Subject: [Python-Dev] Naming conventions in Py3K
In-Reply-To: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
References: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
Message-ID: <dp2ufi$n79$1@sea.gmane.org>

Ka-Ping Yee wrote:
> In a fair number of cases, Python doesn't follow its own recommended
> naming conventions.  Changing these things would break backward
> compatibility, so they are out of the question for Python 2.*, but
> it would be nice to keep these in mind for Python 3K.
> 
>     Constants in all caps:
>         NONE, TRUE, FALSE, ELLIPSIS

That's ugly. I bet with this change you will find modules out there
which do

True = TRUE

at the start.

In fact, I like it that the basic Python functions and most of the types
are all-lowercase. It find the code to be better readable.

Reinhold

-- 
Mail address is perfectly valid!


From martin at v.loewis.de  Fri Dec 30 10:48:52 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 30 Dec 2005 10:48:52 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <dp1nja$6gk$1@sea.gmane.org>
References: <43B3ECEE.9060300@v.loewis.de> <dp1nja$6gk$1@sea.gmane.org>
Message-ID: <43B50284.4070609@v.loewis.de>

Fredrik Lundh wrote:
> well, one thing seems to missing from your PEP: in several modules, you've
> changed the cast used in the type table. e.g.
...
> is this change backwards compatible ?

See the section "Conversion guidelines". I prefer the approach taken in
the patch below, i.e. remove the casts in initializing the type object.
Adding more typedefs for the function types would be the other option.

Regards,
Martin

Index: Modules/_elementtree.c
===================================================================
--- Modules/_elementtree.c	(Revision 41837)
+++ Modules/_elementtree.c	(Arbeitskopie)
@@ -92,6 +92,9 @@
 #endif

 /* compatibility macros */
+#if (PY_VERSION_HEX < 0x02050000)
+typedef int Py_ssize_t;
+#endif
 #if (PY_VERSION_HEX < 0x02040000)
 #define PyDict_CheckExact PyDict_Check
 #if (PY_VERSION_HEX < 0x02020000)
@@ -916,8 +919,9 @@
 }

 static PyObject*
-element_getitem(ElementObject* self, Py_ssize_t index)
+element_getitem(PyObject* _self, Py_ssize_t index)
 {
+    ElementObject* self = (ElementObject*)_self;
     if (!self->extra || index < 0 || index >= self->extra->length) {
         PyErr_SetString(
             PyExc_IndexError,
@@ -931,8 +935,9 @@
 }

 static PyObject*
-element_getslice(ElementObject* self, Py_ssize_t start, Py_ssize_t end)
+element_getslice(PyObject* _self, Py_ssize_t start, Py_ssize_t end)
 {
+    ElementObject* self = (ElementObject*)_self;
     Py_ssize_t i;
     PyObject* list;

@@ -1158,8 +1163,9 @@
 }

 static int
-element_setslice(ElementObject* self, Py_ssize_t start, Py_ssize_t end,
PyObject* item)
+element_setslice(PyObject* _self, Py_ssize_t start, Py_ssize_t end,
PyObject* item)
 {
+    ElementObject* self = (ElementObject*)_self;
     int i, new, old;
     PyObject* recycle = NULL;

@@ -1228,8 +1234,9 @@
 }

 static int
-element_setitem(ElementObject* self, Py_ssize_t index, PyObject* item)
+element_setitem(PyObject* _self, Py_ssize_t index, PyObject* item)
 {
+    ElementObject* self = (ElementObject*)_self;
     int i;
     PyObject* old;

@@ -1371,10 +1378,10 @@
     (inquiry) element_length,
     0, /* sq_concat */
     0, /* sq_repeat */
-    (ssizeargfunc) element_getitem,
-    (ssizessizeargfunc) element_getslice,
-    (ssizeobjargproc) element_setitem,
-    (ssizessizeobjargproc) element_setslice,
+    element_getitem,
+    element_getslice,
+    element_setitem,
+    element_setslice,
 };

 statichere PyTypeObject Element_Type = {

From martin at v.loewis.de  Fri Dec 30 10:58:16 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 30 Dec 2005 10:58:16 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <dp2tmv$l4j$1@sea.gmane.org>
References: <43B3ECEE.9060300@v.loewis.de> <dp2tmv$l4j$1@sea.gmane.org>
Message-ID: <43B504B8.8090406@v.loewis.de>

Travis E. Oliphant wrote:
> Sounds wonderful.   Would love to see this in Python 2.5.  This will fix 
> important 64-bit issues.  Perhaps someone could explain to me what the 
> difference between Py_ssize_t and Py_intptr_t would be? Is this not a 
> satisfactory Py_ssize_t already?

Practically, yes. Conceptually, intptr_t and size_t are different
things: sizeof(void*) might differ from sizeof(size_t) (e.g. when you
have segment and offset in pointers). I believe in flat-address space
machines, they are always the same.

Regards,
Martin

From martin at v.loewis.de  Fri Dec 30 11:26:44 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 30 Dec 2005 11:26:44 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <20051229173041.GA19575@code1.codespeak.net>
References: <43B3ECEE.9060300@v.loewis.de>
	<20051229173041.GA19575@code1.codespeak.net>
Message-ID: <43B50B64.20904@v.loewis.de>

Armin Rigo wrote:
> I guess you mean LONG_MAX instead of MAX_INT, in the event that ssize_t
> is larger than a long.

Right, changed.

> Also, distinguishing between PyInt_AsSsize_t()
> and PyLong_AsSsize_t() doesn't seem to be useful (a quick look in your
> branch makes me guess that they both accept an int or a long object
> anyway).

In changing this, I found there is a difference: callers typically check
for int, then do PyInt_AS_LONG, and use the result right away, then
they check for long, use PyLong_AsSsize_t, check for a -1 result, check
if an exception occurred, and only then use the result.

With PyLong_AsSsize_t gone, this can be unified (done in r41851);
the code becomes clearer, but there will be more type checks in the
usual code path.

> Hum.  It would be much cleaner to introduce a new format character to
> replace '#' and deprecate '#'...

That would certainly be clearer. What character would you suggest?

I see two drawbacks with that approach:
1. writing backwards-compatible modules will become harder.
   Users have to put ifdefs around the ParseTuple calls (or atleast
   around the format strings)
2. I see this as a transitional change; in P3k, indexing should be
   done exclusively through Py_ssize_t. Then '#' can be removed,
   and we are stuck with that other character.

Regards,
Martin

From martin at v.loewis.de  Fri Dec 30 11:33:10 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 30 Dec 2005 11:33:10 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <1f7befae0512291334r77fa03cx62e1e43dce757213@mail.gmail.com>
References: <43B3ECEE.9060300@v.loewis.de>
	<1f7befae0512291334r77fa03cx62e1e43dce757213@mail.gmail.com>
Message-ID: <43B50CE6.2050306@v.loewis.de>

Tim Peters wrote:
> Better to use a new gibberish character and deprecate the old one? 
> Otherwise I'm afraid we'll be stuck supporting PY_SIZE_T_CLEAN
> forever, and it's not good to have the meaning of a format string
> depend on the presence or absence of a #define far away in the file.

See my response to Armin also; I'm concerned that writing code that uses
that character will not easily be backwards compatible. What character
would you suggest?

I would not expect that we are stuck with PY_SIZE_T_CLEAN: In P3k,
Py_ssize_t should become the only type to do indexing, at which
point PY_SIZE_T_CLEAN could go.

> That can be done in one gulp via:
> 
>     some_int = Py_SAFE_DOWNCAST(some_value, Py_ssize_t, int);

Thanks, added (I didn't know it existed).

Regards,
Martin

From martin at v.loewis.de  Fri Dec 30 11:40:50 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 30 Dec 2005 11:40:50 +0100
Subject: [Python-Dev] Naming conventions in Py3K
In-Reply-To: <bbaeab100512292233p4afaec3bme4350042a9ee5c48@mail.gmail.com>
References: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
	<bbaeab100512292233p4afaec3bme4350042a9ee5c48@mail.gmail.com>
Message-ID: <43B50EB2.4020402@v.loewis.de>

Brett Cannon wrote:
> I am fine with changing the built-in types, but changing the built-in
> singletons just looks *really* odd to me.  So odd that I just don't
> want to see them changed.  I mean when I think of constants, I think
> of a variable that references an object and that reference never
> changes.  The built-ins you are referencing, though, are singletons:
> named objects that are not variables.  So keeping their naming scheme
> as-is does not feel like a breaking of the rules to me since they are
> not treated the same (especially at the C level).

Actually, I thought some of them would become reserved words in P3k,
anyway.

Regards,
Martin

From python-dev at zesty.ca  Fri Dec 30 12:10:10 2005
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Fri, 30 Dec 2005 05:10:10 -0600 (CST)
Subject: [Python-Dev] Naming conventions in Py3K
In-Reply-To: <dp2ufi$n79$1@sea.gmane.org>
References: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
	<dp2ufi$n79$1@sea.gmane.org>
Message-ID: <Pine.LNX.4.58.0512300459460.6831@server1.LFW.org>

On Fri, 30 Dec 2005, Reinhold Birkenfeld wrote:
> Ka-Ping Yee wrote:
> >     Constants in all caps:
> >         NONE, TRUE, FALSE, ELLIPSIS
>
> That's ugly.

I know it looks ugly to you now.  But there's a good reason why we use
capitalization for class names -- anyone reading code who comes across
a CapitalizedName can be reasonably certain that it refers to a class.
It's a helpful way to express the intended usage.  And, like it or not,
None, True, False, and Ellipsis aren't classes.

> In fact, I like it that the basic Python functions

I didn't say anything about renaming functions.  Functions in lowercase
are one of the naming conventions that Python does follow consistently.

> and most of the types are all-lowercase.

That's just not true, though.  (Or at least it depends on what you mean
by "most" and by "types".)  The types in the built-in module are in
lowercase, and the vast majority of the other types aren't.


-- ?!ng

From python-dev at zesty.ca  Fri Dec 30 12:14:31 2005
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Fri, 30 Dec 2005 05:14:31 -0600 (CST)
Subject: [Python-Dev] Naming conventions in Py3K
In-Reply-To: <43B50EB2.4020402@v.loewis.de>
References: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
	<bbaeab100512292233p4afaec3bme4350042a9ee5c48@mail.gmail.com>
	<43B50EB2.4020402@v.loewis.de>
Message-ID: <Pine.LNX.4.58.0512300510160.6831@server1.LFW.org>

Brett Cannon wrote:
> I am fine with changing the built-in types, but changing the built-in
> singletons just looks *really* odd to me.  So odd that I just don't
> want to see them changed.  I mean when I think of constants, I think
> of a variable that references an object and that reference never
> changes.  The built-ins you are referencing, though, are singletons:
> named objects that are not variables.

In behaviour and intended usage, they are much more like constants than
like classes, though.

> So keeping their naming scheme
> as-is does not feel like a breaking of the rules to me since they are
> not treated the same (especially at the C level).

I don't know if we share the same philosophy on this, but i don't think
the C level should play a big role here -- the names in Python programs
should express how something works (and more importantly how it is
intended to be used) at the Python level, not details under the hood.

On Fri, 30 Dec 2005, "Martin v. L?wis" wrote:
> Actually, I thought some of them would become reserved words in P3k,
> anyway.

That would be cool.  If so, it would make sense for them to be all in
lowercase.


-- ?!ng

From martin at v.loewis.de  Fri Dec 30 12:28:43 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 30 Dec 2005 12:28:43 +0100
Subject: [Python-Dev] Naming conventions in Py3K
In-Reply-To: <Pine.LNX.4.58.0512300510160.6831@server1.LFW.org>
References: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
	<bbaeab100512292233p4afaec3bme4350042a9ee5c48@mail.gmail.com>
	<43B50EB2.4020402@v.loewis.de>
	<Pine.LNX.4.58.0512300510160.6831@server1.LFW.org>
Message-ID: <43B519EB.70503@v.loewis.de>

Ka-Ping Yee wrote:
>>Actually, I thought some of them would become reserved words in P3k,
>>anyway.
> 
> 
> That would be cool.  If so, it would make sense for them to be all in
> lowercase.

And rename None to null in the process :-)

Regards,
Martin

From python-dev at zesty.ca  Fri Dec 30 12:31:05 2005
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Fri, 30 Dec 2005 05:31:05 -0600 (CST)
Subject: [Python-Dev] Naming conventions in Py3K
In-Reply-To: <43B519EB.70503@v.loewis.de>
References: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
	<bbaeab100512292233p4afaec3bme4350042a9ee5c48@mail.gmail.com>
	<43B50EB2.4020402@v.loewis.de>
	<Pine.LNX.4.58.0512300510160.6831@server1.LFW.org>
	<43B519EB.70503@v.loewis.de>
Message-ID: <Pine.LNX.4.58.0512300530280.6831@server1.LFW.org>

On Fri, 30 Dec 2005, "Martin v. L?wis" wrote:
> Ka-Ping Yee wrote:
> > That would be cool.  If so, it would make sense for them to be all in
> > lowercase.
>
> And rename None to null in the process :-)

Is there a really good reason to do that?  It's not obvious to me.


-- ?!ng

From pedronis at strakt.com  Fri Dec 30 12:37:37 2005
From: pedronis at strakt.com (Samuele Pedroni)
Date: Fri, 30 Dec 2005 12:37:37 +0100
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <43B4A3F6.5040602@gmail.com>
References: <20051229111820.pncqj7pozj7oks84@login.werra.lunarpages.com>	<43B43CCF.8050807@strakt.com>
	<43B4A3F6.5040602@gmail.com>
Message-ID: <43B51C01.3030109@strakt.com>

Nick Coghlan wrote:
> Samuele Pedroni wrote:
> 
>>Michael Chermside wrote:
>>
>>>The F-bot writes:
>>>
>>>
>>>>in reality, some things are carefully thought out and craftily im-
>>>>plemented, some things are engineering tradeoffs made at a certain time,
>>>>and some things are just accidents -- but python-dev will happily defend
>>>>the current solution with the same energy, no matter what it is.
>>>
>>>+1 QOTF.
>>>
>>>Seriously... I've seen this behavior also, but I haven't ever thought
>>>about it as clearly as Fredrik does here. When we go to answer questions
>>>we ought to pause briefly first and decide which of these categories
>>>applies to a given piece of behavior. I think users will be understanding
>>>if we're honest about what are the accidents -- every language or
>>>software package has some, and just because it's an accident does NOT
>>>mean we should "fix" it.
> 
> 
> Most of the times I've asked questions along these lines I've gotten 
> well-considered answers (usually because something I thought was a deliberate 
> design decision on Guido's part turned out to simply be an accident of the way 
> it happened to be implemented in CPython).
> 
> 
>>it's indeed a matter of trade-offs. Converting NameErrors into commands
>>doesn't look like a good trade off in terms of expectations management
>>and understandable behavior. Ka-Ping Yee ExitHatch still seem a 
>>reasonable improvement. Fernando Perez considerations about Python
>>vs. "commands" and prefixing and extra-linguistic extensions seem
>>also on spot. It's not a matter of defending the status quo, more about
>>what kind of price is reasonable for DWIM.
> 
> 
> I think Fredrik has made an excellent case for promoting the quit & exit 
> interpreter-only builtins to be proper callables.
> 

notice that I wrote that Ka-Ping Yee ExitHatch is an improvement!

> Hell, we even have that capability for the callable that displays the 
> *license* text. . . surely quitting the interpreter is far more important than 
> being able to display the license text, but the support for the latter is 
> significantly better:
> 
> 
> Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> Py> exit
> 'Use Ctrl-Z plus Return to exit.'
> Py> quit
> 'Use Ctrl-Z plus Return to exit.'
> Py> license
> Type license() to see the full license text
> Py> type(license)
> <class 'site._Printer'>
> 
> Counting blank lines, 60 lines of site.py are devoted to getting copyright, 
> credit and license to work right, 16 to getting help to work, and only 12 to 
> setting quit and exit to the 'right' strings - and due to modules like 
> readline for Windows and differences in the way interpreters buffer the line 
> input, the exit string for Windows is not always correct.
> 
> Cheers,
> Nick.
> 


From martin at v.loewis.de  Fri Dec 30 12:34:22 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 30 Dec 2005 12:34:22 +0100
Subject: [Python-Dev] Naming conventions in Py3K
In-Reply-To: <Pine.LNX.4.58.0512300530280.6831@server1.LFW.org>
References: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
	<bbaeab100512292233p4afaec3bme4350042a9ee5c48@mail.gmail.com>
	<43B50EB2.4020402@v.loewis.de>
	<Pine.LNX.4.58.0512300510160.6831@server1.LFW.org>
	<43B519EB.70503@v.loewis.de>
	<Pine.LNX.4.58.0512300530280.6831@server1.LFW.org>
Message-ID: <43B51B3E.9020600@v.loewis.de>

Ka-Ping Yee wrote:
> On Fri, 30 Dec 2005, "Martin v. L?wis" wrote:
> 
>>Ka-Ping Yee wrote:
>>
>>>That would be cool.  If so, it would make sense for them to be all in
>>>lowercase.
>>
>>And rename None to null in the process :-)
> 
> 
> Is there a really good reason to do that?  It's not obvious to me.

That was mostly joking: it would sound more familiar to people from
Java and C#.

Recalling the <type 'bool'> introduction, I don't expect Guido can
agree to writing true and false instead of True and False; nobody
has ever proposed to write none (AFAIR).

Regards,
Martin

From abo at minkirri.apana.org.au  Fri Dec 30 12:56:16 2005
From: abo at minkirri.apana.org.au (Donovan Baarda)
Date: Fri, 30 Dec 2005 11:56:16 +0000
Subject: [Python-Dev] [Doc-SIG] that library reference, again
In-Reply-To: <dp2qt7$dib$1@sea.gmane.org>
References: <dope79$flv$2@sea.gmane.org><dp0qrh$r78$1@sea.gmane.org>
	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com>
	<dp1jd8$p4a$1@sea.gmane.org> <43B4A238.2030101@python.org>
	<dp2qt7$dib$1@sea.gmane.org>
Message-ID: <1135943776.8971.77.camel@warna.dub.corp.google.com>

I've been dodging this thread because I don't really have much to add,
apart from a documentation end user point of view...

On Fri, 2005-12-30 at 09:25 +0100, Fredrik Lundh wrote:
[...]
> > Another goal is highly biased toward XML-style markup:
> >
> >     * Make it trivial to do basic rendering to HTML (a few re.sub
> >       calls should suffice), to enable simple tools (CGI scripts, etc)
> >       to be able to render reference documentation.
> >
> > This is the tail wagging the dog.
> 
> No, it's a fundamental goal: to support light-weight generation of rendered
> markup directly from source code, to enable simple tools (CGI scripts, etc)
> to be able to render reference documentation.

Python is run-time interpreted, but I don't think we need its
documentation to be. "trivial" is a relative term. From my point of
view, provided I can apt-get one or to not-too-esoteric packages then do
something like "make html", it's trivial enough for me.

> The issue this is addressing is that the current workflow is way too heavy;
> when I made my first post, it typically took 3-6 months for a contribution to
> the documentation to appear on python.org.  Thanks to Trent and Neal, the
> situation is a bit better today, but it's far from ideal.  (More on this below).
[...]
> That's why I'm listening to people who've tried to use ReST for this pur-
> pose.  They've all failed.  Maybe they also only understood ReST super-
> ficially.  Or maybe it's because ReST is created by people who have a
> very shallow understanding of the issues involved in writing structured
> reference documentation.  Your guess is as good as mine.

As a some-times developer not really interested in writing
documentation, I must admit I really like ReST. It looks like plain
text, so I can write and use it as plain text, but it also can magically
transform to other formats and look pretty good.

I hate writing XML/HTML, and my only experience with LaTex was brief and
painful too. If I was forced to choose, I'd probably pick LaTex even
though I don't know it as well as XML/HTML (I'd rather face an unknown
evil than that particular devil I know).

> > My bias is simple: I am against wasting the time and effort required
> > to replace one form of verbose markup with a different but equivalent
> > form.  The proposed solution is no better than the status quo.
> 
> Support for quick turnaround, edit via the web, richer semantic information,
> and a larger existing toolbase isn't better than the status quo ?
[...]

I'm not going to write docs for stuff I didn't write myself. I'm not
going to submit patches to Javadoc, LaTex or XML/HTML, but I might post
vague "change this to..." bug reports. As an end user, one of the best
things about Python is the on-line documentation... the status-quo looks
pretty good from here. 

...but I'm not the person doing it... remember I'm providing a
documentation end-user POV here :-)

> > IMO, the markup itself is almost irrelevant.  As others have stated,
> > writing good documentation is hard.  People will latch on to any
> > excuse to avoid it, and markup is convenient.  But even if the docs
> > had no markup whatsoever, if they were written in plain text like
> > RFCs, I doubt there would be significantly more patch contributions.

Writing good documentation is hard (which is why I prefer to leave it to
others if I can), but fixing almost good documentation doesn't have to
be, and for that, the markup used can make a difference. 

As an end user that spots a problem, I'll look at the source, and if
it's in a language I know well enough to fix the problem, I'll submit a
patch. If I find it's in a language I don't know and can't grok well
enough to see how to fix the problem in 10 minutes, I'm going to submit
a verbal description as a bug. 

If this is the 5+ time I've stalled on this language, I might consider
learning it so I can do a patch next time. However, in this case the
language in question is a "documentation language" and I want to be a
software programmer, not a document publisher, so I'll probably just
submit bugs every time, and after the 2+ time I won't even bother
looking at the source.

> It's the workflow that's the real problem here, but you cannot fix the workflow
> without fixing the markup.

I must say I disagree with this statement... changing the markup will
not change the workflow at all, just shave 3-6 mins off the compile-test
step. I don't do this enough to really know if that's worth it.

If the markup is changed to something more widely known and/or simple,
more people might participate in the "generate patch" workflow rather
than the "submit bug" workflow, and maybe that will make things easier
for the big picture "update and release docs" workflow. But the speed of
the tool-chain has little to do with this, only the "documentation
language" popularity among the developers and users.

...and if the LaTeX guys don't mind fixing bugs instead of applying
patches and are handling the load... the status quo is fine by me, I'm
happy not to do documentation :-)

-- 
Donovan Baarda <abo at minkirri.apana.org.au>
http://minkirri.apana.org.au/~abo/


From srichter at cosmos.phy.tufts.edu  Fri Dec 30 14:44:50 2005
From: srichter at cosmos.phy.tufts.edu (Stephan Richter)
Date: Fri, 30 Dec 2005 08:44:50 -0500
Subject: [Python-Dev] Naming conventions in Py3K
In-Reply-To: <Pine.LNX.4.58.0512300459460.6831@server1.LFW.org>
References: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
	<dp2ufi$n79$1@sea.gmane.org>
	<Pine.LNX.4.58.0512300459460.6831@server1.LFW.org>
Message-ID: <200512300844.50235.srichter@cosmos.phy.tufts.edu>

On Friday 30 December 2005 06:10, Ka-Ping Yee wrote:
> > In fact, I like it that the basic Python functions
>
> I didn't say anything about renaming functions. ?Functions in lowercase
> are one of the naming conventions that Python does follow consistently.

well, it is not consistent when looking at functions versus methods. The best 
example here is hasattr() versus dict.has_key().

Overall I agree with your proposal to use a consistent naming style. Whatever 
it may turn out to be. We have done the same when we started with Zope 3 and 
while having some few inconsistencies, the overall code base benefitted a 
lot, I think.

Regards,
Stephan
-- 
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training

From fdrake at acm.org  Fri Dec 30 15:46:52 2005
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri, 30 Dec 2005 09:46:52 -0500
Subject: [Python-Dev] Naming conventions in Py3K
In-Reply-To: <Pine.LNX.4.58.0512300530280.6831@server1.LFW.org>
References: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
	<43B519EB.70503@v.loewis.de>
	<Pine.LNX.4.58.0512300530280.6831@server1.LFW.org>
Message-ID: <200512300946.53082.fdrake@acm.org>

On Friday 30 December 2005 06:31, Ka-Ping Yee wrote:
 > Is there a really good reason to do that?  It's not obvious to me.

No more than there is to rename None to none or NONE.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From mcherm at mcherm.com  Fri Dec 30 15:48:19 2005
From: mcherm at mcherm.com (Michael Chermside)
Date: Fri, 30 Dec 2005 06:48:19 -0800
Subject: [Python-Dev] Naming conventions in Py3K
Message-ID: <20051230064819.sjznio8ffbkck8go@login.werra.lunarpages.com>

?!ng proposes:
>     Constants in all caps:
>         NONE, TRUE, FALSE, ELLIPSIS
>
>     Classes in initial-caps:
>         Object, Int, Float, Str, Unicode, Set, List, Tuple, Dict,
>         and lots of classes in the standard library, e.g.
>         anydbm.error, csv.excel, imaplib.error, mutex.mutex...

(All that follows is just my opinion. Feel free to disregard.)

 1. PEP 8 is just some recommended conventions, not absolute rules.

 2. "None", "True", and "False" are the divinely inspired correct
    spellings of these objects. All caps would be incorrect.

 3. "object", "int", "float", "str", "unicode", "set", "list",
    "tuple", and "dict" all follow the common convention that the
    fundamental built-in types are in all lowercase. Note that I
    am distinguishing between built-in types and standard library
    types. I rather like this convention and would favor keeping
    it.

 4. I am a big fan of consistancy in naming. I try to follow PEP 8
    in my own code, even when I don't think it's as pretty as some
    other practice. But I just don't think the consistancy is worth
    the cost of breaking existing code. Python 3000 is ALLOWED to
    break code, but that doesn't mean it should do so gratuitously
    or break more code than necessary.

 5. For some of the classes within the standard library I'm much
    more open to being convinced. They are less often used, thus
    more suitable for a global fix-and-replace or at tweak to the
    input statements at the top of the file. Being less frequently
    used also means that consistancy in naming is more important
    because people don't necessarily use these every day.

-- Michael Chermside


From python-dev at zesty.ca  Fri Dec 30 15:48:48 2005
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Fri, 30 Dec 2005 08:48:48 -0600 (CST)
Subject: [Python-Dev] Naming conventions in Py3K
In-Reply-To: <200512300946.53082.fdrake@acm.org>
References: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
	<43B519EB.70503@v.loewis.de>
	<Pine.LNX.4.58.0512300530280.6831@server1.LFW.org>
	<200512300946.53082.fdrake@acm.org>
Message-ID: <Pine.LNX.4.58.0512300847440.6831@server1.LFW.org>

On Fri, 30 Dec 2005, Fred L. Drake, Jr. wrote:
> On Friday 30 December 2005 06:31, Ka-Ping Yee wrote:
>  > Is there a really good reason to do that?  It's not obvious to me.
>
> No more than there is to rename None to none or NONE.

For that, there is a reason: None is not a class.


-- ?!ng

From barry at python.org  Fri Dec 30 16:16:43 2005
From: barry at python.org (Barry Warsaw)
Date: Fri, 30 Dec 2005 10:16:43 -0500
Subject: [Python-Dev] Naming conventions in Py3K
In-Reply-To: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
References: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
Message-ID: <1135955803.25664.232.camel@geddy.wooz.org>

On Thu, 2005-12-29 at 22:29 -0600, Ka-Ping Yee wrote:
> In a fair number of cases, Python doesn't follow its own recommended
> naming conventions.  Changing these things would break backward
> compatibility, so they are out of the question for Python 2.*, but
> it would be nice to keep these in mind for Python 3K.
> 
>     Constants in all caps:
>         NONE, TRUE, FALSE, ELLIPSIS

Just from a practical standpoint, I'm -1 on this.  These names (at least
the first three) are typed /so/ often in Python programs, just think of
the finger pain caused by excessive use of the shift key.  Now, I
personally swap capslock and control so it would be a PITA <wink> for
me, but doable.  But I know a lot of people who disable capslock
altogether so they'd be performing all kinds of pinkie stretching
gymnastics all the time.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20051230/62054004/attachment.pgp

From guido at python.org  Fri Dec 30 16:41:32 2005
From: guido at python.org (Guido van Rossum)
Date: Fri, 30 Dec 2005 07:41:32 -0800
Subject: [Python-Dev] Naming conventions in Py3K
In-Reply-To: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
References: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
Message-ID: <ca471dc20512300741pe2dcd61s7e7fb1169c5b3d0d@mail.gmail.com>

I think the discussion is coming to a clear conclusion here not to do
this (except for the standard library classes like anydbm.error). I'm
piping in with my own -1 (for all the sane reasons) to hopefully stop
this thread quickly. We don't need more noise here.

--Guido

On 12/29/05, Ka-Ping Yee <python-dev at zesty.ca> wrote:
> In a fair number of cases, Python doesn't follow its own recommended
> naming conventions.  Changing these things would break backward
> compatibility, so they are out of the question for Python 2.*, but
> it would be nice to keep these in mind for Python 3K.
>
>     Constants in all caps:
>         NONE, TRUE, FALSE, ELLIPSIS
>
>     Classes in initial-caps:
>         Object, Int, Float, Str, Unicode, Set, List, Tuple, Dict,
>         and lots of classes in the standard library, e.g.
>         anydbm.error, csv.excel, imaplib.error, mutex.mutex...
>
> I know these probably look a little funny now to most of us, as
> we're used to looking at today's Python (they even look a little
> funny to me) but i'm pretty convinced that consistency will be
> better in the long run.
>
>
> -- ?!ng
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From thomas at xs4all.net  Fri Dec 30 16:51:34 2005
From: thomas at xs4all.net (Thomas Wouters)
Date: Fri, 30 Dec 2005 16:51:34 +0100
Subject: [Python-Dev] Naming conventions in Py3K
In-Reply-To: <1135955803.25664.232.camel@geddy.wooz.org>
References: <Pine.LNX.4.58.0512292214510.6831@server1.LFW.org>
	<1135955803.25664.232.camel@geddy.wooz.org>
Message-ID: <20051230155134.GC18916@xs4all.nl>

On Fri, Dec 30, 2005 at 10:16:43AM -0500, Barry Warsaw wrote:
> On Thu, 2005-12-29 at 22:29 -0600, Ka-Ping Yee wrote:
> > In a fair number of cases, Python doesn't follow its own recommended
> > naming conventions.  Changing these things would break backward
> > compatibility, so they are out of the question for Python 2.*, but
> > it would be nice to keep these in mind for Python 3K.
> > 
> >     Constants in all caps:
> >         NONE, TRUE, FALSE, ELLIPSIS

> Just from a practical standpoint, I'm -1 on this.  These names (at least
> the first three) are typed /so/ often in Python programs, just think of
> the finger pain caused by excessive use of the shift key.  Now, I
> personally swap capslock and control so it would be a PITA <wink> for
> me, but doable.  But I know a lot of people who disable capslock
> altogether so they'd be performing all kinds of pinkie stretching
> gymnastics all the time.

Well, we all know code is read many more times than written, so that's a bit
of a weak argument. Personally, I'd be -1 on this because just reading the
example makes me want to jerk upright, stand at attention, salute and shout
"SIR SORRY I ASKED SIR".

There's a lot to be said for consistency and accomodating newbies (given
that we assume oldbies are already used to the current Spelling.) For
example, I've encountered many new programmers who are confused by the (lack
of) difference between builtins and variables. This would seem to be an
argument to use a form of sigils or declarations to indicate variables.
Peronally, I prefer to explain the (lack of) difference, and trust in the
fact that they'll encounter the situation in real code often enough not to
have a problem with it.

The same can be said for the suggested changes (except the stdlib modules.)
I've never seen anyone be confused about int vs Int, None vs none vs NONE,
etc. It's also not hard to get used to the particular spelling in each case;
there's more than enough exposure to burn it into even the casual
programmer's mind. And that is how people will remember it. Not by some rule
("oh, it's a constant, it has to be all caps") but by seeing it all caps all
the time. This also suggests people actively using the old names in Py3K (by
using None = NONE or what not) will actively be inhibiting the acceptance of
the new names ;)

This is all much less the case with stdlib modules, I guess, and I see no
real objection to making those conform to PEP 8.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From goodger at python.org  Fri Dec 30 17:27:20 2005
From: goodger at python.org (David Goodger)
Date: Fri, 30 Dec 2005 11:27:20 -0500
Subject: [Python-Dev] [Doc-SIG] that library reference, again
In-Reply-To: <dp2qt7$dib$1@sea.gmane.org>
References: <dope79$flv$2@sea.gmane.org><dp0qrh$r78$1@sea.gmane.org>	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com><dp1jd8$p4a$1@sea.gmane.org>	<43B4A238.2030101@python.org>
	<dp2qt7$dib$1@sea.gmane.org>
Message-ID: <43B55FE8.1040401@python.org>

[David Goodger]
>> The second sentence lacks a rationale.  What's wrong with "--
>> dashes"?  What's "silly" about "``quotes''"?

[Fredrik Lundh]
> don't you know *anything* about typography ?

Yes, for a layman, I know plenty.  I am not a typographer though.

Simply put, your "list of goals" provides no context for your
statements.  *I* know that "--" is turned into an en-dash in TeX, and
that "``these''" are turned into curly quotes.  So?  What's "silly"
about that?  Spell it out, man!  And what about 99% of the people who
read your page?  They won't know the first thing about what you're
talking about.

>> The reference to ReST is wrong here; ReST certainly can know the
>> semantics of a given identifier.
>
>> I don't think you understand ReST except superficially.
>
> That's why I'm listening to people who've tried to use ReST for this
> purpose.  They've all failed.  Maybe they also only understood ReST
> superficially.

Perhaps, since ReST is *not* designed as a semantic markup language.
It's designed as an implicit markup language, with explicit extensions
for semantic markup.  In any case, ReST is *not* being proposed here.

> Or maybe it's because ReST is created by people who have a very
> shallow understanding of the issues involved in writing structured
> reference documentation.  Your guess is as good as mine.

Why does Fredrik find it necessary to descend to personal insults?
Your guess is as good as mine.

> Support for quick turnaround, edit via the web, richer semantic
> information, and a larger existing toolbase isn't better than the
> status quo ?

Those would be good features.  Those features are not mentioned in
your list of goals though!  (http://effbot.org/zone/pythondoc-lib.htm)
AFAICT, you just went off on a tangent to create a new markup
language, which we don't need.

> The problem is that the WORKFLOW doesn't work.

So fix the workflow.  Something like Ian Bicking's Commentary system,
or something more specific to Python's docs, seems to fit the bill.

> It's the workflow that's the real problem here, but you cannot fix
> the workflow without fixing the markup.

I disagree.  The markup doesn't need an overhaul to fix the workflow.

> I assume this means that we're going to keep getting more "ReST can
> certainly do this but we're not going to show anyone how" posts from
> the ReST camp ?

You assume incorrectly.  I'm not talking about ReST.  I'm not
proposing ReST for anything.  Please ignore the fact that I'm the
author of ReST; I never brought it up, I never proposed it.  *You* are
the one harping on it.

I'm just against an arbitrary and unnecessary change of markup in
Python's docs.

> Your reply makes it obvious that you don't understand the issues
> involved here, nor how the goals address them.

Your reply, and your regular descent to personal insults, make it
excruciatingly obvious that you're a troll.

I hesitated before first replying to this thread, suspecting (from
past experience) that this would be the response I'd get.  From now
on, I'll avoid feeding this particular troll.

> I'd say that the fact that you're asking this should disqualify you
> from ever working on documentation tools again...

Your writing malicious crap like this ought to disqualify you from
ever participating in python-dev discussions again.

Fredrik, we all know that you you are a superb developer who makes
wonderful contributions to Python.  We also know that when faced with
disagreement (and sometimes, seemingly, without any provocation at
all) you are a malicious prick.

-- 
David Goodger <http://python.net/~goodger>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 249 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20051230/b6af558a/signature.pgp

From mal at egenix.com  Fri Dec 30 18:31:26 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 30 Dec 2005 18:31:26 +0100
Subject: [Python-Dev] [Doc-SIG] that library reference, again
In-Reply-To: <dp2mmh$45b$1@sea.gmane.org>
References: <dope79$flv$2@sea.gmane.org><dp0qrh$r78$1@sea.gmane.org>	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com><dp1jd8$p4a$1@sea.gmane.org>	<43B4A238.2030101@python.org>	<51A7717B-3832-477E-BED5-3C36DCA20336@lag.net>
	<dp2mmh$45b$1@sea.gmane.org>
Message-ID: <43B56EEE.9090904@egenix.com>

I haven't followed the thread, so many I'm repeating things.

Has anyone considered using e.g. MediaWiki (the wiki used for
Wikipedia) for Python documentation ?

I'm asking because this wiki has proven to be ideally suited
for creating complex documentation tasks and offers many features
which would make Python documentation a lot easier and more
accessible:

* people wouldn't have to learn LaTeX to commit doc-patches
* it's easy to monitor and revert changes, discuss changes
* there's version history available
* new docs would be instantly available on the web
* builtin search facility, categories and all the other nifty
  wiki stuff
* it's one of the more popular wikis around and due to Wikipedia
  it's here to stay
* conversion to XML and DocBook is possible, providing
  entry points for conversion to other formats (including
  LaTeX)
* more following means more tools (over time)

Just a thought.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 30 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From trentm at ActiveState.com  Fri Dec 30 19:45:34 2005
From: trentm at ActiveState.com (Trent Mick)
Date: Fri, 30 Dec 2005 10:45:34 -0800
Subject: [Python-Dev] slight inconsistency in svn checkin email subject lines
Message-ID: <20051230184534.GC30751@ActiveState.com>

Here are the subject lines for two recent svn commit emails:

   [Python-checkins] commit of r41847 - in python/trunk: Lib/test/test__locale.py Python/as...
   [Python-checkins]  commit of r41848 - python/trunk/setup.py
                     ^
                     `--- one extra space

There is an extra space when the checkin includes exactly one file (at
least, I think that is the condition).

Is this intentional? If not, could someone point me to where the svn
trigger scripts are maintained so I could poke around for a fix? (Or
just fix it themselves. :)

Cheers,
Trent

-- 
Trent Mick
TrentM at ActiveState.com

From ianb at colorstudy.com  Fri Dec 30 20:48:25 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 30 Dec 2005 13:48:25 -0600
Subject: [Python-Dev] [Doc-SIG] that library reference, again
In-Reply-To: <43B55FE8.1040401@python.org>
References: <dope79$flv$2@sea.gmane.org><dp0qrh$r78$1@sea.gmane.org>	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com><dp1jd8$p4a$1@sea.gmane.org>	<43B4A238.2030101@python.org>	<dp2qt7$dib$1@sea.gmane.org>
	<43B55FE8.1040401@python.org>
Message-ID: <43B58F09.6090803@colorstudy.com>

David Goodger wrote:
>>The problem is that the WORKFLOW doesn't work.
> 
> 
> So fix the workflow.  Something like Ian Bicking's Commentary system,
> or something more specific to Python's docs, seems to fit the bill.

I'll just note that Commentary works on any HTML, so it doesn't matter 
what the original source is written in.

Obviously it is better if the markup of submitted comments match the 
original format; but it's unreasonable to accept LaTeX input in comments 
and render that to HTML for inline display.  When I have a chance I want 
to add ReST input, since some level of markup in comments is really 
called for, and I prefer not to create Yet Another Wikiish Markup (and I 
like ReST).  But manually translating ReST to LaTeX when integrating 
comments is no harder than any other manual translation, and people have 
indicated they are willing to do that translation.

Because Commentary writes to flat files, workflow should be fairly 
simple -- you submit most updates as comments.  Maybe a login or CAPTCHA 
should be added to avoid spam flooding.  rel=nofollow on links is a 
no-brainer as well (since unlike a Wiki these are relatively transient 
bits of content, so you aren't trapping link information forever by 
putting this on all links), but nofollow isn't likely itself to stop spam.

Anyway, submitted comments can be edited by hand through a text editor, 
alongside the original documentation, or through the web interface 
(right now anyone can delete a comment, but we could restrict that to 
documentation maintainers).  If the documentation contains good ids 
(that can be traced back to the original source) I think it is 
reasonable to do the entire process using the text files (comments are 
marked by their relation to a nearby id) -- but the current 
documentation has rather meaningless ids, so this might not work.

The flat files can also go in Subversion, with useful diffs and complete 
history, living safely in the same repository as the documentation or in 
a separate repository.  Comments can be branched or dumped or whatever, 
with a workflow parallel to the documentation.  Notification and 
tracking tools already exist for Subversion; additional feeds and 
whatnot could also certainly be built for Commentary, but it's not clear 
that is necessary.

Commentary is still rough in places, but it's pretty much orthogonal to 
all the other parts of the documentation system, so it doesn't require 
any big investment or conversion process.  Heck, it doesn't even require 
buy-in by anyone (except, perhaps, at least one person to generate 
comments and one person to consume them), though at some point if it 
works well it would be useful to link it from the official 
documentation.  But I think it can be useful in a non-official state as 
well.

Right now most people who might be willing to contribute to the Python 
documentation don't.  Well, "most don't" is an understatement.  "Hardly 
any" would be more accurate.  If just a small portion of people could 
contribute fairly easily that would be a big step forward.

Anyway, another even more expedient option would be setting up a 
separate bug tracker (something simpler to submit to than SF) and 
putting a link on the bottom of every page, maybe like: 
http://trac.python.org/trac/newticket?summary=re:+/path/to/doc&component=docs 
-- heck, we all know SF bug tracking sucks, this is a good chance to 
experiment with a different tracker, and documentation has softer 
requirements other parts of Python.


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From shane at hathawaymix.org  Fri Dec 30 21:03:31 2005
From: shane at hathawaymix.org (Shane Hathaway)
Date: Fri, 30 Dec 2005 13:03:31 -0700
Subject: [Python-Dev] [Doc-SIG] that library reference, again
In-Reply-To: <43B58F09.6090803@colorstudy.com>
References: <dope79$flv$2@sea.gmane.org><dp0qrh$r78$1@sea.gmane.org>	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com><dp1jd8$p4a$1@sea.gmane.org>	<43B4A238.2030101@python.org>	<dp2qt7$dib$1@sea.gmane.org>	<43B55FE8.1040401@python.org>
	<43B58F09.6090803@colorstudy.com>
Message-ID: <43B59293.20506@hathawaymix.org>

Ian Bicking wrote:
> Right now most people who might be willing to contribute to the Python 
> documentation don't.  Well, "most don't" is an understatement.  "Hardly 
> any" would be more accurate.  If just a small portion of people could 
> contribute fairly easily that would be a big step forward.

+1.  I've often wanted to contribute some small patch to the docs, but 
then went back to work and forgot my patch.

Shane

From radeex at gmail.com  Sat Dec 31 03:29:34 2005
From: radeex at gmail.com (Christopher Armstrong)
Date: Sat, 31 Dec 2005 13:29:34 +1100
Subject: [Python-Dev] [Doc-SIG] that library reference, again
In-Reply-To: <51A7717B-3832-477E-BED5-3C36DCA20336@lag.net>
References: <dope79$flv$2@sea.gmane.org> <dp0qrh$r78$1@sea.gmane.org>
	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com>
	<dp1jd8$p4a$1@sea.gmane.org> <43B4A238.2030101@python.org>
	<51A7717B-3832-477E-BED5-3C36DCA20336@lag.net>
Message-ID: <60ed19d40512301829q7a0df01bv3ed782fe28cbcf7a@mail.gmail.com>

On 12/30/05, Robey Pointer <robey at lag.net> wrote:
>
> On 29 Dec 2005, at 18:58, David Goodger wrote:
>
> > [Fredrik Lundh]
> >>>> I'm beginning to fear that I've wasted my time on a project
> >>>> that nobody's interested in.
> >
> > [David Goodger]
> >>> Could be. I don't see HTML+PythonDoc as a significant improvement
> >>> over LaTeX.
> >
> > [Fredrik Lundh]
> >> Really?
> >
> > Yes, really.
>
> Just out of curiosity (really -- not trying to jump into the flames)
> why not just use epydoc?  If it's good enough for 3rd-party python
> libraries, isn't that just a small step from being good enough for
> the builtin libraries?

It's not really even "good enough" for a lot of my usage without some
seriously evil hacks. The fundamental design decision of epydoc to
import code, plus some other design decisions on the way it figures
types and identity seriously hinder its utility. Ever notice how
trying to document your third-party-library-using application will
*also* document that third party library, for example? Or how it'll
blow up when you're trying to document your gtk-using application on a
remote server without an X server running? Or how it just plain blows
right up with most Interface systems? etc.

--
  Twisted   |  Christopher Armstrong: International Man of Twistery
   Radix    |    -- http://radix.twistedmatrix.com
            |  Release Manager, Twisted Project
  \\\V///   |    -- http://twistedmatrix.com
   |o O|    |
w----v----w-+

From ncoghlan at gmail.com  Sat Dec 31 06:32:36 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 31 Dec 2005 15:32:36 +1000
Subject: [Python-Dev] Gaming on Sunday (Jan 1)
In-Reply-To: <43B5F905.3070202@iinet.net.au>
References: <43B533FD.8060809@iinet.net.au> <43B5F905.3070202@iinet.net.au>
Message-ID: <43B617F4.1070300@gmail.com>

Grant Crawley wrote:
> no worries....I assume that we will be gaming till somewhat late?

Well, Monday's a public holiday, so. . .


Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sat Dec 31 06:41:50 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 31 Dec 2005 15:41:50 +1000
Subject: [Python-Dev] [Doc-SIG] that library reference, again
In-Reply-To: <43B58F09.6090803@colorstudy.com>
References: <dope79$flv$2@sea.gmane.org><dp0qrh$r78$1@sea.gmane.org>	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com><dp1jd8$p4a$1@sea.gmane.org>	<43B4A238.2030101@python.org>	<dp2qt7$dib$1@sea.gmane.org>	<43B55FE8.1040401@python.org>
	<43B58F09.6090803@colorstudy.com>
Message-ID: <43B61A1E.9080300@gmail.com>

Ian Bicking wrote:
> Anyway, another even more expedient option would be setting up a 
> separate bug tracker (something simpler to submit to than SF) and 
> putting a link on the bottom of every page, maybe like: 
> http://trac.python.org/trac/newticket?summary=re:+/path/to/doc&component=docs 
> -- heck, we all know SF bug tracking sucks, this is a good chance to 
> experiment with a different tracker, and documentation has softer 
> requirements other parts of Python.

While I quite like this idea, would it make it more difficult when the bug 
tracking for the main source code is eventually migrated off SF? And what 
would happen to existing documentation bug reports/patches on the SF trackers?

Is it possible to do something similar for the online version of the current 
docs, simply pointing them at the SF tracker? (I know this doesn't help people 
without an SF account. . .)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sat Dec 31 06:47:23 2005
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 31 Dec 2005 15:47:23 +1000
Subject: [Python-Dev] Gaming on Sunday (Jan 1)
In-Reply-To: <43B617F4.1070300@gmail.com>
References: <43B533FD.8060809@iinet.net.au> <43B5F905.3070202@iinet.net.au>
	<43B617F4.1070300@gmail.com>
Message-ID: <43B61B6B.9030603@gmail.com>

Sorry about that folks - finger trouble in the mail client ;)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From noamraph at gmail.com  Sat Dec 31 18:15:54 2005
From: noamraph at gmail.com (Noam Raphael)
Date: Sat, 31 Dec 2005 19:15:54 +0200
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <b348a0850512281522r1c4f8a01v4288a5e11586b33d@mail.gmail.com>
References: <b348a0850512281522r1c4f8a01v4288a5e11586b33d@mail.gmail.com>
Message-ID: <b348a0850512310915y3c659739ue5e541631bb8eb7c@mail.gmail.com>

Hello,

I thought about another reason to resize the hash table when it has
too few elements. It's not only a matter of memory usage, it's also a
matter of time usage: iteration over a set or a dict requires going
over all the table. For example, iteration over a set which once had
1,000,000 members and now has 2 can take 1,000,000 operations every
time you traverse all the (2) elements.

Apologies:
1. It may be trivial to you - I'm sorry, I thought about it just now.
2. You can, of course, still do whatever tradeoff you like.

Noam

From lac at strakt.com  Sat Dec 31 18:12:52 2005
From: lac at strakt.com (Laura Creighton)
Date: Sat, 31 Dec 2005 18:12:52 +0100
Subject: [Python-Dev] [Doc-SIG]  that library reference, again
In-Reply-To: Message from Nick Coghlan <ncoghlan@gmail.com> 
	of "Sat, 31 Dec 2005 15:41:50 +1000." <43B61A1E.9080300@gmail.com> 
References: <dope79$flv$2@sea.gmane.org><dp0qrh$r78$1@sea.gmane.org>
	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com><dp1jd8$p4a$1@sea.gmane.org>
	<43B4A238.2030101@python.org> <dp2qt7$dib$1@sea.gmane.org>
	<43B55FE8.1040401@python.org> <43B58F09.6090803@colorstudy.com>
	<43B61A1E.9080300@gmail.com> 
Message-ID: <200512311712.jBVHCqPD025298@theraft.strakt.com>

In a message of Sat, 31 Dec 2005 15:41:50 +1000, Nick Coghlan writes:
>Ian Bicking wrote:
>> Anyway, another even more expedient option would be setting up a 
>> separate bug tracker (something simpler to submit to than SF) and 
>> putting a link on the bottom of every page, maybe like: 
>> http://trac.python.org/trac/newticket?summary=re:+/path/to/doc&componen
>t=docs 
>> -- heck, we all know SF bug tracking sucks, this is a good chance to 
>> experiment with a different tracker, and documentation has softer 
>> requirements other parts of Python.
>
>While I quite like this idea, would it make it more difficult when the bu
>g 
>tracking for the main source code is eventually migrated off SF? And what
> 
>would happen to existing documentation bug reports/patches on the SF trac
>kers?
>
>Is it possible to do something similar for the online version of the curr
>ent 
>docs, simply pointing them at the SF tracker? (I know this doesn't help p
>eople 
>without an SF account. . .)
>
>Cheers,
>Nick.

Not if the problem is that documentation changes are not 'patches' and
'bugs' and the sourceforge bug tracker, which isn't even particularly
good at tracking bugs is particularly ill-suited for the collaborative
sharing of documents.

Laura


From raymond.hettinger at verizon.net  Sat Dec 31 21:52:39 2005
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sat, 31 Dec 2005 15:52:39 -0500
Subject: [Python-Dev] When do sets shrink?
Message-ID: <002f01c60e4c$23b97d60$2827a044@oemcomputer>

[Noam]
> For example, iteration over a set which once had
> 1,000,000 members and now has 2 can take 1,000,000 operations every
> time you traverse all the (2) elements.

Do you find that to be a common or plausible use case?

Was Guido's suggestion of s=set(s) unworkable for some reason?  dicts
and sets emphasize fast lookups over fast iteration -- apps requiring
many iterations over a collection may be better off converting to a list
(which has no dummy entries or empty gaps between entries).

Would the case be improved by incurring the time cost of 999,998 tests
for possible resizing (one for each pop) and some non-trivial number of
resize operations along the way (each requiring a full-iteration over
the then current size)?

Even if this unique case could be improved, what is the impact on common
cases?  Would a downsizing scheme risk thrashing with the
over-allocation scheme in cases with mixed adds and pops?

Is there any new information/research beyond what has been obvious from
the moment the dict resizing scheme was born?



Raymond


From ianb at colorstudy.com  Sat Dec 31 23:05:22 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Sat, 31 Dec 2005 16:05:22 -0600
Subject: [Python-Dev] [Doc-SIG]   that library reference, again
In-Reply-To: <43B61A1E.9080300@gmail.com>
References: <dope79$flv$2@sea.gmane.org><dp0qrh$r78$1@sea.gmane.org>	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com><dp1jd8$p4a$1@sea.gmane.org>	<43B4A238.2030101@python.org>	<dp2qt7$dib$1@sea.gmane.org>	<43B55FE8.1040401@python.org>	<43B58F09.6090803@colorstudy.com>
	<43B61A1E.9080300@gmail.com>
Message-ID: <43B700A2.70701@colorstudy.com>

Nick Coghlan wrote:
>>Anyway, another even more expedient option would be setting up a 
>>separate bug tracker (something simpler to submit to than SF) and 
>>putting a link on the bottom of every page, maybe like: 
>>http://trac.python.org/trac/newticket?summary=re:+/path/to/doc&component=docs 
>>-- heck, we all know SF bug tracking sucks, this is a good chance to 
>>experiment with a different tracker, and documentation has softer 
>>requirements other parts of Python.
> 
> 
> While I quite like this idea, would it make it more difficult when the bug 
> tracking for the main source code is eventually migrated off SF? And what 
> would happen to existing documentation bug reports/patches on the SF trackers?

I think the requirements for documentation are a bit lighter, so it's 
not as big a deal.  E.g., the history of bug reports on documentation 
isn't as important, either the ones from SF, or if all of Python moves 
to a new system then the history of the transitional system. 
Documentation is mostly self-describing.

> Is it possible to do something similar for the online version of the current 
> docs, simply pointing them at the SF tracker? (I know this doesn't help people 
> without an SF account. . .)

Perhaps; I haven't played with the SF interface at all, so I don't know 
if you can prefill fields.  But it's still a pain, since logging into SF 
isn't seemless (since you don't get redirected back to where you started 
from).  Also, I don't know if the requirements for documentation match 
the code generally.  Being able to follow up on documentation bugs isn't 
as important, so if you don't always collect the submitters email 
address it's not that big a deal.  Doc maintainers may be willing to 
filter through a bit more spam if it means that they get more 
submissions, and so forth.  The review process probably isn't as 
important.  So I think it could be argued that code and documentation 
shouldn't even be on the same tracker.  (I'm not really arguing that, 
but at least it doesn't seem like a big a deal if they aren't on the 
same system)

-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org