From fred at zope.com  Thu Jul  1 00:01:48 2004
From: fred at zope.com (Fred Drake)
Date: Thu Jul  1 00:02:02 2004
Subject: [Python-Dev] making weakref.ref objects subclassable
Message-ID: <200407010001.48784.fred@zope.com>

On Wednesday 30 June 2004 11:46 pm, Tim Peters wrote:
 > Well, there is no change to gcmodule.c, except for the indirect change in
 > what the PyWeakref_Check() macro expands to.  The risks with weakrefs

The change in the macro expansion is exactly what I was referring to; not a 
source change, but a real change none the less.

 > For reasons given above, I believe gc performance will be virtually
 > unchanged for almost all programs.

That was my conclusion as well.  Empirical evidence with the Zope 3 test suite 
suggests that's right.  If anyone has an application that would be 
meaningfully affected by this change, I'd be rather surprised, and would 
appreciate hearing about it.


  -Fred

-- 
Fred L. Drake, Jr.  <fred at zope.com>
Zope Corporation


From tim.one at comcast.net  Thu Jul  1 00:12:27 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu Jul  1 00:12:33 2004
Subject: [Python-Dev] Version 3 Proposal: thread-local data
Message-ID: <mailman.32.1088655153.9339.python-dev@python.org>

[Jim]
...
>         def __del__(self):
>             key = object.__getattribute__(self, '_local__key')
>             for thread in enumerate():
>                 if key in thread.__dict__:
>                     del thread.__dict__[key]

Note that a __del__ method should never reference a module global
"enumerate" in this case) -- it's all but certain to lead to "iteration over
non-sequence" Mystery Errors at Python shutdown time (due to module globals
getting "None'd out").  The conventional workaround is to give the class an
attribute initialized from the module global; e.g.,

class local(_localbase):
    _enumerate = threading.enumerate

    ...

    def __del__(self):
        ...
        for thread in self._enumerate():

The conventional workaround that doesn't work is to spell that

        for thread in local._enumerate():

instead; it doesn't work because "local" is also a module global.



From anthony at interlink.com.au  Thu Jul  1 00:13:50 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Jul  1 00:14:46 2004
Subject: [Python-Dev] Including translated doc strings in 2.4
In-Reply-To: <200406301509.56390.stephan.richter@tufts.edu>
References: <40D5C17C.2010008@v.loewis.de> <40E26254.6000903@interlink.com.au>
	<1088605153.7378.84.camel@localhost>
	<200406301509.56390.stephan.richter@tufts.edu>
Message-ID: <40E38F7E.80407@interlink.com.au>

Stephan Richter wrote:
> The standard directory for gettext files is "locales".


"locales", however, is a valid package name. I'd much prefer
that something going into lib/python2.4/ that is _not_ a package,
not have a name that _could_ be a package. And yes, I'm aware
that the very very old 'config' directory violates this rule -
but I don't think we can move that without breaking all the
old Makefile.pre.in-based installation scripts.


From tim.one at comcast.net  Thu Jul  1 00:15:52 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu Jul  1 00:15:59 2004
Subject: [Python-Dev] making weakref.ref objects subclassable
In-Reply-To: <200407010001.48784.fred@zope.com>
Message-ID: <mailman.33.1088655359.9339.python-dev@python.org>

[Tim]
>> For reasons given above, I believe gc performance will be virtually
>> unchanged for almost all programs.

[Fred Drake]
> That was my conclusion as well.  Empirical evidence with the Zope 3 test
> suite suggests that's right.  If anyone has an application that would be
> meaningfully affected by this change, I'd be rather surprised, and would
> appreciate hearing about it.

For the reasons given before, that won't happen.  The analysis I gave was
based on a complete understanding of every gory detail of how Python's gc
works -- there's simply nothing risky about this change wrt gc performance.
Empirical validation is unnecessary <wink>.

If people want to surprise us, better they spend time writing nasty new
tests of weakref subclassing!



From ncoghlan at iinet.net.au  Thu Jul  1 00:24:08 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu Jul  1 00:24:14 2004
Subject: [Python-Dev] PEP 276 (simple iterator for ints)
In-Reply-To: <200407010217.i612HFU5007069@cosc353.cosc.canterbury.ac.nz>
References: <200407010217.i612HFU5007069@cosc353.cosc.canterbury.ac.nz>
Message-ID: <40E391E8.5020104@iinet.net.au>

Greg Ewing wrote:

> "Chermside, Michael" <mchermside@ingdirect.com>:
> 
> 
>>    >>> for i in len(myList):
>>    ...     doSomethingWithIndexes(i)
>>
>>is simple and elegant.
> 
> 
> IMO it would be clearer, and equally elegant, to write
> this as something like
> 
>   for i in indices(myList):
>     ...
> 
> which is easily accomplished with the aid of a suitable
> definition for indices(). No language changes needed.

And I believe most of those use cases are better covered by enumerate() 
anyway.

Interesting that in the course of this discussion I've gone from 'nice 
idea' to 'no way!'

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268

From tdelaney at avaya.com  Thu Jul  1 00:28:54 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Thu Jul  1 00:29:00 2004
Subject: [Python-Dev] PEP 276 (simple iterator for ints)
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01A45CEB@au3010avexu1.global.avaya.com>

Nick Coghlan wrote:

> Interesting that in the course of this discussion I've gone from 'nice
> idea' to 'no way!'

That's exactly what happened to me when the PEP was first proposed. On
the face of it, it's a nice, simple, easily-understood thing. However,
when you think about the use cases, it starts looking more like a hack.

Tim Delaney

From anthony at interlink.com.au  Thu Jul  1 01:11:07 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Jul  1 01:11:42 2004
Subject: [Python-Dev] making weakref.ref objects subclassable
In-Reply-To: <200407010416.i614GFu04495@valdez.interlink.com.au>
References: <200407010416.i614GFu04495@valdez.interlink.com.au>
Message-ID: <40E39CEB.7040203@interlink.com.au>

Tim Peters wrote:
> For the reasons given before, that won't happen.  The analysis I gave was
> based on a complete understanding of every gory detail of how Python's gc
> works -- there's simply nothing risky about this change wrt gc performance.
> Empirical validation is unnecessary <wink>.
> 
> If people want to surprise us, better they spend time writing nasty new
> tests of weakref subclassing!

Why can I forsee the strings "garbage collector" and "subclasses of
weakrefs", along with "discovered in Zope3", appearing in the 2.4.1 
2.4.2 and 2.4.3 release announcements? <0.5 wink>

Anthony

From gmccaughan at synaptics-uk.com  Thu Jul  1 04:01:39 2004
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Thu Jul  1 04:02:14 2004
Subject: [Python-Dev] PEP 276 (simple iterator for ints)
In-Reply-To: <200407010217.i612HFU5007069@cosc353.cosc.canterbury.ac.nz>
References: <200407010217.i612HFU5007069@cosc353.cosc.canterbury.ac.nz>
Message-ID: <200407010901.39880.gmccaughan@synaptics-uk.com>

On Thursday 2004-07-01 03:17, Greg Ewing wrote:

> I suspect most people other than number theorists would
> find the concept of a set of integers being contained
> in another integer quite wierd.

<pedant type="mathematician">
Actually, most number theorists would too. It's only
set theorists to whom that seems like a natural definition.
And the first (so far as I know) attempt to define the
natural numbers set-theoretically did it quite differently:
Gottlob Frege proposed to define n as the set of all
sets of size n. Unfortunately, Frege's version of set
theory famously didn't work (it was demolished by Russell's
paradox) and his construction doesn't work in ZF set theory,
which is what most set theorists use nowadays.

In any case, saying that (e.g.) 3 is a member of 17
is generally regarded as an implementation detail.
Mathematicians understand the different between interface
and implementation just about as well as software people.
(Which is to say: some understand it very well, and others
don't.)

By the by, I don't think it's true that sets.Set(42)
would, under PEP276, be the usual implementation of 42
in set theory. It would be a set whose elements are
ordinary Python integers, which are not themselves sets.
If you see what I mean. :-)
</pedant>

-- 
g



From tim.one at comcast.net  Thu Jul  1 04:31:24 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu Jul  1 04:31:36 2004
Subject: [Python-Dev] making weakref.ref objects subclassable
In-Reply-To: <40E39CEB.7040203@interlink.com.au>
Message-ID: <mailman.34.1088670696.9339.python-dev@python.org>

[Anthony Baxter]
> Why can I forsee the strings "garbage collector" and "subclasses of
> weakrefs", along with "discovered in Zope3", appearing in the 2.4.1 2.4.2
> and 2.4.3 release announcements? <0.5 wink>

Because you've recently seen the strings "garbage collector" and "subclasses
of weakrefs", and suffer a compulsion to parrot them <wink>?  If so, review
the patch instead.  If there's a bug in the patch, it's probably already
there.



From pedronis at bluewin.ch  Thu Jul  1 06:30:55 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Thu Jul  1 06:29:58 2004
Subject: [Python-Dev] Version 3 Proposal: thread-local data
In-Reply-To: <40E32858.2020505@zope.com>
References: <40E03020.7010101@zope.com>
 <40E03020.7010101@zope.com>
Message-ID: <5.2.1.1.0.20040701122853.021b2710@pop.bluewin.ch>

At 16:53 30.06.2004 -0400, Jim Fulton wrote:

>     def _patch(self):
>         key = object.__getattribute__(self, '_local__key')
>         d = currentThread().__dict__.get(key)
>         if d is None:
>             d = {}
>             currentThread().__dict__[key] = d
>             object.__setattr__(self, '__dict__', d)
>
>             # we have a new instance dict, so call out __init__ if we have
>             # one
>             cls = type(self)
>             if cls.__init__ is not object.__init__:
>                 args, kw = object.__getattribute__(self, '_local__args')
>                 cls.__init__(self, *args, **kw)
>         else:
>             object.__setattr__(self, '__dict__', d)
>
>     class local(_localbase):
>
>         def __getattribute__(self, name):
>             _patch(self)

what happens if there's a thread switch, to another thread using the local 
object, here? between patching __dict__ and accessing __dict__


>             return object.__getattribute__(self, name)
>
>         def __setattr__(self, name, value):
>             _patch(self)
>             return object.__setattr__(self, name, value)
>
>         def __delattr__(self, name):
>             _patch(self)
>             return object.__delattr__(self, name)
>
>         def __del__(self):
>             key = object.__getattribute__(self, '_local__key')
>             for thread in enumerate():
>                 if key in thread.__dict__:
>                     del thread.__dict__[key]
>
>Any objections? Any more tricks up your sleeve Armin? :)
>
>Jim
>
>--
>Jim Fulton           mailto:jim@zope.com       Python Powered!
>CTO                  (540) 361-1714            http://www.python.org
>Zope Corporation     http://www.zope.com       http://www.zope.org
>
>_______________________________________________
>Python-Dev mailing list
>Python-Dev@python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: 
>http://mail.python.org/mailman/options/python-dev/pedronis%40bluewin.ch


From mal at egenix.com  Thu Jul  1 07:42:46 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Jul  1 07:42:55 2004
Subject: [Python-Dev] Re: [Python-checkins] python/nondist/sandbox/decimal
 Decimal.py, 1.37,	1.38
In-Reply-To: <E1Bfypx-0005xM-1A@sc8-pr-cvs1.sourceforge.net>
References: <E1Bfypx-0005xM-1A@sc8-pr-cvs1.sourceforge.net>
Message-ID: <40E3F8B6.6010403@egenix.com>

rhettinger@users.sourceforge.net wrote:
> Update of /cvsroot/python/python/nondist/sandbox/decimal
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22541
> 
> Modified Files:
> 	Decimal.py 
>   #       rename Decimal.py and test_Decimal.py to lowercase decimal
>   #       retarget the test directory to decimaltestdata

Shouldn't these steps be applied *before* adding it to the
core lib ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 01 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From amk at amk.ca  Thu Jul  1 07:58:09 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Thu Jul  1 07:58:25 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Doc/whatsnew
	whatsnew24.tex, 1.55, 1.56
In-Reply-To: <E1Bg06z-0001Qv-Qt@sc8-pr-cvs1.sourceforge.net>
References: <E1Bg06z-0001Qv-Qt@sc8-pr-cvs1.sourceforge.net>
Message-ID: <20040701115809.GA22778@rogue.amk.ca>

On Thu, Jul 01, 2004 at 04:52:17AM -0700, rhettinger@users.sourceforge.net wrote:
> Modified Files:
> 	whatsnew24.tex 
> Log Message:
> Move Decimal from the sandbox into production.

Um, I'd really like to write my own text for this document.  Please
don't go to the effort of writing "What's New" sections.

--amk

From mal at egenix.com  Thu Jul  1 08:11:09 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Jul  1 08:11:13 2004
Subject: [Python-Dev] Re: [Python-checkins] python/nondist/sandbox/decimal
	Decimal.py, 1.37,	1.38
In-Reply-To: <00a501c45efd$f9c9b820$e841fea9@oemcomputer>
References: <00a501c45efd$f9c9b820$e841fea9@oemcomputer>
Message-ID: <40E3FF5D.4050807@egenix.com>

Raymond Hettinger wrote:
>>rhettinger@users.sourceforge.net wrote:
>>
>>>Update of /cvsroot/python/python/nondist/sandbox/decimal
>>>In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22541
>>>
>>>Modified Files:
>>>	Decimal.py
>>>  #       rename Decimal.py and test_Decimal.py to lowercase decimal
>>>  #       retarget the test directory to decimaltestdata
>>
>>Shouldn't these steps be applied *before* adding it to the
>>core lib ?
> 
> Done *during* the move to the core.
> It should all be fine now.

Perfect :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 01 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From jim at zope.com  Thu Jul  1 09:33:16 2004
From: jim at zope.com (Jim Fulton)
Date: Thu Jul  1 09:33:21 2004
Subject: [Python-Dev] Revised Proposal: thread.get_dict
In-Reply-To: <5.2.1.1.0.20040630214302.03170848@pop.bluewin.ch>
References: <40E2CD1A.8080602@zope.com> <40E03020.7010101@zope.com>
	<40E2CD1A.8080602@zope.com>
	<5.2.1.1.0.20040630214302.03170848@pop.bluewin.ch>
Message-ID: <40E4129C.7090904@zope.com>

Samuele Pedroni wrote:
...
> 
> I'm puzzled, is this thread-safe?

Nope, but that's easily fixed.  Thanks for pointing it out.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From jim at zope.com  Thu Jul  1 09:34:46 2004
From: jim at zope.com (Jim Fulton)
Date: Thu Jul  1 09:34:49 2004
Subject: [Python-Dev] Version 3 Proposal: thread-local data
In-Reply-To: <5.2.1.1.0.20040701122853.021b2710@pop.bluewin.ch>
References: <40E03020.7010101@zope.com> <40E03020.7010101@zope.com>
	<5.2.1.1.0.20040701122853.021b2710@pop.bluewin.ch>
Message-ID: <40E412F6.5080102@zope.com>

Samuele Pedroni wrote:
> At 16:53 30.06.2004 -0400, Jim Fulton wrote:
> 
...
> 
> what happens if there's a thread switch, to another thread using the 
> local object, here? between patching __dict__ and accessing __dict__

That's a problem, which is why I've added the use of a reentrant lock in
__getattribute__, __setattr__ and __delattr__.  I'll make an updated
version available soon.

Thanks.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From mwh at python.net  Thu Jul  1 10:36:52 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Jul  1 10:36:57 2004
Subject: [Python-Dev] SHA-256 module
In-Reply-To: <5.2.0.9.0.20040630082845.0294ecf0@pop.sbcglobal.yahoo.com>
	(Trevor Perrin's message of "Wed, 30 Jun 2004 09:33:18 -0700")
References: <5.2.0.9.0.20040629125948.0292a5a0@pop.sbcglobal.yahoo.com>
	<1088534421.27300.153.camel@localhost>
	<005001c45da1$cfb9e1c0$e841fea9@oemcomputer>
	<1088534421.27300.153.camel@localhost>
	<5.2.0.9.0.20040629125948.0292a5a0@pop.sbcglobal.yahoo.com>
	<5.2.0.9.0.20040630082845.0294ecf0@pop.sbcglobal.yahoo.com>
Message-ID: <2msmcb3ibf.fsf@starship.python.net>

Trevor Perrin <trevp@trevp.net> writes:

> At 10:15 AM 6/30/2004 +0100, Michael Hudson wrote:
>>Unfortunately, distributing crypto software is still a hideous
>>international mess (just because the *US* is less silly these
>>days...).
>
> Things have been liberalizing rapidly.  I'm not sure how true that is
> anymore, though I don't have direct experience (aside from offering
> some crypto software on a website; people download it from all over
> the place, but maybe they're scofflaws, who knows).
>
> I know US export is no problem.  According to [1], most countries have
> no laws restricting imports, with the notable exception of ex-USSR
> countries and China, which require licenses.  I've heard anecdotally
> the Russian requirements are mostly ignored [2].  I don't know about
> China.

Interesting, thanks.  The concept of Python being 'imported' or
'exported' still strikes me as meaningless, but I'm not going to worry
about it.

Cheers,
mwh

-- 
  Many of the posts you see on Usenet are actually from moths.  You
  can tell which posters they are by their attraction to the flames.
                                      -- Internet Oracularity #1279-06

From mwh at python.net  Thu Jul  1 10:43:16 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Jul  1 10:43:20 2004
Subject: [Python-Dev] making weakref.ref objects subclassable
In-Reply-To: <200406301705.31590.fdrake@acm.org> (Fred L. Drake, Jr.'s
	message of "Wed, 30 Jun 2004 17:05:30 -0400")
References: <200406301705.31590.fdrake@acm.org>
Message-ID: <2mlli33i0r.fsf@starship.python.net>

"Fred L. Drake, Jr." <fdrake@acm.org> writes:

> I'm planning to commit a patch (http://www.python.org/sf/983019) that makes 
> weak references subclassable.  This includes the following changes:

Would "why?" be an apposite question?  I'm sure there is a reason, but
I haven't seen it yet :-)

Cheers,
mwh

-- 
  incidentally, asking why things are "left out of the language" is
  a good sign that the asker is fairly clueless.
                                        -- Erik Naggum, comp.lang.lisp

From barry at python.org  Thu Jul  1 10:53:59 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Jul  1 10:53:45 2004
Subject: [Python-Dev] SHA-256 module
In-Reply-To: <1f7befae040630195552738a9d@mail.gmail.com>
References: <5.2.0.9.0.20040629125948.0292a5a0@pop.sbcglobal.yahoo.com>
	<1088534421.27300.153.camel@localhost>
	<005001c45da1$cfb9e1c0$e841fea9@oemcomputer>
	<1088534421.27300.153.camel@localhost>
	<5.2.0.9.0.20040629125948.0292a5a0@pop.sbcglobal.yahoo.com>
	<5.2.0.9.0.20040630082845.0294ecf0@pop.sbcglobal.yahoo.com>
	<20040630225214.GW11545@zot.electricrain.com>
	<1f7befae040630195552738a9d@mail.gmail.com>
Message-ID: <1088693639.7381.73.camel@localhost>

On Wed, 2004-06-30 at 22:55, Tim Peters wrote:

> Improving this story.would probably require a volunteer who actually
> knows something about BDB.

And cared about Windows <wink>.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040701/b7c389b7/attachment.bin
From fdrake at acm.org  Thu Jul  1 11:16:28 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Jul  1 11:16:38 2004
Subject: [Python-Dev] making weakref.ref objects subclassable
In-Reply-To: <2mlli33i0r.fsf@starship.python.net>
References: <200406301705.31590.fdrake@acm.org>
	<2mlli33i0r.fsf@starship.python.net>
Message-ID: <200407011116.28561.fdrake@acm.org>

On Thursday 01 July 2004 10:43 am, Michael Hudson wrote:
 > Would "why?" be an apposite question?  I'm sure there is a reason, but
 > I haven't seen it yet :-)

Absolutely!  I hinted at the reason in my original email, but it was not made 
explicit that it was the reason.

The change allows the WeakValueDictionary to be implemented in a much more 
memory-efficient manner.  The current implementation causes each entry to 
create and retain the following objects in addition to the actual key:

- a weakref to the value, with a callback

- a function object used as the callback; a new function is created for every 
entry, since it contains a reference to the key in a default parameter value

- a weakref to the WeakValueDictionary itself, to avoid creating a cycle.

The last object could be removed without any change to the C implementation; 
all that's really needed is a single weakref to the WeakValueDictionary that 
can be shared by all such callbacks.

Using a subclass of the weakref.ref type (KeyedRef in the patch), the key can 
be stored as an attribute of the weakref, allowing a single object to be 
created and retained as overhead (with the allocation for that object being 
at most 8 bytes larger than a stock weakref).


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>


From barry at python.org  Thu Jul  1 11:21:20 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Jul  1 11:21:06 2004
Subject: [Python-Dev] Including translated doc strings in 2.4
In-Reply-To: <40E38F7E.80407@interlink.com.au>
References: <40D5C17C.2010008@v.loewis.de> <40E26254.6000903@interlink.com.au>
	<1088605153.7378.84.camel@localhost>
	<200406301509.56390.stephan.richter@tufts.edu>
	<40E38F7E.80407@interlink.com.au>
Message-ID: <1088695280.7377.75.camel@localhost>

On Thu, 2004-07-01 at 00:13, Anthony Baxter wrote:

> "locales", however, is a valid package name. I'd much prefer
> that something going into lib/python2.4/ that is _not_ a package,
> not have a name that _could_ be a package.

+1.  What about stdlib-locales?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040701/c5801bbf/attachment.bin
From theller at python.net  Thu Jul  1 11:31:27 2004
From: theller at python.net (Thomas Heller)
Date: Thu Jul  1 11:31:34 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <40D1FC47.4010508@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Thu,
	17 Jun 2004 22:17:11 +0200")
References: <20040614011919.35A0D2EFF0@mail.egenix.com>
	<40D0AEAE.5000606@egenix.com> <40D1FC47.4010508@v.loewis.de>
Message-ID: <4qorsq0g.fsf@python.net>

"Martin v. L?wis" <martin@v.loewis.de> writes:

> M.-A. Lemburg wrote:
>> Are we going to have a VC7.1 binary of Python 2.4 for Windows ?
>> (Just asking whether it's time to finally open the VC7 shrink-
>>  wrap box ;-)
>
> Yes. I think it hasn't been decided whether Thomas will produce
> that, or yours truly.

Well, I would be much happier to build an installer with inno setup
instead of Martin's msi code, which I don't understand ;-(.

Are there really technical reasons the VC7.1 built Python binaries must
be distributed via msi technology or not?

Thomas


From dave at boost-consulting.com  Thu Jul  1 11:44:43 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Jul  1 11:44:56 2004
Subject: [Python-Dev] Re: PEP 276 (simple iterator for ints)
References: <200407010217.i612HFU5007069@cosc353.cosc.canterbury.ac.nz>
	<200407010901.39880.gmccaughan@synaptics-uk.com>
Message-ID: <u6597wx3o.fsf@boost-consulting.com>

Gareth McCaughan <gmccaughan@synaptics-uk.com> writes:

> By the by, I don't think it's true that sets.Set(42)
> would, under PEP276, be the usual implementation of 42
> in set theory. It would be a set whose elements are
> ordinary Python integers, which are not themselves sets.
> If you see what I mean. :-)

And that's what confused me most of all about Andy's assertion.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com


From tim.peters at gmail.com  Thu Jul  1 12:13:22 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Jul  1 12:13:27 2004
Subject: [Python-Dev] SHA-256 module
In-Reply-To: <1088693639.7381.73.camel@localhost>
References: <5.2.0.9.0.20040629125948.0292a5a0@pop.sbcglobal.yahoo.com>
	<1088534421.27300.153.camel@localhost>
	<005001c45da1$cfb9e1c0$e841fea9@oemcomputer>
	<1088534421.27300.153.camel@localhost>
	<5.2.0.9.0.20040629125948.0292a5a0@pop.sbcglobal.yahoo.com>
	<5.2.0.9.0.20040630082845.0294ecf0@pop.sbcglobal.yahoo.com>
	<20040630225214.GW11545@zot.electricrain.com>
	<1f7befae040630195552738a9d@mail.gmail.com>
	<1088693639.7381.73.camel@localhost>
Message-ID: <1f7befae040701091387b1313@mail.gmail.com>

[Tim Peters]
>> Improving this story.would probably require a volunteer who actually
>> knows something about BDB.

[Barry Warsaw]
> And cared about Windows <wink>.

Well, yes.  I assumed that everyone on python-dev is professional
enough to care about remaining employable <wink>.

From barry at python.org  Thu Jul  1 13:09:32 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Jul  1 13:09:17 2004
Subject: [Python-Dev] SHA-256 module
In-Reply-To: <1f7befae040701091387b1313@mail.gmail.com>
References: <5.2.0.9.0.20040629125948.0292a5a0@pop.sbcglobal.yahoo.com>
	<1088534421.27300.153.camel@localhost>
	<005001c45da1$cfb9e1c0$e841fea9@oemcomputer>
	<1088534421.27300.153.camel@localhost>
	<5.2.0.9.0.20040629125948.0292a5a0@pop.sbcglobal.yahoo.com>
	<5.2.0.9.0.20040630082845.0294ecf0@pop.sbcglobal.yahoo.com>
	<20040630225214.GW11545@zot.electricrain.com>
	<1f7befae040630195552738a9d@mail.gmail.com>
	<1088693639.7381.73.camel@localhost>
	<1f7befae040701091387b1313@mail.gmail.com>
Message-ID: <1088701771.7416.146.camel@localhost>

On Thu, 2004-07-01 at 12:13, Tim Peters wrote:
> [Tim Peters]
> >> Improving this story.would probably require a volunteer who actually
> >> knows something about BDB.
> 
> [Barry Warsaw]
> > And cared about Windows <wink>.
> 
> Well, yes.  I assumed that everyone on python-dev is professional
> enough to care about remaining employable <wink>.

Oh Tim, since you've gotten your gmail account, you've become such a
tool of The Man. :)

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040701/b6271373/attachment.bin
From pinard at iro.umontreal.ca  Thu Jul  1 13:37:16 2004
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Thu Jul  1 13:43:45 2004
Subject: [Python-Dev] Comparison and (user) confusion
Message-ID: <20040701173715.GA7389@titan.progiciels-bpi.ca>

Hi, people.

Yesterday, as a user, I made a little bug which I quickly resolved, but
over which I could have spent some time, having been younger to Python.

The problem appeared when I sub-classed `list', and used a `__cmp__'
method in my sub-type to effect a specialised sort.  The sort did not
behave as expected, as the `list' type provides rich comparisons, and
that `sort', finding `__lt__', merely ignored my own `__cmp__'.

I wonder if Python could not have helped better here, by diagnosing
a possible comparison confusion while creating the sub-class.  I do
not know much know Python internals, but if C slots happen to be used
for comparison methods, it might not be so unreasonable to expect such
confusion to be detected whenever those slots are filled.

I presume here, maybe wrongly, that it does not make good sense mixing
rich and non-rich comparisons in the same class.  Just a thought. :-)

P.S. - Of course, replacing `__cmp__' by `__lt__' solved my problem.

P.P.S. - This is Python 2.3.3 on Linux, in case it matters.

-- 
Fran?ois Pinard   http://www.iro.umontreal.ca/~pinard

From tim.one at comcast.net  Thu Jul  1 14:16:01 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu Jul  1 14:16:10 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <4qorsq0g.fsf@python.net>
Message-ID: <mailman.36.1088705770.9339.python-dev@python.org>

[Thomas Heller]
> Well, I would be much happier to build an installer with inno setup
> instead of Martin's msi code, which I don't understand ;-(.

So the choice is Martin's MSI code you don't understand, or your Inno
RemObjects Object Pascal code Martin doesn't understand <wink>?

> Are there really technical reasons the VC7.1 built Python binaries must
> be distributed via msi technology or not?

The most frequently requested thing MSI provides is the ability for admins
to use MS Group Policy to push installs out to boxes on a network.  This
hasn't been requested often ("most frequently" is a relative measure ...).



From theller at python.net  Thu Jul  1 14:30:17 2004
From: theller at python.net (Thomas Heller)
Date: Thu Jul  1 14:39:06 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <20040701181655.6375E2CEEEC@mail.ion-tof.com> (Tim Peters's
	message of "Thu, 1 Jul 2004 14:16:01 -0400")
References: <20040701181655.6375E2CEEEC@mail.ion-tof.com>
Message-ID: <7jtnr35y.fsf@python.net>

"Tim Peters" <tim.one@comcast.net> writes:

> [Thomas Heller]
>> Well, I would be much happier to build an installer with inno setup
>> instead of Martin's msi code, which I don't understand ;-(.
>
> So the choice is Martin's MSI code you don't understand, or your Inno
> RemObjects Object Pascal code Martin doesn't understand <wink>?

Probably I'm not smart enough to understand (or even maintain) Martin's
MSI code, but I guess Martin will be smart enough to understand a little
Pascal.

>> Are there really technical reasons the VC7.1 built Python binaries must
>> be distributed via msi technology or not?
>
> The most frequently requested thing MSI provides is the ability for admins
> to use MS Group Policy to push installs out to boxes on a network.  This
> hasn't been requested often ("most frequently" is a relative measure ...).

I was asking because I'm not sure if there will be problems with Windows
file system protection with non-MSI installers.

Thomas


From jim at zope.com  Thu Jul  1 15:12:44 2004
From: jim at zope.com (Jim Fulton)
Date: Thu Jul  1 15:12:51 2004
Subject: [Python-Dev] Version 3 Proposal: thread-local data
In-Reply-To: <40E412F6.5080102@zope.com>
References: <40E03020.7010101@zope.com> <40E03020.7010101@zope.com>
	<5.2.1.1.0.20040701122853.021b2710@pop.bluewin.ch>
	<40E412F6.5080102@zope.com>
Message-ID: <40E4622C.30904@zope.com>

Jim Fulton wrote:
...

> I'll make an updated
> version available soon.

The latest Python and C versions can be found at:

   http://svn.zope.org/Zope3/trunk/src/zope/thread/

and checked out with subversion from:

   svn://svn.zope.org/repos/main/Zope3/trunk/src/thread

See __init__.py for the Python version and _zope_thread.c.

I'll get some experience with these in Zope 3 over the next few days
and then rework them into the Python thread, threading and dummy_threading
modules as proposed early next week.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From fred at zope.com  Thu Jul  1 15:21:39 2004
From: fred at zope.com (Fred Drake)
Date: Thu Jul  1 15:21:49 2004
Subject: [Python-Dev] Version 3 Proposal: thread-local data
In-Reply-To: <40E4622C.30904@zope.com>
References: <40E03020.7010101@zope.com> <40E412F6.5080102@zope.com>
	<40E4622C.30904@zope.com>
Message-ID: <200407011521.39201.fred@zope.com>

On Thursday 01 July 2004 03:12 pm, Jim Fulton wrote:
 > and checked out with subversion from:
 >
 >    svn://svn.zope.org/repos/main/Zope3/trunk/src/thread

This might work better:

    svn://svn.zope.org/repos/main/Zope3/trunk/src/zope/thread

(There was a missing "zope/" in the URL.)


  -Fred

-- 
Fred L. Drake, Jr.  <fred at zope.com>
Zope Corporation


From ixokai at gmail.com  Thu Jul  1 16:24:34 2004
From: ixokai at gmail.com (IxokaI)
Date: Thu Jul  1 16:24:40 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <92871571195465397@unknownmsgid>
References: <92871571195465397@unknownmsgid>
Message-ID: <f8ef9ae404070113242e221106@mail.gmail.com>

> > Are there really technical reasons the VC7.1 built Python binaries must
> > be distributed via msi technology or not?
> 
> The most frequently requested thing MSI provides is the ability for admins
> to use MS Group Policy to push installs out to boxes on a network.  This
> hasn't been requested often ("most frequently" is a relative measure ...).

Can I jump in with a, "That is a *wonderful* thing" comment? We're
about to go into beta with a complete re-implementation of our
software in Python, and for our Windows clients, Group Policies will
make everything substantially easier.

So, I hope the build ends up using MSI. :)

--Stephen

From martin at v.loewis.de  Thu Jul  1 17:25:23 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul  1 17:25:29 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <4qorsq0g.fsf@python.net>
References: <20040614011919.35A0D2EFF0@mail.egenix.com>	<40D0AEAE.5000606@egenix.com>
	<40D1FC47.4010508@v.loewis.de> <4qorsq0g.fsf@python.net>
Message-ID: <40E48143.3020300@v.loewis.de>

Thomas Heller wrote:
> Well, I would be much happier to build an installer with inno setup
> instead of Martin's msi code, which I don't understand ;-(.

I would be willing to explain it if you would ask questions :-(
What is it that you don't understand about it?

> Are there really technical reasons the VC7.1 built Python binaries must
> be distributed via msi technology or not?

No. However, an MSI package allows distribution through group policy;
an Inno Setup binary doesn't. Also, MSI supports Win64, and Inno Setup
doesn't.

Regards,
Martin


From martin at v.loewis.de  Thu Jul  1 17:28:34 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul  1 17:28:38 2004
Subject: [Python-Dev] Including translated doc strings in 2.4
In-Reply-To: <200406301509.56390.stephan.richter@tufts.edu>
References: <40D5C17C.2010008@v.loewis.de>
	<40E26254.6000903@interlink.com.au>	<1088605153.7378.84.camel@localhost>
	<200406301509.56390.stephan.richter@tufts.edu>
Message-ID: <40E48202.4020901@v.loewis.de>

Stephan Richter wrote:
> The standard directory for gettext files is "locales".

In what sense is that "standard"? In a source distribution,
the standard directory is called "po". In an installed system,
the .mo files are in <prefix>/share/locale/<lang>/LC_MESSAGES.
However, that structure is not appropriate if you don't have
complete locales, but just message catalogs.

Regards,
Martin


From martin at v.loewis.de  Thu Jul  1 17:37:33 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul  1 17:37:38 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Doc/whatsnew
	whatsnew24.tex, 1.55, 1.56
In-Reply-To: <20040701115809.GA22778@rogue.amk.ca>
References: <E1Bg06z-0001Qv-Qt@sc8-pr-cvs1.sourceforge.net>
	<20040701115809.GA22778@rogue.amk.ca>
Message-ID: <40E4841D.3050207@v.loewis.de>

A.M. Kuchling wrote:
> On Thu, Jul 01, 2004 at 04:52:17AM -0700, rhettinger@users.sourceforge.net wrote:
> 
>>Modified Files:
>>	whatsnew24.tex 
>>Log Message:
>>Move Decimal from the sandbox into production.
> 
> 
> Um, I'd really like to write my own text for this document.  Please
> don't go to the effort of writing "What's New" sections.

Is that a general comment, or specific for this change?

If it is a general comment, what is the process for requesting
changes to the draft whatsnew?

Regards,
Martin


From martin at v.loewis.de  Thu Jul  1 17:41:37 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul  1 17:41:42 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <7jtnr35y.fsf@python.net>
References: <20040701181655.6375E2CEEEC@mail.ion-tof.com>
	<7jtnr35y.fsf@python.net>
Message-ID: <40E48511.6010203@v.loewis.de>

Thomas Heller wrote:
> I was asking because I'm not sure if there will be problems with Windows
> file system protection with non-MSI installers.

There is nothing inherent in Windows Installer that competing technology
couldn't provide. Microsoft has documented the procedures for
manipulating protected files, so competing installer can integrate the
necessary changes. Whether Inno Setup supports protected files, I don't
know. Nor do I know whether Inno Setup properly deals with file versions
for files that are not protected, or with any other of the Windows
mechanisms necessary for software installation.

Regards,
Martin


From martin at v.loewis.de  Thu Jul  1 17:45:52 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul  1 17:45:59 2004
Subject: [Python-Dev] Allowing u.encode() to return non-strings
In-Reply-To: <04Jun30.202138pdt."58612"@synergy1.parc.xerox.com>
References: <04Jun30.202138pdt."58612"@synergy1.parc.xerox.com>
Message-ID: <40E48610.4010408@v.loewis.de>

Bill Janssen wrote:
> I assume, though, that the args to "read()" and friends are still
> about bytes.

Yes. It is not possible to determine, in advance, the number of
bytes needed to decode a given number of characters. Therefore, a
codec typically needs to either read more bytes than requested,
or return less characters (if the bytes read don't happen to
end on a character boundary).

So the size parameter to .read() is just a hint - a codec might
chose to completely ignore it.

Regards,
Martin


From bac at OCF.Berkeley.EDU  Thu Jul  1 18:40:50 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Jul  1 18:40:57 2004
Subject: [Python-Dev] Re: Candidate Itertools
In-Reply-To: <00ff01c45a18$a42ddac0$21ba958d@oemcomputer>
References: <00ff01c45a18$a42ddac0$21ba958d@oemcomputer>
Message-ID: <40E492F2.5030907@ocf.berkeley.edu>

Raymond Hettinger wrote:

> [David Eppstein]
> 
>>In another place I used the dictionary directly
>>but it amounted to the same thing.
> 
> 
> All respondents prefer having a dictionary (elements mapped to counts)
> instead of an itertool returning (cnt, elem) ...
> 
> Just to be clear before I do it, does everyone support having
> dict.countkeys() as a dictionary classmethod instead of the original
> proposal?
> 

I would rather not have it as a class method personally.  I believe Bob 
suggested creating a bucket type.  That sounds reasonable and could go 
into 'collections'

-Brett

From tim.peters at gmail.com  Thu Jul  1 19:57:30 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Jul  1 19:57:39 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <7jtnr35y.fsf@python.net>
References: <20040701181655.6375E2CEEEC@mail.ion-tof.com>
	<7jtnr35y.fsf@python.net>
Message-ID: <1f7befae040701165773cc0269@mail.gmail.com>

[Thomas Heller]
> I was asking because I'm not sure if there will be problems with Windows
> file system protection with non-MSI installers.

I doubt it, and Python doesn't try to replace system files anyway. 
The Zope 2.7, Z4I and ZRS Windows installers are all written with
InnoSetup, and nobody has reported problems with them; two of those
include their own Python and win32all installations; I've tried all
those myself on Win98SE, Win2K, and WinXP Pro.

It's possible (don't know) that InnoSetup doesn't play "correctly"
with XP system restore points.  MSI does.  I've found restore points
to be an essentially useless feature, so I don't really care about
that.

If it were up to me (and it's certainly not!), I'd go with MSI at this
point.  I don't know how to drive that either, and am an
advanced-level InnoSetup hacker, so that's not a self-serving position
<wink>.  The thing is that MSI is certain to play well with all
Windows gimmicks (like Group Policy, install-on-demand, restore
points, whatever new nightmares XP SP2 unleashes, etc), and free tools
to make it palatable are appearing.  For example, I've been told (but
don't know) that this one is worth its weight in electrons:

   http://www.labyrinth.net.au/~dbareis/makemsi.htm

From edcjones at erols.com  Thu Jul  1 20:25:38 2004
From: edcjones at erols.com (Edward C. Jones)
Date: Thu Jul  1 20:31:19 2004
Subject: [Python-Dev] -1 on @[decorator]
Message-ID: <40E4AB82.60008@erols.com>

There are three 7-bit ASCII characters not used in Python: "@", "$", and 
"?". Please don't waste one of them for the special purpose of decorators:

    @[staticmethod]
    def fun(args):
        ...

To me "@" says "not python". I use it in templates that generate Python 
code.

-1 for "@" in decorators.


From bac at OCF.Berkeley.EDU  Thu Jul  1 20:31:55 2004
From: bac at OCF.Berkeley.EDU (Brett Cannon)
Date: Thu Jul  1 20:32:04 2004
Subject: [Python-Dev] python-dev Summary for 2004-06-16 through 2004-06-30
	[draft]
Message-ID: <40E4ACFB.2010403@ocf.berkeley.edu>

For as long as I can remember, when I have sent out a rough draft of a 
summary, it has been delayed long enough that I had a bunch of emails 
already for the next summary.  Well everyone, I managed to finally have 
my python-dev folder have an email count less than the number of fingers 
I have on a hand (and I still have all of my fingers).  You can thank a 
hiccup in my workload for that (and thus me not earning any money 
today).  It almost seems wrong that I don't have enough emails to cause 
a scrollbar to appear.  =)

Won't send this out any earlier than the 4th so you have at least until 
then to send in corrections.


------------------------------------

Summary Announcements
=====================
The trouble plaguing mail.python.org mid-month have been cleared up.  It 
had led to some rather delayed emails so if you sent one in and it was 
ignored for no good reason you might want to consider bringing the topic 
back up.

In the little comment made by the summary count, I questioned whether 
that count was correct.  Turns out it was off by one; this is the true 
forty-third semi-monthly summary by me (month-long summaries I counted 
twice).  So counting the two week span that I took off when Raymond 
Hettinger summarized for a week way back in October 2002 (still waiting 
for the second week, Raymond  =) I have been doing this for 22 months 
straight.  And yet after all of that email I still have my eye sight. 
Amazing.

Summaries
=========
---------------------------------------------
Decimal type still chugging along
---------------------------------------------
Facundo Batista updated `PEP 327`_ with what are close to the last 
issues for the PEP.  There has also been some recent activity in the 
sandbox on the package.  It could still make it into 2.4 ...

And it has!  As I wrote this Decimal.py graduated kindergarten, left the 
sandbox, and is now out on the playground with the rest of Lib.  Play nice.

.. _PEP 327: http://www.python.org/peps/pep-0327.html

Contributing threads:
   - `PEP 327 (Decimal data type) updated and status 
<http://mail.python.org/pipermail/python-dev/2004-June/045434.html>`__

-------------------------------------------------
CVS docs, now in handy tarball form
-------------------------------------------------
Fred Drake has tweaked the script that puts up new versions of the CVS 
docs to also upload a tarball of the HTML.  You can find that and links 
to the other in-development documentation at 
http://www.python.org/dev/doc/ .

Contributing threads:
   - Downloadable HTML from the trunk 
<`http://mail.python.org/pipermail/python-dev/2004-June/045452.html`>__

---------------------------------------------
Proposed additions to heapq API
---------------------------------------------
Raymond Hettinger proposed adding two new functions to the heapq API. 
heappushpop() would take an item, push it on to the heap, and then pop 
off the smallest value.  heapiter() would return a destructive iterator. 
  The actual implementation of heappushpop() got the most discussion.

No finally decision has been made, so if either function sounds good to 
you then do speak up.

Contributing threads:
   - `heapq API 
<http://mail.python.org/pipermail/python-dev/2004-June/045348.html>`__

---------------------------------------
Consolidation of CJK codecs
---------------------------------------
Hye-Shik Chang consolidated the various CJK codecs into various files 
and managed to save some space in terms of the files themselves.  As M-A 
Lemburg pointed out, though, there was no need to in terms of loading 
size since most linkers mmap only what is needed to run.

Contributing threads:
   - `Planned updates for cjkcodecs before 2.4a1 
<http://mail.python.org/pipermail/python-dev/2004-June/045415.html>`__

------------------------------------------------
Function decorators in 2.4 ... maybe
------------------------------------------------
The topic of whether function decorators would go into 2.4, and if so 
with what syntax, continued to be debated from the `last summary`_. 
Beyond reference implementations, nothing really changed.  Overall 
people agreed they would like to see it in 2.4 using whatever syntax 
Guido chooses (and he has three in mind, so no more suggestions!), but 
if he felt the need to wait then functionality would be held off for 2.5 .

Contributing threads:
   - `functions decorators in 2.4? 
<http://mail.python.org/pipermail/python-dev/2004-June/045440.html>`__
   - `decorators and 2.4 
<http://mail.python.org/pipermail/python-dev/2004-June/045516.html>`__

-------------------------------------------------------------------------
Making ``''.join(list_of_strs)`` an inconsequential idiom
-------------------------------------------------------------------------
As most people know, concatenating strings in a 'for' loop is a bad 
idea; the idiom you are supposed to use is to store the strings you want 
to concatenate in a list and the pass it to ``''.join(list_of_strs)``. 
This is because string addition requires allocating memory for the size 
of the concatenation and then copying from both strings into the new 
memory.  The constant new memory allocation is costly when you have a 
loop that goes for more than a few iterations.

Raymond Hettinger suggested altering the str object to have a pointer 
that would point to any strings that were meant to be treated as 
concatenated to the current string.  The problem with this is that it 
grew the str object the size of a pointer and that is costly for a 
object as common as string.

Armin Rigo gave this challenge a shot and ended up with a bunch of code 
that special-cases the BINARY_ADD opcode.  Neither solution as been 
accepted at this moment.

Contributing threads:
   - `Wild Idea for the Year 
<http://mail.python.org/pipermail/python-dev/2004-June/045488.html>`__
   - `Building strings with += 
<http://mail.python.org/pipermail/python-dev/2004-June/045504.html>`__

----------------------------------------------------------------------------------------
Possible exposure of universal newlines outside of the file object
----------------------------------------------------------------------------------------
Scott David Daniels suggested a new itertools functioned that would work 
like universal newlines but for strings or files already opened without 
universal newlines turned on.  People were receptive to the idea, just 
not for putting it into itertools.  Suggestions ranged from exposing the 
functionality as a static method of the file type to sticking into 
'string'.  No conclusion of where to put it was reached.

Contributing threads:
   - `Possible addition to itertools 
<http://mail.python.org/pipermail/python-dev/2004-June/045456.html>`__

----------------------------------
NOP does exist I tell you!
----------------------------------
Raymond Hettinger asked if anyone objected to adding a no-op opcode 
(aptly named NOP).  People said nope (see?  "NOP", "nope", sound the 
same?  <eddie izzard>Forget it</eddie izzard>).  So it's in.

Contributing threads:
   - `NOP 
<http://mail.python.org/pipermail/python-dev/2004-June/045491.html>`__

---------------------------------------------------------------
Yes, the Windows binary for 2.4 will use VC 7.1
---------------------------------------------------------------
Yes, Python 2.4 will use VC 7.1 .  No, we are not going to have a VC 6 
binary (you can build it if you want, though).  No, distributing the 
needed DLLs with the installer go against the GPL (even though the 
codebase is not GPL but more BSD and no one from the FSF has told it 
including the DLLs is wrong).  Maybe, we will use Martin v. Loewis' MSI 
installer, maybe not (still being debated as this was being written).

Contributing threads:
   - `VC 7.1 maintenance? 
<http://mail.python.org/pipermail/python-dev/2004-June/045357.html>`__

-------------------------------------
Possible itertools additions
-------------------------------------
Raymond Hettinger suggested two itertools additions; count_elements and 
pairswap.  The former counts the number of times an item appears in the 
passed-in iterable and returns pairs of the count and the item.  The 
latter takes an iterable that returns pairs and returns and iterable 
that returns those pairs swapped.

In the end people didn't see the need for pairswap.  People did want 
count_elements, but not as an iterator but as something that returned a 
dictionary.  It was suggested as both a class method of dict and as a 
new bucket type that could possibly go into the 'collections' module. 
No final decisions yet, though.

Contributing threads:
   - `Candidate Itertools 
<http://mail.python.org/pipermail/python-dev/2004-June/045394.html>`__

--------------------------------
Gettin' PEP 292 into 2.4
--------------------------------
Barry Warsaw brought up the point that `PEP 292`_ was slated to go into 
2.4 (it's currently in the sandbox).  The two biggest questions was the 
names of the new functions and where to stick them.  The name issue was 
not quite as bad once Barry explained the reasoning behind the existing 
names.

Where to put the code, though, was not so easily dealt with.  The 
original idea (which Barry and I came up with, so I am biased here) was 
to turn the string module into a package with the new code going there. 
  Barry also pointed out that the code in 'string' that is deprecated 
and will eventually go away can be moved into an individual module that 
can just simply be deleted from the package later and change a single 
``import *`` line in __init__.py for the package.

Not everyone saw the utopia Barry and I were trying to build.  People 
suggested just not bothering with turning 'string' into a package. 
Still others suggested a new package altogether; 'stringlib' and 'text' 
were suggested names.  It's still not resolved.

.. _PEP 292: http://www.python.org/peps/pep-0292.html

Contributing threads:
   - `PEP 292 for Python 2.4 
<http://mail.python.org/pipermail/python-dev/2004-June/045406.html>`__

-------------------------------------------------------------------------
Having u.encode() return something other than strings
-------------------------------------------------------------------------
Currently unicode.encode() returns only strings.  M-A Lemburg wanted to 
lift this restriction.  Guido pointed out that this could possibly cause 
problems for type inferencing since the return type would vary based on 
an argument (which made me happy since my masters thesis ties into this 
stuff).  A compromise was reached where the method will only return 
8-bit or Unicode strings; basically only subtypes of basestring.

Contributing threads:
   - `Allowing u.encode() to return non-strings 
<http://mail.python.org/pipermail/python-dev/2004-June/045417.html>`__

--------------------------------------------------
Comparing floats and longs is a pain
--------------------------------------------------
Basically this all boiled down to this (quoted from Michael Chermside): 
"when comparing a long with a float where the exponent is larger than 
the precision, that the float should be treated as if it were EXACTLY 
EQUAL TO <coefficient>*2^<exponent>, instead of trying to treat it as 
some sort of a range."  Guido said he would like to see this in 2.4 if 
someone would write up the code and commit it.

Contributing threads:
   - `Comparing heterogeneous types 
<http://mail.python.org/pipermail/python-dev/2004-June/045460.html>`__

----------------------------------------------------------
German and Turkish docstrings from pydoc
----------------------------------------------------------
Martin v. Loewis is currently trying to figure out where to put partial 
translations of docstrings in the stdlib for pydoc to use as appropriate.

Contributing threads:
   - `Including translated doc strings in 2.4 
<http://mail.python.org/pipermail/python-dev/2004-June/045490.html>`__

-------------------------------------------------------------------------------------------
What to do when 'raise' is called after the last exception was caught
-------------------------------------------------------------------------------------------
What should happen if you call 'raise' when you already caught the last 
exception that was raised?  People debated what the official docs said. 
  Seemed to coalesce to be what ever the last exception was; caught or 
not.  This ties into being what exception was currently stored.

Contributing threads:
   - `Re-raise in absence of an "active" exception 
<http://mail.python.org/pipermail/python-dev/2004-June/045497.html>`__

---------------------------------------------------------------------
Can you help with cleaning up the Demo directory?
---------------------------------------------------------------------
The Demo directory is in need of some TLC.  A bunch of modules in there 
are out of date and could use an update to current Python abilities.  If 
you care to help by submitting patches to update the code that would be 
great.  There is a large swath of code in there varying from extremely 
simple to involved so if the mood ever strikes you there will be 
something for everyone.

And if you attend the next Python Bug Day you can also work on it then.

Contributing threads:
   - `What can we do about dealing with the Demo directory? 
<http://mail.python.org/pipermail/python-dev/2004-June/045654.html>`__

--------------------------------------------------------------------
No more expected test failures on Windows for 2.4
--------------------------------------------------------------------
The bsddb tests are now expected to pass on Windows.  What this means is 
that failures are no longer the norm on Windows for the Python 
regression testing suite.

Contributing threads:
   - `Joy in Berkeley Windowsland 
<http://mail.python.org/pipermail/python-dev/2004-June/045669.html>`__

------------------------------------------------------
Getting accessible local dicts per thread
------------------------------------------------------
Jim Fulton wanted to expose the pre-thread local dict .  He has an 
implementation going that he is going to play with and work the kinks 
out of.  After that the code will most likely be committed.

Contributing threads:
   - `Proposal: thread.get_dict 
<http://mail.python.org/pipermail/python-dev/2004-June/045681.html>`__

--------------------------------
Where to stick SHA-256
--------------------------------
Someone contributed a patch for adding SHA-256 support.  A discussion 
built up over where to put it, if at all.  It's still going on at this 
moment.  Most people seem to agree that if it is included, it should 
either just be another function in the sha module or that sha.sha() 
should accept an optional argument specifying how many bits to use for 
the hash.

Contributing threads:
   - `SHA-256 module 
<http://mail.python.org/pipermail/python-dev/2004-June/045724.html>`__

------------------------------------------
Making weakrefs subclassable
------------------------------------------
Fred Drake has a patch to add the ability to subclass weakrefs.  He 
wasn't sure if there would be a performance penalty though.  But Tim 
Peters stepped up and said there wouldn't be.  So it looks like it 
should go in shortly.

Contributing threads:
   - `making weakref.ref objects subclassable 
<http://mail.python.org/pipermail/python-dev/2004-June/045787.html>`__

From hannu at tm.ee  Thu Jul  1 20:37:29 2004
From: hannu at tm.ee (Hannu Krosing)
Date: Thu Jul  1 20:38:41 2004
Subject: [Python-Dev] -1 on @[decorator]
In-Reply-To: <40E4AB82.60008@erols.com>
References: <40E4AB82.60008@erols.com>
Message-ID: <1088728648.13165.1.camel@fuji.krosing.net>

On R, 2004-07-02 at 03:25, Edward C. Jones wrote:
> There are three 7-bit ASCII characters not used in Python: "@", "$", and 
> "?". Please don't waste one of them for the special purpose of decorators:
> 
>     @[staticmethod]
>     def fun(args):
>         ...
> 
> To me "@" says "not python". I use it in templates that generate Python 
> code.
> 
> -1 for "@" in decorators.

a nice backward-compatible notation would be
#[staticmethod]#

but I'm sure that someone is using that in his templates ;)

---------------
Hannu


From nidoizo at yahoo.com  Thu Jul  1 22:17:36 2004
From: nidoizo at yahoo.com (Nicolas Fleury)
Date: Thu Jul  1 22:16:44 2004
Subject: [Python-Dev] Re: -1 on @[decorator]
In-Reply-To: <40E4AB82.60008@erols.com>
References: <40E4AB82.60008@erols.com>
Message-ID: <cc2gi5$je6$1@sea.gmane.org>

Edward C. Jones wrote:
> To me "@" says "not python".

To me it says Perl;)

Nicolas


From jcarlson at uci.edu  Thu Jul  1 22:25:20 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu Jul  1 22:30:51 2004
Subject: [Python-Dev] -1 on @[decorator]
In-Reply-To: <40E4AB82.60008@erols.com>
References: <40E4AB82.60008@erols.com>
Message-ID: <20040701184505.7941.JCARLSON@uci.edu>

I'm -1 on the @-syntax for the same reason as Fernando Perez; IPython
from Scipy.

Scipy has its own conference, so seemingly has a fairly large user base.

IPython is an enhanced Python shell that uses @ for certain shell
accesses and aliasing.

Since there exists a piece of Python software (which may be used by a
not-insignificant number of people) that uses @, IMO, using @ for
decorators, when there exists two other valid syntaxes (either of which
many have declared to be just fine), seems a bit irresponsible.

 - Josiah


From pete at shinners.org  Fri Jul  2 02:19:35 2004
From: pete at shinners.org (Pete Shinners)
Date: Fri Jul  2 02:22:57 2004
Subject: [Python-Dev] Advanced C Types
Message-ID: <cc2uvq$crm$1@sea.gmane.org>

Hello Python developers, I have been learning my way through new style 
type in Python, but it hasn't been exactly easy. I have narrowed my 
remaining troubles down to two specific questions for this list. Thanks 
for pointers.

First, I have a base type written in C. Python classes are inheriting 
from these and those instances are being passed to another C function. 
This function needs access to data from the original type's PyObject 
structure. How do I get from the instance PyObject* to my base type's 
PyObject* data? My workaround is a hackish, the base has a method like this:
     PyObject *dirtydirty(PyObject *self, PyObject *args) {
          Py_INCREF(self);
          return self;
     }
Now my external C code call something like this:
     PyObject *GetBaseObjectFromInstance(PyObject *o) {
         PyObject *meth = PyObject_GetAttrString(o, "dirtydirty");
         PyObject *baseobj = PyObject_CallObject(meth, NULL);
	Py_DECREF(meth);
         return baseobj;
     }
How can I recreate this without requiring the 'dirtydirty' method? 
Obviously Python is internally keeping this data around, as it is passed 
to the C methods on my type. I cannot find a more direct route.


Second, I am very uncomfortable with the newstyle type allocation 
mechanisms. I understand the use of separate tp_init, tp_alloc, and 
tp_new. I do not understand how to use these from C. This is more a 
matter of clarification, my question is this; What is the proper way for 
one of my C functions to create my C type? Especially when my C type is 
derived from other C type? I feel wrong digging into my PyTypeObject for 
various tp_ pointers. I was comfortable with the PyObject_New() type 
functions, but from what I can see those are illegal for newstyle types?


Hmm, it appears my questions have expanded from simple and consise. 
Thanks again.


From trevp at trevp.net  Fri Jul  2 04:54:05 2004
From: trevp at trevp.net (Trevor Perrin)
Date: Fri Jul  2 04:54:24 2004
Subject: [Python-Dev] SHA-256 module
In-Reply-To: <20040630225214.GW11545@zot.electricrain.com>
References: <5.2.0.9.0.20040630082845.0294ecf0@pop.sbcglobal.yahoo.com>
	<5.2.0.9.0.20040629125948.0292a5a0@pop.sbcglobal.yahoo.com>
	<1088534421.27300.153.camel@localhost>
	<005001c45da1$cfb9e1c0$e841fea9@oemcomputer>
	<1088534421.27300.153.camel@localhost>
	<5.2.0.9.0.20040629125948.0292a5a0@pop.sbcglobal.yahoo.com>
	<5.2.0.9.0.20040630082845.0294ecf0@pop.sbcglobal.yahoo.com>
Message-ID: <5.2.0.9.0.20040702012559.01f6a4d0@pop.sbcglobal.yahoo.com>

At 03:52 PM 6/30/2004 -0700, Gregory P. Smith wrote:
[...]
>The point about SSL being included is interesting.  The OpenSSL library
>provides implementations of all of the important hash algorithms (and
>uses them in order to implement ssl!).  Its hashing code is much better
>optimized on various architectures than the python module ever will
>be.

On my P4, OpenSSL SHA-1 looks around 25% faster (75 vs. 60 MB/s).


FWIW, I've changed the patch to support SHA224, 384, and 512.  There are 
"sha256" and "sha512" modules, with an extra function in each module for 
the truncated algorithm:

 >>> from sha256 import sha256, sha224
 >>> from sha512 import sha512, sha384
 >>>
 >>> sha256("abc").hexdigest()
'ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad'
 >>>
 >>> sha384("abc").hexdigest()
'cb00753f45a35e8bb5a03d699ac65007272c32ab0eded1631a8b605a43ff5bed8086072ba1e7cc2358baeca134c825a7'

http://sourceforge.net/tracker/index.php?func=detail&aid=935454&group_id=5470&atid=305470


Since there are some module-level functions and constants (new(), 
digestsize, blocksize), I like using separate modules instead of sticking 
everything in 'sha'.

We could also add some simple wrapper modules for sha224 and sha384 to make 
them appear as top-level modules, like the other ones.


Trevor 


From mwh at python.net  Fri Jul  2 07:00:07 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Jul  2 07:00:10 2004
Subject: [Python-Dev] Advanced C Types
In-Reply-To: <cc2uvq$crm$1@sea.gmane.org> (Pete Shinners's message of "Thu,
	01 Jul 2004 23:19:35 -0700")
References: <cc2uvq$crm$1@sea.gmane.org>
Message-ID: <2mwu1m1xoo.fsf@starship.python.net>

Pete Shinners <pete@shinners.org> writes:

> Hello Python developers, I have been learning my way through new style
> type in Python, but it hasn't been exactly easy. I have narrowed my
> remaining troubles down to two specific questions for this
> list. Thanks for pointers.
>
> First, I have a base type written in C. Python classes are inheriting
> from these and those instances are being passed to another C
> function. This function needs access to data from the original type's
> PyObject structure. How do I get from the instance PyObject* to my
> base type's PyObject* data? My workaround is a hackish, the base has a
> method like this:
>      PyObject *dirtydirty(PyObject *self, PyObject *args) {
>           Py_INCREF(self);
>           return self;
>      }
> Now my external C code call something like this:
>      PyObject *GetBaseObjectFromInstance(PyObject *o) {
>          PyObject *meth = PyObject_GetAttrString(o, "dirtydirty");
>          PyObject *baseobj = PyObject_CallObject(meth, NULL);
> 	Py_DECREF(meth);
>          return baseobj;
>      }
> How can I recreate this without requiring the 'dirtydirty' method?
> Obviously Python is internally keeping this data around, as it is
> passed to the C methods on my type. I cannot find a more direct route.

Um.  What am I missing that makes this not totally pointless?  The
"instance PyObject*" *is* the "base type's PyObject* data".  Cast it.

> Second, I am very uncomfortable with the newstyle type allocation
> mechanisms. I understand the use of separate tp_init, tp_alloc, and
> tp_new. I do not understand how to use these from C. This is more a
> matter of clarification, my question is this; What is the proper way
> for one of my C functions to create my C type?

The same way _randommodule.c does it :-)

> Especially when my C type is derived from other C type? 

Hmm, not sure about that.  Have you looked at xxsubtype in the Python
source?  (Mind you, almost all C types derive from another C type:
PyBaseObjectType, so maybe this isn't a big deal).

> I feel wrong digging into my PyTypeObject for various tp_
> pointers. 

Why?

> I was comfortable with the PyObject_New() type functions, but from
> what I can see those are illegal for newstyle types?

I think so, yes.

Cheers,
mwh

-- 
  Considering that this thread is completely on-topic in the way only
  c.l.py threads can be, I think I can say that you should replace
  "Oblivion" with "Gravity", and increase your Radiohead quotient.
                                      -- Ben Wolfson, comp.lang.python

From amk at amk.ca  Fri Jul  2 08:44:01 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Fri Jul  2 08:44:18 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Doc/whatsnew
	whatsnew24.tex, 1.55, 1.56
In-Reply-To: <40E4841D.3050207@v.loewis.de>
References: <E1Bg06z-0001Qv-Qt@sc8-pr-cvs1.sourceforge.net>
	<20040701115809.GA22778@rogue.amk.ca>
	<40E4841D.3050207@v.loewis.de>
Message-ID: <20040702124401.GA29293@rogue.amk.ca>

On Thu, Jul 01, 2004 at 11:37:33PM +0200, "Martin v. L?wis" wrote:
> Is that a general comment, or specific for this change?
> If it is a general comment, what is the process for requesting
> changes to the draft whatsnew?

It's a general comment; the document has my name on it, so I'd like it
to be my prose.

Trivial changes -- typo fixes, etc. -- are fine; go ahead and check
those in.  Higher-level conceptual errors (e.g. I don't understand a
feature correctly) should be pointed out to me; send me a private
e-mail or post to python-dev.  Suggesting new changes shouldn't really
be necessary; I'm on python-checkins and make note of significant
changes that should be included.

--amk

From jim.jewett at eds.com  Fri Jul  2 10:01:53 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Fri Jul  2 10:02:03 2004
Subject: [Python-Dev]  @[decorator]
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C22955C3F@usahm010.exmi01.exch.eds.com>

Edward C. Jones:

> There are three 7-bit ASCII characters not used in
> Python: "@", "$", and "?".  Please don't waste one ...

Perhaps even fewer;  "$" is used by the simplified-string 
module that seems likely to go in.

"@" does often mean attribute, which works well with
decorations, but not so well with transformations.

But it is also an obvious target character for substitutions,
and already in use by some common extensions.

Would @[...] or [...]@ be any better than [...] or @ alone?

-jJ

From amk at amk.ca  Fri Jul  2 10:16:22 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Fri Jul  2 10:16:37 2004
Subject: [Python-Dev] PEP 327: from_float()
Message-ID: <20040702141622.GB29917@rogue.amk.ca>

ePEP 327 says "it was decided to omit from_float() from the API for
Py2.4".  Does this imply that from_float might be in some future
version?  

--amk

From barry at python.org  Fri Jul  2 10:27:34 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Jul  2 10:27:18 2004
Subject: [Python-Dev]  @[decorator]
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C22955C3F@usahm010.exmi01.exch.eds.com>
References: <B8CDFB11BB44D411B8E600508BDF076C22955C3F@usahm010.exmi01.exch.eds.com>
Message-ID: <1088778454.7513.8.camel@localhost>

On Fri, 2004-07-02 at 10:01, Jewett, Jim J wrote:
> Edward C. Jones:
> 
> > There are three 7-bit ASCII characters not used in
> > Python: "@", "$", and "?".  Please don't waste one ...
> 
> Perhaps even fewer;  "$" is used by the simplified-string 
> module that seems likely to go in.

But currently only in the literal, so it's not actually special... yet. 
There have been discussions about promoting '$' as a prefix to string
literals (a la r-strings and u-strings), but nothing concrete has been
proposed.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040702/43faf2ae/attachment.bin
From FBatista at uniFON.com.ar  Fri Jul  2 10:43:13 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Fri Jul  2 10:46:47 2004
Subject: [Python-Dev] PEP 327: from_float()
Message-ID: <A128D751272CD411BC9200508BC2194D03383AFD@escpl.tcp.com.ar>

[A.M. Kuchling]

#- ePEP 327 says "it was decided to omit from_float() from the API for
#- Py2.4".  Does this imply that from_float might be in some future
#- version?  

Proven that there were big probabilities of this method to bring confussion,
based on experiencie of Java with a Decimal implementation and its
relationship
with the binary float data type, other Decimal core developers and I took
the
decision to extract it from the module in its first conception for Python
2.4.

.	Facundo





. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040702/5368750e/attachment.html
From tim.peters at gmail.com  Fri Jul  2 11:03:31 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Jul  2 11:03:36 2004
Subject: [Python-Dev] PEP 327: from_float()
In-Reply-To: <20040702141622.GB29917@rogue.amk.ca>
References: <20040702141622.GB29917@rogue.amk.ca>
Message-ID: <1f7befae04070208031333cc3f@mail.gmail.com>

[A.M. Kuchling]
> ePEP 327 says "it was decided to omit from_float() from the API for
> Py2.4".  Does this imply that from_float might be in some future
> version?

I don't think it *implies* it, but it might happen anyway.

There eventually needs to be a way to combine Decimal with native
binary fp, and to work naturally with Decimal in many other parts of
Python too.  But the immediate need for Decimal is in apps that can't
tolerate the representation errors in converting between base 10 and
base 2 floats, and I expect it's actually an aid to such apps if
there's no builtin way to confuse binary floats with Decimals at the
start.

This is meant not to make the same mistake we made with Unicode
<wink>:  the idea that 8-bit strings can be mixed with Unicode
strings, except sometimes you get mysterious data-dependent
exceptions, has probably caused more overall pain than if we had
forced a clean separation there at the start.

From pete at shinners.org  Fri Jul  2 11:24:26 2004
From: pete at shinners.org (Pete Shinners)
Date: Fri Jul  2 11:27:48 2004
Subject: [Python-Dev] Re: Advanced C Types
In-Reply-To: <2mwu1m1xoo.fsf@starship.python.net>
References: <cc2uvq$crm$1@sea.gmane.org> <2mwu1m1xoo.fsf@starship.python.net>
Message-ID: <cc3utc$s6k$1@sea.gmane.org>

Michael Hudson wrote:
> Um.  What am I missing that makes this not totally pointless?  The
> "instance PyObject*" *is* the "base type's PyObject* data".  Cast it.

If this is the case, then terrific. I didn't think it could be since the 
PyObject must have to be a PyClassObject, and not PyMyBaseObject. I'm 
still working with it all, so I'll test this out. If this just works I 
am in good shape.

> Hmm, not sure about that.  Have you looked at xxsubtype in the Python
> source? 

No, but thanks for this tip, this is great! It is tucked into the 
Modules directory.

>>I feel wrong digging into my PyTypeObject for various tp_
>>pointers. 
> 
> Why?

Up until tp_alloc, and tp_init, there was a PyObject_MACRO or 
PyObject_Function to deal with these for me. I've never once before 
looked inside the PyTypeObject itself. It just seems strange to start 
doing it now. That's what makes me feel I am not doing something right. 
but if that is correct, I will access with confidence (like the pros).


From trevp at trevp.net  Fri Jul  2 12:21:16 2004
From: trevp at trevp.net (Trevor Perrin)
Date: Fri Jul  2 12:21:35 2004
Subject: [Python-Dev] SHA-256 module
In-Reply-To: <20040702155533.GB30677@zot.electricrain.com>
References: <5.2.0.9.0.20040702012559.01f6a4d0@pop.sbcglobal.yahoo.com>
	<5.2.0.9.0.20040630082845.0294ecf0@pop.sbcglobal.yahoo.com>
	<5.2.0.9.0.20040629125948.0292a5a0@pop.sbcglobal.yahoo.com>
	<1088534421.27300.153.camel@localhost>
	<005001c45da1$cfb9e1c0$e841fea9@oemcomputer>
	<1088534421.27300.153.camel@localhost>
	<5.2.0.9.0.20040629125948.0292a5a0@pop.sbcglobal.yahoo.com>
	<5.2.0.9.0.20040630082845.0294ecf0@pop.sbcglobal.yahoo.com>
	<5.2.0.9.0.20040702012559.01f6a4d0@pop.sbcglobal.yahoo.com>
Message-ID: <5.2.0.9.0.20040702090540.01f6b2f8@pop.sbcglobal.yahoo.com>

At 08:55 AM 7/2/2004 -0700, Gregory P. Smith wrote:
>On Fri, Jul 02, 2004 at 01:54:05AM -0700, Trevor Perrin wrote:
> > At 03:52 PM 6/30/2004 -0700, Gregory P. Smith wrote:
> > [...]
> > >The point about SSL being included is interesting.  The OpenSSL library
> > >provides implementations of all of the important hash algorithms (and
> > >uses them in order to implement ssl!).  Its hashing code is much better
> > >optimized on various architectures than the python module ever will
> > >be.
> >
> > On my P4, OpenSSL SHA-1 looks around 25% faster (75 vs. 60 MB/s).
>
>Exactly.  On my pentium2 class celeron its >200% faster.  A future version
>of openssl should include an sse2 version of sha1 thats even faster yet
>on modern cpus, approaching md5 speed on some cpus.  (not integrated
>into openssl cvs as of today, but its much nicer to let the openssl
>project worry about that than us).

Interesting, I didn't know there was still so much speedup to be had!


>I think we should start by using your base code.  I'm willing to
>do up a patch later to have it use OpenSSL conditionally at compilation
>time if someone else doesn't beat me to it.

That sounds like a good way to go.


>I'm still -1 on adding top level modules as I think it pollutes the top
>level namespace.  I'm happy with separate modules if we stick them down
>one level underneath a hashes or digest namespace.
>
>for example:
>
>from digest import md5, sha1, sha256, sha224, sha384, sha512

[...]

>(sha1 and md5 are included in for completeness even though their top
>level modules would still exist for legacy reasons)

[...]

>I also don't mind the hashes.sha.new("abc", bits=224) interface that was
>proposed but prefer the above one as people think of them by algorithm
>name rather than bits (sha1 rather than 160 bit sha).

I agree with all the above - that's my preference too.


Trevor 


From bac at OCF.Berkeley.EDU  Fri Jul  2 12:49:29 2004
From: bac at OCF.Berkeley.EDU (Brett Cannon)
Date: Fri Jul  2 12:49:42 2004
Subject: [Python-Dev] python-dev Summary for 2004-06-16 through 2004-06-30
	[draft]
In-Reply-To: <40E4ACFB.2010403@ocf.berkeley.edu>
References: <40E4ACFB.2010403@ocf.berkeley.edu>
Message-ID: <40E59219.20006@ocf.berkeley.edu>



Brett Cannon wrote:


> ---------------------------------------------------------------
> Yes, the Windows binary for 2.4 will use VC 7.1
> ---------------------------------------------------------------
> Yes, Python 2.4 will use VC 7.1 .  No, we are not going to have a VC 6 
> binary (you can build it if you want, though).  No, distributing the 
> needed DLLs with the installer go against the GPL (even though the 
                                 ^
Yes, there is supposed to be a "does not".  Thanks to everyone who has 
emailed me this correction or was about to.

-Brett

From dave at boost-consulting.com  Fri Jul  2 13:32:08 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Fri Jul  2 13:32:22 2004
Subject: [Python-Dev] Re: python-dev Summary for 2004-06-16 through
	2004-06-30 [draft]
References: <40E4ACFB.2010403@ocf.berkeley.edu>
Message-ID: <u4qoqjox3.fsf@boost-consulting.com>

Brett Cannon <bac@OCF.Berkeley.EDU> writes:

> ------------------------------------------------
> Function decorators in 2.4 ... maybe
> ------------------------------------------------
> The topic of whether function decorators would go into 2.4, and if so
> with what syntax, continued to be debated from the `last
> summary`_. Beyond reference implementations, nothing really changed.
> Overall people agreed they would like to see it in 2.4 using whatever
> syntax Guido chooses (and he has three in mind, so no more
> suggestions!), but if he felt the need to wait then functionality
> would be held off for 2.5 .
>
> Contributing threads:
>    - `functions decorators in 2.4?
>      <http://mail.python.org/pipermail/python-dev/2004-June/045440.html>`__
>    - `decorators and 2.4
>      <http://mail.python.org/pipermail/python-dev/2004-June/045516.html>`__

Was the possibility of "no special syntax; do it within the language"
really unworthy of mention in the summary?

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com


From mchermside at ingdirect.com  Fri Jul  2 13:56:46 2004
From: mchermside at ingdirect.com (Chermside, Michael)
Date: Fri Jul  2 13:56:59 2004
Subject: [Python-Dev] PEP 327: from_float()
Message-ID: <0CFFADBB825C6249A26FDF11C1772AE1550B09@ingdexj1.ingdirect.com>

Facundo writes:
> Proven that there were big probabilities of this method to bring confussion, 
> based on experiencie of Java with a Decimal implementation and its relationship 
> with the binary float data type, other Decimal core developers and I took the 
> decision to extract it from the module in its first conception for Python 2.4.

While I don't object to removing from_float() for now, I want to encourage
us to consider adding it back later -- I think it does have some uses. In
the meantime, people can just convert to strings and back... which is
easy to explain and to implement, although it probably suffers from minor
rounding problems (I haven't really thought it through).

-- Michael Chermside


This email may contain confidential or privileged information. If you believe you have received the message in error, please notify the sender and delete the message without copying or disclosing it.


From bac at OCF.Berkeley.EDU  Fri Jul  2 13:57:30 2004
From: bac at OCF.Berkeley.EDU (Brett Cannon)
Date: Fri Jul  2 13:57:47 2004
Subject: [Python-Dev] Re: python-dev Summary for 2004-06-16
	through	2004-06-30 [draft]
In-Reply-To: <u4qoqjox3.fsf@boost-consulting.com>
References: <40E4ACFB.2010403@ocf.berkeley.edu>
	<u4qoqjox3.fsf@boost-consulting.com>
Message-ID: <40E5A20A.3080409@ocf.berkeley.edu>



David Abrahams wrote:

> Brett Cannon <bac@OCF.Berkeley.EDU> writes:
> 
> 
>>------------------------------------------------
>>Function decorators in 2.4 ... maybe
>>------------------------------------------------
>>The topic of whether function decorators would go into 2.4, and if so
>>with what syntax, continued to be debated from the `last
>>summary`_. Beyond reference implementations, nothing really changed.
>>Overall people agreed they would like to see it in 2.4 using whatever
>>syntax Guido chooses (and he has three in mind, so no more
>>suggestions!), but if he felt the need to wait then functionality
>>would be held off for 2.5 .
>>
>>Contributing threads:
>>   - `functions decorators in 2.4?
>>     <http://mail.python.org/pipermail/python-dev/2004-June/045440.html>`__
>>   - `decorators and 2.4
>>     <http://mail.python.org/pipermail/python-dev/2004-June/045516.html>`__
> 
> 
> Was the possibility of "no special syntax; do it within the language"
> really unworthy of mention in the summary?
> 

I would be happy to add a mention, but with Guido not saying that was a 
possibility I just didn't think about it.  It is no way meant to suggest 
I think anything is "unworthy" in terms of what you, Philip, and 
everyone else who helped come up with that implementation.  The 
summaries are just what I happen to think of when I write the summary 
(which is a first draft; I don't proof-read as I am sure some of you may 
have noticed  =).

I am not trying to make any value-judgments, but I am in no way putting 
a major effort into being unbiased; I'm not paid enough to care about 
pushing a little propoganda for my views.  =)  But that was not 
happening here.  I just didn't think that much about it.  It honestly 
just connected in my brain as another syntax suggestion, albeit an 
unobtrusive one.

Personally the whole thread did not interest me terribly.  This whole 
topic has just worn me out slightly in terms of reading it since it has 
been discussed for ages and I have read **every** email (by forcing 
myself at this point) on the subject on this list.  I could quite easily 
live the rest of my life without reading another syntax suggestion for 
function decorators (as I am sure Guido could as well).

-Brett

From FBatista at uniFON.com.ar  Fri Jul  2 14:24:22 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Fri Jul  2 14:27:59 2004
Subject: [Python-Dev] PEP 327: from_float()
Message-ID: <A128D751272CD411BC9200508BC2194D03383AFE@escpl.tcp.com.ar>

[Chermside, Michael]

#--While I don't object to removing from_float() for now, I 
#--want to encourage
#--us to consider adding it back later -- I think it does have 
#--some uses. In
#--the meantime, people can just convert to strings and back... which is
#--easy to explain and to implement, although it probably 
#--suffers from minor
#--rounding problems (I haven't really thought it through).

But maybe forcing the user to take in consideration those rounding problems
is the best solution to avoid that user to be surprised about the issues of
binary floating point conversion to decimal data point.

I don't know: regarding user binary floating point usage (and other
subjects, :), Tim Peters' word is law for me.

.	Facundo

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.713 / Virus Database: 469 - Release Date: 30/06/2004
 

From dan.gass at gmail.com  Fri Jul  2 14:42:09 2004
From: dan.gass at gmail.com (Dan Gass)
Date: Fri Jul  2 14:42:21 2004
Subject: [Python-Dev] Side by side HTML difflib.py support: REQUEST FOR U/I
	REVIEW
Message-ID: <bc15ba280407021142b5ebdf2@mail.gmail.com>

Python patch 914575
(https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914575&group_id=5470)
 adds the ability to generate HTML side by side file differences (full
or contextual) to the difflib.py module.  This patch has been updated
to:

1) simplify the user interface
2) improve the look to use the same color schemes as viewCVS.
3) generate valid XHTML 1.0 Transitional
4) clean up cr/lf issues

Any help reviewing this code would be appreciated especially in the
area of the user interface, namely the public methods and the passed
parameters to __init__.  Please use sourceforge to submit comments
otherwise respond to this and I will post them for you.

Thanks,
Dan Gass

From pje at telecommunity.com  Fri Jul  2 15:08:39 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Jul  2 15:05:47 2004
Subject: [Python-Dev] Re: python-dev Summary for 2004-06-16
	through	2004-06-30 [draft]
In-Reply-To: <40E5A20A.3080409@ocf.berkeley.edu>
References: <u4qoqjox3.fsf@boost-consulting.com>
	<40E4ACFB.2010403@ocf.berkeley.edu>
	<u4qoqjox3.fsf@boost-consulting.com>
Message-ID: <5.1.1.6.0.20040702150549.032a7ec0@mail.telecommunity.com>

At 10:57 AM 7/2/04 -0700, Brett Cannon wrote:
>I would be happy to add a mention, but with Guido not saying that was a 
>possibility I just didn't think about it.  It is no way meant to suggest I 
>think anything is "unworthy" in terms of what you, Philip, and everyone 
>else who helped come up with that implementation.  The summaries are just 
>what I happen to think of when I write the summary (which is a first 
>draft; I don't proof-read as I am sure some of you may have noticed  =).

Since David's idea allows decorators to be implemented even in Python 2.2 
without any special language support, this is actually pretty big news for 
anybody who's been waiting with bated breath for Python to support 
decorators.  They can now go "roll their own" for 2.2 and 2.3 using a 
variant of the syntax Guido first proposed.  So, a mention would be a good 
thing, I think.



From dave at boost-consulting.com  Fri Jul  2 15:14:12 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Fri Jul  2 15:14:25 2004
Subject: [Python-Dev] Re: python-dev Summary for 2004-06-16 through
	2004-06-30 [draft]
In-Reply-To: <40E5A20A.3080409@ocf.berkeley.edu> (Brett Cannon's message of
	"Fri, 02 Jul 2004 10:57:30 -0700")
References: <40E4ACFB.2010403@ocf.berkeley.edu>
	<u4qoqjox3.fsf@boost-consulting.com>
	<40E5A20A.3080409@ocf.berkeley.edu>
Message-ID: <uk6xmi5mj.fsf@boost-consulting.com>

Brett Cannon <bac@OCF.Berkeley.EDU> writes:

> I could quite easily live the rest of my life without reading
> another syntax suggestion for function decorators (as I am sure
> Guido could as well).

Me too.  "No syntax" was designed to put a stop to 'em ;-)

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com


From theller at python.net  Fri Jul  2 15:45:09 2004
From: theller at python.net (Thomas Heller)
Date: Fri Jul  2 15:45:22 2004
Subject: [Python-Dev] useless question in bdist_wininst installers
Message-ID: <acyip516.fsf@python.net>

I just submitted <http://www.python.org/sf/984290>, which contains this text:

Before a bdist_wininst created installer overwrites files it asks the
user whether files should be overwritten or not. This question is
annoying and doesn't seem very useful - it is too late to cancel the
installation (because some files might already have been installed).

I suggest to remove this question (MessageBox), and always overwrite
files during the installation.

Any comments?

Thomas


From martin at v.loewis.de  Fri Jul  2 17:37:22 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Jul  2 17:37:28 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <1f7befae040701165773cc0269@mail.gmail.com>
References: <20040701181655.6375E2CEEEC@mail.ion-tof.com>	<7jtnr35y.fsf@python.net>
	<1f7befae040701165773cc0269@mail.gmail.com>
Message-ID: <40E5D592.5090008@v.loewis.de>

Tim Peters wrote:
> For example, I've been told (but
> don't know) that this one is worth its weight in electrons:
> 
>    http://www.labyrinth.net.au/~dbareis/makemsi.htm

If I ever spend time on another setup tool, I'd probably
look at WIX. However, I will do that only if there is a chance
that it gets used in the end.

Regards,
Martin


From martin at v.loewis.de  Fri Jul  2 17:40:25 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Jul  2 17:40:31 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Doc/whatsnew
	whatsnew24.tex, 1.55, 1.56
In-Reply-To: <20040702124401.GA29293@rogue.amk.ca>
References: <E1Bg06z-0001Qv-Qt@sc8-pr-cvs1.sourceforge.net>	<20040701115809.GA22778@rogue.amk.ca>	<40E4841D.3050207@v.loewis.de>
	<20040702124401.GA29293@rogue.amk.ca>
Message-ID: <40E5D649.2080505@v.loewis.de>

A.M. Kuchling wrote:
> It's a general comment; the document has my name on it, so I'd like it
> to be my prose.

That is unfortunate. Should I remove all the changes I made over the
last year to that document?

> Trivial changes -- typo fixes, etc. -- are fine; go ahead and check
> those in.  Higher-level conceptual errors (e.g. I don't understand a
> feature correctly) should be pointed out to me; send me a private
> e-mail or post to python-dev.  Suggesting new changes shouldn't really
> be necessary; I'm on python-checkins and make note of significant
> changes that should be included.

Is the change to marshal significant?

Regards,
Martin


From raymond.hettinger at verizon.net  Fri Jul  2 06:11:00 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Fri Jul  2 18:12:03 2004
Subject: [Python-Dev] decimal API
Message-ID: <005401c4601c$e035f5c0$fe05a044@oemcomputer>

Currently, calling the Decimal constructor with an invalid literal (such
as Decimal("Fred")) returns a quiet NaN.  This was done because the spec
appeared to require it (in fact, there are IBM test cases to confirm
that behavior). 

I've discussed this with Mike Cowlishaw (author of the spec and test
cases) and he has just clarified that, "... the intent here was not to
disallow an exception.   The analogy, perhaps, is to a divide-by-zero:
the latter raises Invalid Operation and returns a qNaN.  The string
conversion is similar.   (Again, in some implementations/languages, the
result after such an exception is not available.)   I'll see if I can
clarify that, at least making it clear that Invalid Operation is OK at
that point."

So, my question for the group is whether to:

* leave it as-is
* raise a ValueError just like float('abc') or int('abc')
* raise an Invalid Operation and return a quiet NaN.

Either of the last two involves editing the third-party test cases which
I am loathe to do.  The second is the most Pythonic but does not match
Mike's clarification.  The third keeps within context of the spec but
doesn't bode well for Decimal interacting well with the rest of python.
The latter issue is unavoidable to some degree because no other python
numeric type has context sensitive operations, settable traps, and
result flags.




A separate question is determining the default precision.  Currently, it
is set at 9 which conveniently matches the test cases, the docstring
examples, and examples in the spec.  It is also friendly to running
time.

Tim had suggested that 20 or so would handle many user requirements
without needing a context change.

Mike had suggested default single and double precision matching those
proposed in 754R.  The rationale behind those sizes has nothing to do
with use cases; rather, they were chosen so that certain representations
(not the ones we use) fit neatly into byte/word sized multiples (once
again showing the hardware orientation of the spec).

No matter what the default, the precision is easy to change:

    >>> getcontext().prec = 42
    >>> Decimal(1) / Decimal(7)
    Decimal("0.142857142857142857142857142857142857142857")



Raymond


From python at rcn.com  Fri Jul  2 06:24:08 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Jul  2 18:25:13 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Doc/whatsnewwhatsnew24.tex, 1.55, 1.56
In-Reply-To: <20040702124401.GA29293@rogue.amk.ca>
Message-ID: <005501c4601e$b61177e0$fe05a044@oemcomputer>

[A.M. Kuchling]
> It's a general comment; the document has my name on it, so I'd like it
> to be my prose.


So it does.

My suggestion is continue to have implementers write the first draft and
that you use that as a basis for altering to your own style.  This
matches what is done with the tutorial which has Guido's name on it but
has many sections authored by others.

The implementers are in a good position to highlight how something was
designed to be used.  The examples in the whatsnew document will likely
shape how a feature is used, why it was introduced, and whether users
will respond well to it (IOW, the implementer has a strong vested
interest in the presentation).

Like Martin, I would offer to take out my entries, but that would
transform it from being completely current to being nearly vacant.  

Do what you want with it; I fully respect that it has your name on it.



Raymond


From aahz at pythoncraft.com  Fri Jul  2 18:59:01 2004
From: aahz at pythoncraft.com (Aahz)
Date: Fri Jul  2 18:59:06 2004
Subject: [Python-Dev] decimal API
In-Reply-To: <005401c4601c$e035f5c0$fe05a044@oemcomputer>
References: <005401c4601c$e035f5c0$fe05a044@oemcomputer>
Message-ID: <20040702225901.GA2554@panix.com>

On Fri, Jul 02, 2004, Raymond Hettinger wrote:
>
> So, my question for the group is whether to:
> 
> * leave it as-is
> * raise a ValueError just like float('abc') or int('abc')
> * raise an Invalid Operation and return a quiet NaN.
> 
> Either of the last two involves editing the third-party test cases which
> I am loathe to do.  The second is the most Pythonic but does not match
> Mike's clarification.  The third keeps within context of the spec but
> doesn't bode well for Decimal interacting well with the rest of python.
> The latter issue is unavoidable to some degree because no other python
> numeric type has context sensitive operations, settable traps, and
> result flags.

There's another option: allow both options one and two, with option two
the default; the test cases can manually set option one, while adding a
few extra ones to cover the default usage.

> A separate question is determining the default precision.  Currently, it
> is set at 9 which conveniently matches the test cases, the docstring
> examples, and examples in the spec.  It is also friendly to running
> time.

When this was discussed earlier (may have been in private), it was
decided to leave this because it's so easy to change.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Typing is cheap.  Thinking is expensive."  --Roy Smith, c.l.py

From tim.peters at gmail.com  Fri Jul  2 19:45:55 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Jul  2 19:53:21 2004
Subject: [Python-Dev] decimal API
In-Reply-To: <005401c4601c$e035f5c0$fe05a044@oemcomputer>
References: <005401c4601c$e035f5c0$fe05a044@oemcomputer>
Message-ID: <1f7befae0407021645fe05f98@mail.gmail.com>

[Raymond Hettinger]
> Currently, calling the Decimal constructor with an invalid literal (such
> as Decimal("Fred")) returns a quiet NaN.  This was done because the spec
> appeared to require it (in fact, there are IBM test cases to confirm
> that behavior).
>
> I've discussed this with Mike Cowlishaw (author of the spec and test
> cases) and he has just clarified that, "... the intent here was not to
> disallow an exception.   The analogy, perhaps, is to a divide-by-zero:
> the latter raises Invalid Operation and returns a qNaN.

Don't be confused by his odd wording here.  There's a concept of
"raising" in Python, but not in his spec!  Exceptional conditions in
the spec are "signaled", not raised.  Whether they go on to raise (in
the Python sense) a user-visible exception is to be controlled by the
specific signaled exceptional condition's trap-enable flag.

> The string conversion is similar.   (Again, in some implementations/languages,
> the result after such an exception is not available.)   I'll see if I can
> clarify that, at least making it clear that Invalid Operation is OK at
> that point."

Sorry, but I'm really confused now.  The current version of the spec
defines an exceptional condition specifically for this purpose:

"""
Conversion syntax 

This occurs and signals invalid-operation if an string is being
converted to a number and it does not conform to the numeric string
syntax. The result is [0,qNaN].
"""

In other words, the current spec *requires* that trying to convert a
nonsense string signal invalid-operation, not merely allows it. 
AFAICT, "our" code already does that, too(!).

Note that I've suggested that some trap-enable flags should be set by
default for Python:  those for invalid operation, divide by 0, and
overflow.  So, in my mind (unsullied by reality <wink>):  (a)
conversion of a nonsense string already signals invalid-operation;
and, (b) because the invalid-operation trap-enable flag is set by
default in Python, it also already raises an exception.

> So, my question for the group is whether to:
>
> * leave it as-is
> * raise a ValueError just like float('abc') or int('abc')
> * raise an Invalid Operation and return a quiet NaN.

That last choice is impossible:  you can *signal* invalid operation
and return a quiet NaN, but if you *raise* something then it's
impossible to return anything.  We could make InvalidOperation a
subclass of ValueError, BTW.  That would make sense (InvalidOperation
always has to do with an insane input value).

> Either of the last two involves editing the third-party test cases which
> I am loathe to do.

Or, until Mike fixes the test vectors, special-case the snot out of
these specific tests in the test driver.  That's assuming the test
vectors don't actually match the spec now.  But I suspect that they
do, and that there's a different confusion at work here.

> The second is the most Pythonic but does not match Mike's clarification.
>  The third keeps within context of the spec but doesn't bode well for Decimal
> interacting well with the rest of python.

That's one reason I want Python to enable the invalid-operation trap
by default.  That choice isn't suitable for all apps all the time, but
I have no hesitation Pronouncing that it will be suitable for most
Python apps most of the time.

> The latter issue is unavoidable to some degree because no other python
> numeric type has context sensitive operations, settable traps, and
> result flags.

That's because Python has blissfully ignored IEEE-754 for its current
floating-point operations.  754 requires all of those for binary fp,
and a small part of my hope for Decimal is that it nudges Python into
coming to grips with that 20-year-old standard.  Continuing to ignore
it creates problems for scientific programmers too, both those who
hate 754 and those who love it.  Python's binary fp support will end
up looking more like Decimal over time; the fact that Python's binary
fp is 20 years out of date isn't a good reason to cripple Decimal too.

> A separate question is determining the default precision.  Currently, it
> is set at 9 which conveniently matches the test cases, the docstring
> examples, and examples in the spec.  It is also friendly to running
> time.

I expect that's a red herring.  Financial apps are usually dominated
by the speed of I/O conversions and of addition, and those all take
time proportional to the number of significant decimal digits in the
operands, not proportional to precision.  Division certainly takes
time proportional to precision, but that's a rarer operation.  Even
after division or multiplication, financial apps will usually round
the result back, again reducing the number of significant decimal
digits (e.g., your tax forms generally make you round on every line,
not accumulate everything to 100 digits and round once at the end).

> Tim had suggested that 20 or so would handle many user requirements
> without needing a context change.

It would be pathetic to ship with a default precision smaller than
mid-range hand calculators.

> Mike had suggested default single and double precision matching those
> proposed in 754R.  The rationale behind those sizes has nothing to do
> with use cases; rather, they were chosen so that certain representations
> (not the ones we use) fit neatly into byte/word sized multiples (once
> again showing the hardware orientation of the spec).

I don't care about that either.

> No matter what the default, the precision is easy to change:
> 
>    >>> getcontext().prec = 42
>    >>> Decimal(1) / Decimal(7)
>    Decimal("0.142857142857142857142857142857142857142857")

It should be at least 12 (copying HP hand calculators).   I believe
current COBOL requires 36 decimal digits in its decimal type.  VB's
Currency type has 19 decimal digits, and its Decimal type has 28
decimal digits.  Those are all defaults decided by people who care
about applications more than word boundaries <wink>.

From jbone at deepfile.com  Sat Jul  3 02:04:40 2004
From: jbone at deepfile.com (Jeff Bone)
Date: Sat Jul  3 02:04:50 2004
Subject: [Python-Dev] Re: decorators and 2.4
In-Reply-To: <40E218A0.9060609@iinet.net.au>
References: <B8CDFB11BB44D411B8E600508BDF076C22955BFA@usahm010.exmi01.exch.eds.com>	<uacyms47p.fsf@boost-consulting.com>
	<1088528107.7527.141.camel@localhost>
	<40E218A0.9060609@iinet.net.au>
Message-ID: <DF2CD30E-CCB6-11D8-A224-000A95CFE9DE@deepfile.com>


+1

But only if semantically regarded *by the language* as literals rather 
than higher-order applications, as is (admittedly) implied if not 
explicit in the PEP.

jb

On Jun 29, 2004, at 8:34 PM, Nick Coghlan wrote:

> Barry Warsaw wrote:
>> I doubt he will respond.  I think Guido has decided what the semantics
>> will be and is just waffling over what syntax to choose.  /I'm/ 
>> waiting
>> for Tim to respond, because he'll get a clear image on his BCR (BDFL
>> Channel Receptor) a few days before Guido makes up his mind and posts
>> his pronouncement.
>
> I'm now hoping that he goes for the list-before-def approach, since 
> Phillip seems to have shown that it would then be possible to:
> a) get to use decorators
> b) write backwards compatible code (albeit with a little magic to do 
> so)
>
> Cheers,
> Nick.
>
> -- 
> Nick Coghlan               |     Brisbane, Australia
> Email: ncoghlan@email.com  | Mobile: +61 409 573 268
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jbone%40place.org
>


From jjl at pobox.com  Sat Jul  3 07:15:42 2004
From: jjl at pobox.com (John J Lee)
Date: Sat Jul  3 07:15:49 2004
Subject: [Python-Dev] VC 7.1 maintenance? [getting OT]
In-Reply-To: <1f7befae040701165773cc0269@mail.gmail.com>
References: <20040701181655.6375E2CEEEC@mail.ion-tof.com>
	<7jtnr35y.fsf@python.net>
	<1f7befae040701165773cc0269@mail.gmail.com>
Message-ID: <Pine.LNX.4.58.0407031206110.1659@alice>

On Thu, 1 Jul 2004, Tim Peters wrote:
[...]
> It's possible (don't know) that InnoSetup doesn't play "correctly"
> with XP system restore points.  MSI does.  I've found restore points
> to be an essentially useless feature, so I don't really care about
> that.
[...]

Why do you find system restore points useless?

Aren't they convenient, when they work?  (as long as you regard it as a
convenience, not a proper backup)

+1 on making multi-machine installation easier


John

From tim.peters at gmail.com  Sat Jul  3 15:24:22 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Jul  3 15:24:52 2004
Subject: [Python-Dev] VC 7.1 maintenance? [getting OT]
In-Reply-To: <Pine.LNX.4.58.0407031206110.1659@alice>
References: <20040701181655.6375E2CEEEC@mail.ion-tof.com>
	<7jtnr35y.fsf@python.net>
	<1f7befae040701165773cc0269@mail.gmail.com>
	<Pine.LNX.4.58.0407031206110.1659@alice>
Message-ID: <1f7befae040703122429781190@mail.gmail.com>

[John J Lee]
> Why do you find system restore points useless?

Tautologically, because I have yet to see a case where they were of real use.

> Aren't they convenient, when they work?

If they worked, I suppose they could be.

> (as long as you regard it as a convenience, not a proper backup)

GoBack is useful and convenient, because it can undo all changes on
disk.  In that respect, it's as good as (but much faster and easier
than) a full disaster-recovery restoration.  System Restore undoes a
vague subset of changes.  If, e.g., a rogue installation damages data
files, System Restore undoes parts of the installation, but leaves the
data files damaged.  It's good that System Restore (and other
gimmicks) protect OS-critical executables and the registry, and that's
better than nothing -- especially if all you run is the OS <wink>.

From mhammond at skippinet.com.au  Sat Jul  3 21:19:28 2004
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Sat Jul  3 21:19:28 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <40E48143.3020300@v.loewis.de>
Message-ID: <0c3d01c46164$f49e85b0$0200a8c0@eden>

I've been meaning to look at the MSI stuff - this thread prompted me to!

> Thomas Heller wrote:
> > Well, I would be much happier to build an installer with inno setup
> > instead of Martin's msi code, which I don't understand ;-(.
>
> I would be willing to explain it if you would ask questions :-(
> What is it that you don't understand about it?

On one hand, the code has almost no comments and assumes some knowledge of
the underlying MSI mindset.  On the other hand, it does seem to work!

With just a few tweaks (mainly relating to missing files), I managed to get
an MSI that appears to install correctly.  I think that is pretty cool :)  I
believe it has the basis for a reasonable MSI generator library.  The
hard-coding of the UI elements is unfortunate, but I see no alternative at
this stage.  Using distutils functions for building file lists etc would be
a nice for a few reasons, least of all for a future bdist_msi :)

Martin - links to articles you found enlightening while putting this
together is probably the largest omission - how did you come up with this?

Mark.


From t-meyer at ihug.co.nz  Sun Jul  4 02:35:42 2004
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Sun Jul  4 02:35:55 2004
Subject: [Python-Dev] RE: [Python-checkins] python/nondist/sandbox/decimal
	libdecimal.tex, NONE, 1.1
In-Reply-To: <1ED4ECF91CDED24C8D012BCF2B034F1306FA0895@its-xchg4.massey.ac.nz>
Message-ID: <1ED4ECF91CDED24C8D012BCF2B034F130467807B@its-xchg4.massey.ac.nz>

[...]
> \moduleauthor{Tim Peters}{guido@python.org}
[...]

Is this correct?  (I guess it could be, but it does look odd).

=Tony Meyer


From martin at v.loewis.de  Sun Jul  4 05:00:39 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Jul  4 05:00:49 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <0c3d01c46164$f49e85b0$0200a8c0@eden>
References: <0c3d01c46164$f49e85b0$0200a8c0@eden>
Message-ID: <40E7C737.6000702@v.loewis.de>

Mark Hammond wrote:
> On one hand, the code has almost no comments and assumes some knowledge of
> the underlying MSI mindset.

That's a valid complaint indeed.

> With just a few tweaks (mainly relating to missing files), I managed to get
> an MSI that appears to install correctly.  I think that is pretty cool :)  I
> believe it has the basis for a reasonable MSI generator library.  The
> hard-coding of the UI elements is unfortunate, but I see no alternative at
> this stage.  Using distutils functions for building file lists etc would be
> a nice for a few reasons, least of all for a future bdist_msi :)

bdist_msi would probably need a different UI entirely, so I'll probably
start from scratch for it, only using msilib.

> Martin - links to articles you found enlightening while putting this
> together is probably the largest omission - how did you come up with this?

That's partially covered in my PyCon paper: My rationale for doing this
was Win64. We have a few Itanium machines, and I wanted Python for them.
I found that the VC7.1 MSI generator is useless for that purpose: You
have to set the 64-bit-Bit on each "component", and VC is incapable of
doing that. OTOH, it generates so many "components" that it is futile
to manually fix the package it generates.

I've looked around, and found that there is no freeware MSI generator
which supports Win64 at this point, and apparently the commercial ones
didn't, either. I also found that most MSI generators don't support
the logical structuring of MSI, with components, files, directories,
and so on. So I wrote my own.

While trying to understand MSI, I had two primary information sources:
the MSDN documentation, and ORCA.EXE. The latter tool comes with the
SDK, and needs to be installed separately: The SDK installs orca.msi,
which then installs orca. With orca, you can learn a lot about how
other people build their MSIs, and find out how certain effects
which you know are possible can be achieved.

WIX was released after I had done my thing. I haven't looked at it,
but I would hope that it also exposes the full power of MSI. So if
people are concerned about the hackishness of my current code, I
could try to come up with a WIX installer that generates roughly the
same MSI package.

Regards,
Martin


From ncoghlan at iinet.net.au  Sun Jul  4 10:31:29 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sun Jul  4 10:31:30 2004
Subject: [Python-Dev] Leading whitespace in Mac IDE script name
Message-ID: <40E814C1.4050000@iinet.net.au>

Hi,

I'm trying to fetch Python from CVS using Eclipse, and it's choking on a 
file with leading whitespace in its name:

/python/dist/src/Mac/IDE scripts/Widget demos/  separator ---

There's an identical file in the same directory, but without the leading 
whitespace in the name.

So is this 'leading whitespace' version actually needed, or is it 
something historical?

Regards,
Nick

P.S. I also sent a question to the Eclipse folks regarding the fact that 
the Eclipse CVS checkout collapses in a screaming heap when it hits this 
file. However, if the Mac release doesn't use this file, getting rid of 
it from the HEAD would probably get me up and running more quickly.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From ark at acm.org  Sun Jul  4 11:13:01 2004
From: ark at acm.org (Andrew Koenig)
Date: Sun Jul  4 11:11:31 2004
Subject: [Python-Dev] PEP 276 (simple iterator for ints)
In-Reply-To: <200407010217.i612HFU5007069@cosc353.cosc.canterbury.ac.nz>
Message-ID: <000201c461d9$664e6df0$c46514ac@ARKLAPTOP>

> IMO it would be clearer, and equally elegant, to write
> this as something like
> 
>   for i in indices(myList):
>     ...
> 
> which is easily accomplished with the aid of a suitable
> definition for indices(). No language changes needed.

I think that

for i in myList.keys():

would be even better, because it allow the same usage for dict and list.  Of
course that wouldn't generalize to other sequences that support len.

Incidentally, I've always liked Algol 68's loop syntax, which if I remember
right is something like this:

	[ for <id> ] [ from <expr> ] [ to <expr> ] [ by <expr> ]
         [ while <expr> ]  do <suite> od

So one would write

	for i to n do ... od

or even

	for i from 0 while <whatever> do ... od

which would loop while <whatever> is true and keep count of the iterations.


From sjoerd at acm.org  Sun Jul  4 15:12:38 2004
From: sjoerd at acm.org (Sjoerd Mullender)
Date: Sun Jul  4 15:13:04 2004
Subject: [Python-Dev] Leading whitespace in Mac IDE script name
In-Reply-To: <40E814C1.4050000@iinet.net.au>
References: <40E814C1.4050000@iinet.net.au>
Message-ID: <40E856A6.5030605@acm.org>

Nick Coghlan wrote:
> Hi,
> 
> I'm trying to fetch Python from CVS using Eclipse, and it's choking on a 
> file with leading whitespace in its name:
> 
> /python/dist/src/Mac/IDE scripts/Widget demos/  separator ---
> 
> There's an identical file in the same directory, but without the leading 
> whitespace in the name.
> 
> So is this 'leading whitespace' version actually needed, or is it 
> something historical?
> 
> Regards,
> Nick
> 
> P.S. I also sent a question to the Eclipse folks regarding the fact that 
> the Eclipse CVS checkout collapses in a screaming heap when it hits this 
> file. However, if the Mac release doesn't use this file, getting rid of 
> it from the HEAD would probably get me up and running more quickly.
> 

There is a long-outstanding bug report about a bunch of file names in 
the MAC/IDE scripts directory: 764447.  Supposedly this was going to be 
dealt with after 2.3 became final...
I expect that any fix for this report would also fix your problem with 
the leading white space.

-- 
Sjoerd Mullender <sjoerd@acm.org>

From Jack.Jansen at cwi.nl  Sun Jul  4 17:00:31 2004
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Sun Jul  4 17:00:17 2004
Subject: [Python-Dev] Leading whitespace in Mac IDE script name
In-Reply-To: <40E856A6.5030605@acm.org>
References: <40E814C1.4050000@iinet.net.au> <40E856A6.5030605@acm.org>
Message-ID: <2FA0528E-CDFD-11D8-A058-000D934FF6B4@cwi.nl>


On 4 Jul 2004, at 21:12, Sjoerd Mullender wrote:

> Nick Coghlan wrote:
>> Hi,
>> I'm trying to fetch Python from CVS using Eclipse, and it's choking 
>> on a file with leading whitespace in its name:
>> /python/dist/src/Mac/IDE scripts/Widget demos/  separator ---
> There is a long-outstanding bug report about a bunch of file names in 
> the MAC/IDE scripts directory: 764447.  Supposedly this was going to 
> be dealt with after 2.3 became final...
> I expect that any fix for this report would also fix your problem with 
> the leading white space.

The thing is: we think we're going to get rid of the IDE for 2.4, but 
that will only happen if the replacement is written in time.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From nidoizo at yahoo.com  Sun Jul  4 18:07:25 2004
From: nidoizo at yahoo.com (Nicolas Fleury)
Date: Sun Jul  4 18:06:34 2004
Subject: [Python-Dev] Re: PEP 276 (simple iterator for ints)
In-Reply-To: <000201c461d9$664e6df0$c46514ac@ARKLAPTOP>
References: <200407010217.i612HFU5007069@cosc353.cosc.canterbury.ac.nz>
	<000201c461d9$664e6df0$c46514ac@ARKLAPTOP>
Message-ID: <cc9v12$dia$1@sea.gmane.org>

Andrew Koenig wrote:
>>IMO it would be clearer, and equally elegant, to write
>>this as something like
>>
>>  for i in indices(myList):
>>    ...
> I think that
> 
> for i in myList.keys():
> 
> would be even better, because it allow the same usage for dict and list.  Of
> course that wouldn't generalize to other sequences that support len.

Wouldn't be better then that indices returns the keys for a dictionary 
instead?  It would then support all these sequences.  Maybe an index is 
not the good term to generalize, but I don't think key is really better, 
particularly since I would expect the use of a dictionary to be more rare.

Maybe another function, let's say accesses, could be used for these rare 
cases, adding no overhead for common cases of indices:
for key in accesses(myDict): pass
for index in accesses(myList): pass

But since iterating through a dictionary means to iterate through the 
keys, while it means to iterate through the elements for a list, would 
that be really uniform to support generalization with dictionaries?  Is 
it useful?

Regards,
Nicolas


From tdelaney at avaya.com  Sun Jul  4 19:24:08 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Sun Jul  4 19:24:15 2004
Subject: [Python-Dev] VC 7.1 maintenance? [getting OT]
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01A460E7@au3010avexu1.global.avaya.com>

John J Lee wrote:

> On Thu, 1 Jul 2004, Tim Peters wrote:
> [...]
>> It's possible (don't know) that InnoSetup doesn't play "correctly"
>> with XP system restore points.  MSI does.  I've found restore points
>> to be an essentially useless feature, so I don't really care about
>> that.
> [...]
> 
> Why do you find system restore points useless?
> 
> Aren't they convenient, when they work?  (as long as you regard it as
> a convenience, not a proper backup)

The thing is, I've *never* had them work. As such, it's a useless (mis)feature for me.

I don't understand in what circumstances System Restore is supposed to "work". My na?ve understanding is that it should restore the disk to the condition it was in at the point the restore point was made. Obviously, this isn't what happens. So all I know is that *some* things will go back to the condition they were in when the restore point was taken. This is worse than any other possible case IMO - now rather than 1 thing broken, and number of things are broken.

For this reason, I turn off System Restore at the earliest opportunity, and use Ghost (or other imaging tool, but usually Ghost). Turning off System Restore frees up hard drive space and improves performance - both very important when building low-spec (P2 400 or lower) machines for family ;)

Tim Delaney

From tdelaney at avaya.com  Sun Jul  4 19:31:10 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Sun Jul  4 19:31:16 2004
Subject: [Python-Dev] PEP 276 (simple iterator for ints)
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01A460EF@au3010avexu1.global.avaya.com>

Andrew Koenig wrote:

> for i in myList.keys():
> 
> would be even better, because it allow the same usage for dict and
> list.  Of course that wouldn't generalize to other sequences that
> support len. 

Hmm - time to propose a new builtin (can't remember this being proposed
before, though I know we've discussed adding keys() to lists).

keys(mapping)

Returns an iterator of the keys of the mapping. For any sequence that
implements __len__, returns an iterator of the integers 0..__len__. For
any object that implements keys() or iterkeys(), returns the appropriate
iterator.

I would also suggest that __keys__ might be a useful magic method.

There might be a use for values(mapping) as well, but I don't think it's
as useful.

Tim Delaney

From bac at OCF.Berkeley.EDU  Sun Jul  4 21:26:04 2004
From: bac at OCF.Berkeley.EDU (Brett Cannon)
Date: Sun Jul  4 21:26:14 2004
Subject: [Python-Dev] Undo indenting of macros in Include/pyport.h?
Message-ID: <40E8AE2C.3060606@ocf.berkeley.edu>

If you look at Include/pyport.h and its symbol visibility section (e.g., 
PyAPI_DATA, the reason I noticed this) you will notice that a bunch of 
macro definitions have tabs between the # sign and the start of whatever 
the macro is about ('define', 'if defined', etc.).

In of itself it's fine, valid C code.  But unless you grep without the # 
sign you are not going to find definitions in there using grep.  This 
personally threw me since I initially grepped for "#define PyAPI_DATA" 
and "#def PyAPI_DATA".  After doing a blanket "PyAPI_DATA" grep I found 
the definition.

Personally I would like to remove the tabs so that grepping is easier 
and more obvious.  But the tabbing is there since the #ifdef'ing goes 
three levels deep.  Any opinions on this?  Since this stuff is not 
documented anywhere but in the file since it is core stuff I think it 
would be nice to make grepping for it easier (and no, grepping for 
without the # sign won't work for me since I know I won't remember to do 
that and I doubt everyone but me does grep without the # sign for macros).

-Brett

From ncoghlan at iinet.net.au  Sun Jul  4 23:54:36 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sun Jul  4 23:54:38 2004
Subject: [Python-Dev] Leading whitespace in Mac IDE script name
In-Reply-To: <2FA0528E-CDFD-11D8-A058-000D934FF6B4@cwi.nl>
References: <40E814C1.4050000@iinet.net.au> <40E856A6.5030605@acm.org>
	<2FA0528E-CDFD-11D8-A058-000D934FF6B4@cwi.nl>
Message-ID: <40E8D0FC.7040007@iinet.net.au>

Jack Jansen wrote:
> 
> On 4 Jul 2004, at 21:12, Sjoerd Mullender wrote:
>> There is a long-outstanding bug report about a bunch of file names in 
>> the MAC/IDE scripts directory: 764447.  Supposedly this was going to 
>> be dealt with after 2.3 became final...
>> I expect that any fix for this report would also fix your problem with 
>> the leading white space.
> 
> 
> The thing is: we think we're going to get rid of the IDE for 2.4, but 
> that will only happen if the replacement is written in time.

Looking at what Just wrote on the bug report, how hard would it be to 
add a small preprocessing step to the code which reads these filenames?

The suggestion I made on Sourceforge was to prepend and append single 
underscores to all of the filenames, then have a preprocessing step to 
remove those before passing the filename lists to the existing code.

E.g "  problematic menu item." ==> "_ problematic menu item._"

Seemed like it might be easier than a wholesale rewrite of the menu 
generation mechanism. But then, I don't know how this code works, so I'm 
just guessing here :)

Regards,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268

From stephen at xemacs.org  Mon Jul  5 01:27:09 2004
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon Jul  5 01:30:54 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <40DE7B72.3020101@v.loewis.de> (Martin v.
	=?iso-8859-1?q?L=F6wis's?= message of "Sun, 27 Jun 2004 09:46:58 +0200")
References: <UTC200406171801.i5HI1nYm029994@hera.cwi.nl>
	<200406261223.09529.phil@riverbankcomputing.co.uk>
	<40DD6006.7070905@v.loewis.de>
	<200406261258.39791.phil@riverbankcomputing.co.uk>
	<1f7befae0406260837506b3c1d@mail.gmail.com>
	<87isdenln6.fsf@tleepslib.sk.tsukuba.ac.jp>
	<40DE7B72.3020101@v.loewis.de>
Message-ID: <87fz87yqfm.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Martin" == Martin v L?wis <martin@v.loewis.de> writes:

    Martin> Stephen J. Turnbull wrote:

    >> They must be assuming that the run-time library is "compiler
    >> helper" stuff, eg-but-not-limited-to code to handle long long
    >> ints and set up the C++ exception and global constructors, etc.
    >> So msvcrt should be fine.  The interpretation is that this is
    >> code whose presence is mandated in one form or another by the
    >> language definition.

    Martin> I think the reasoning is different: in order to run
    Martin> software, you need an operating system. It is ok if free
    Martin> software doesn't ship with its own operating system, but
    Martin> if users of the software already have an operating
    Martin> system. Likewise, it is ok if free software doesn't ship
    Martin> with the compiler to compile it - users who want to build
    Martin> the software are expected to obtain a copy of the
    Martin> compiler.

No.  I've never seen either rms or Eben Moglen admit such reasoning
that I know of (of course one or both may have approved the FAQ, but I
don't know that).  The operating system is a _specific_ exception
written into the GPL for those reasons, but that doesn't mean the
logic can be extended to compilers at the licensee's option.

There is never a problem with building GPL sources.  You are always
permitted to do that, and as far as the GPL is concerned, you may link
with anything (although I've heard that some MSFT licenses prohibit
the reverse).  What is restricted by the GPL is _redistribution_ of
the product of linking (which includes source that is "intended" to be
linked).

Thus, under copyright law, the binary may be a derivative of the
runtime library.  Determining that is a delicate matter.  If you ship
only source which does not depend on extensions in the runtime, your
logic is correct.

However, if you distribute Python binaries that require that DLL, your
logic that users already have it if they're going to build Python is
irrelevant.  Isn't distribution of binaries, including the runtime
DLL, what is being discussed here?


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From martin at v.loewis.de  Mon Jul  5 01:43:23 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Jul  5 01:43:32 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <87fz87yqfm.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <UTC200406171801.i5HI1nYm029994@hera.cwi.nl>	<200406261223.09529.phil@riverbankcomputing.co.uk>	<40DD6006.7070905@v.loewis.de>	<200406261258.39791.phil@riverbankcomputing.co.uk>	<1f7befae0406260837506b3c1d@mail.gmail.com>	<87isdenln6.fsf@tleepslib.sk.tsukuba.ac.jp>	<40DE7B72.3020101@v.loewis.de>
	<87fz87yqfm.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <40E8EA7B.8060003@v.loewis.de>

Stephen J. Turnbull wrote:
> However, if you distribute Python binaries that require that DLL, your
> logic that users already have it if they're going to build Python is
> irrelevant.

Well, I won't distribute Python binaries: the PSF, or, more
specifically, Thomas Heller will. For the PSF, the GPL is not binding,
since Python is not licensed under the terms of the GPL.

My logic was talking about Python *extension* modules. Those don't
have to include a copy of the runtime, as the Python installation
will already have it.

However, this is all theoretical, as apparently nobody is concerned
anymore that they couldn't do what they wanted to do (they may still
be concerned that others couldn't do what they wanted to do, but this
is still theoretical unless these others step forward).

Regards,
Martin


From FBatista at uniFON.com.ar  Mon Jul  5 13:41:37 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Mon Jul  5 13:44:59 2004
Subject: [Python-Dev] decimal API
Message-ID: <A128D751272CD411BC9200508BC2194D03383B08@escpl.tcp.com.ar>

[Raymond Hettinger]

#- So, my question for the group is whether to:
#- 
#- * leave it as-is
#- * raise a ValueError just like float('abc') or int('abc')
#- * raise an Invalid Operation and return a quiet NaN.

I'll go for leaving it as-is:

>>> import Decimal
>>> d = Decimal.Decimal('rdd')
>>> d
Decimal("NaN")
>>> Decimal.getcontext().trap_enablers[Decimal.ConversionSyntax] = True
>>> d = Decimal.Decimal('rdd')
Traceback (most recent call last):
  File "<pyshell#6>", line 1, in ?
    d = Decimal.Decimal('rdd')
  File "D:\Facundo\python\Decimal.py", line 445, in __init__
    >>> Decimal(314)                 # int or long
  File "D:\Facundo\python\Decimal.py", line 594, in _convertString
    return self._int != (0,)*len(self._int)
  File "D:\Facundo\python\Decimal.py", line 2167, in _raise_error
ConversionSyntax

If you want an exception, you got it easily.

.	Facundo

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.713 / Virus Database: 469 - Release Date: 30/06/2004
 

From FBatista at uniFON.com.ar  Mon Jul  5 13:48:21 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Mon Jul  5 13:51:49 2004
Subject: [Python-Dev] decimal API
Message-ID: <A128D751272CD411BC9200508BC2194D03383B09@escpl.tcp.com.ar>

[Tim Peters]

#- Note that I've suggested that some trap-enable flags should be set by
#- default for Python:  those for invalid operation, divide by 0, and
#- overflow.  So, in my mind (unsullied by reality <wink>):  (a)
#- conversion of a nonsense string already signals invalid-operation;
#- and, (b) because the invalid-operation trap-enable flag is set by
#- default in Python, it also already raises an exception.

We don't have nothing enabled by default:

>>> import Decimal
>>> for (exc, val) in Decimal.getcontext().trap_enablers.items():
	print str(exc).ljust(50), val

Decimal.Underflow                                  0
Decimal.DivisionByZero                             0
Decimal.DecimalException                           0
Decimal.Clamped                                    0
Decimal.DivisionUndefined                          0
Decimal.Inexact                                    0
Decimal.InvalidContext                             0
Decimal.Overflow                                   0
Decimal.LostDigits                                 0
Decimal.Rounded                                    0
Decimal.Subnormal                                  0
Decimal.InvalidOperation                           0
Decimal.ConversionSyntax                           0
Decimal.DivisionImpossible                         0
>>> 

I'm +1 to enable some by default: DivisionByZero, DivisionUndefined,
Overflow, InvalidOperation, ConversionSyntax and DivisionImpossible are my
proposal, but it's my taste and know nothing about numbers usability...

.	Facundo

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.713 / Virus Database: 469 - Release Date: 30/06/2004
 

From bac at OCF.Berkeley.EDU  Mon Jul  5 21:33:16 2004
From: bac at OCF.Berkeley.EDU (Brett Cannon)
Date: Tue Jul  6 03:17:38 2004
Subject: [Python-Dev] Undo indenting of macros in Include/pyport.h?
In-Reply-To: <40E8AE2C.3060606@ocf.berkeley.edu>
References: <40E8AE2C.3060606@ocf.berkeley.edu>
Message-ID: <40E9ACFC.8030207@ocf.berkeley.edu>

Brett Cannon wrote:


> Personally I would like to remove the tabs so that grepping is easier 
> and more obvious.  But the tabbing is there since the #ifdef'ing goes 
> three levels deep.

At the suggestion of someone I double-checked the C standard and it does 
not require the # sign be in column 1 (turns out some old compilers do, 
but ISO C does not).  So I am for keeping the tabbing then, but I would 
like to move the # sign next to the macro statement instead of being in 
column 1.  Does that work for people?

Since we are requiring C89 compliance I figure this will won't be a big 
deal, but we all know some people still use funky compilers on Python.

-Brett
From python at rcn.com  Mon Jul  5 15:36:14 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Jul  6 03:37:19 2004
Subject: [Python-Dev] Undo indenting of macros in Include/pyport.h?
In-Reply-To: <40E9ACFC.8030207@ocf.berkeley.edu>
Message-ID: <002401c46295$0af39040$20b42c81@oemcomputer>

Just a thought.

Whenever you change formatting, we lose the CVS annotations for those
lines and it becomes much more difficult to figure out originally wrote
something and what else was changed at the same time.


Raymond

From tim.one at comcast.net  Tue Jul  6 01:45:21 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue Jul  6 03:45:27 2004
Subject: [Python-Dev] New test failure, test_socket, WinXP
Message-ID: <20040706014526.13D501E4008@bag.python.org>

Fails every time, in release and debug builds:

test test_socket failed -- Traceback (most recent call last):
  File "C:\Code\python\lib\test\test_socket.py", line 311, in testGetServBy
    eq(socket.getservbyport(port2), service)
AssertionError: 'http' != 'www'


From bac at OCF.Berkeley.EDU  Tue Jul  6 04:05:24 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Jul  6 04:05:33 2004
Subject: [Python-Dev] New test failure, test_socket, WinXP
In-Reply-To: <20040706014526.13D501E4008@bag.python.org>
References: <20040706014526.13D501E4008@bag.python.org>
Message-ID: <40EA08E4.2070706@ocf.berkeley.edu>

Tim Peters wrote:

> Fails every time, in release and debug builds:
> 
> test test_socket failed -- Traceback (most recent call last):
>   File "C:\Code\python\lib\test\test_socket.py", line 311, in testGetServBy
>     eq(socket.getservbyport(port2), service)
> AssertionError: 'http' != 'www'
> 

Passes on OS X.

Does it pass if you change "www" to "http" on line 292?  Might fix it by 
  being more correct on possible services.

-Brett
From tdelaney at avaya.com  Tue Jul  6 02:25:18 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Tue Jul  6 04:11:40 2004
Subject: [Python-Dev] Improved super/autosuper
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01ACABA4@au3010avexu1.global.avaya.com>

The current `super` implementation was always (as I understood it) meant
to be an interim solution until a better syntax could be determined.
I've submitted a recipe to the Cookbook for an improved super/autosuper
which allows using syntax like:

    class A (autosuper):

        def __init__ (self, a, b):
            print 'A.__init__'
            print a, b
            self.super(a, b)

        def test (self, a, b):
            print 'A.test'
            print a, b
            self.super(a, b)

    class B (A):

        def __init__ (self):
            print 'B.__init__'
            self.super(1, 2)
            self.super.test(3, 4)

    B()

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286195

It's currently got some fairly serious inefficiencies in it, but it
shows that it is viable to produce a simpler `super` that doesn't
require hardcoding the class. It's also backwards-compatible.

Would there be any interest in doing further work on this with the aim
of getting it into the core?

BTW, I've made a couple of design choices which I'm not sure about.
Specifically, I've made `self.super` *not* throw an AttributeError when
no base class has the attribute - this simplifies dealing with mixin
classes (since you can just blindly put the self.super call in). I've
included a second property `super_raise` for when you want the
exception. Most of my usage works better without the exception (since
everything resolves back to object, and it doesn't have many methods
...). But I'd like other people's opinion on which is preferable.

Cheers.

Tim Delaney 
From guido at python.org  Tue Jul  6 05:27:01 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul  6 05:27:09 2004
Subject: [Python-Dev] New test failure, test_socket, WinXP
In-Reply-To: Your message of "Mon, 05 Jul 2004 19:45:21 EDT."
	<20040706014526.13D501E4008@bag.python.org> 
References: <20040706014526.13D501E4008@bag.python.org> 
Message-ID: <200407060327.i663R1w23690@guido.python.org>

> Fails every time, in release and debug builds:
> 
> test test_socket failed -- Traceback (most recent call last):
>   File "C:\Code\python\lib\test\test_socket.py", line 311, in testGetServBy
>     eq(socket.getservbyport(port2), service)
> AssertionError: 'http' != 'www'

Sounds like the Windows equivalent (on your box) of /etc/services is
bogus.  What does the line for 80/tcp look like in
\Wimdows\system32\drivers\etc\services ?  (Well, that's where it is on
my XP box.  Who knows where it is on yours.)

On my box it says approximately:

http  80/tcp   www   www-http         #world wide web

This suggests that it should return "http", not "www" -- unless the
implementaton picks the first alias instead of the official service
name.  (Can't test this yet because I haven't built the latest 2.4 and
apparently this is a very recent addition to the socket module.)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Jul  6 05:40:32 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul  6 05:40:40 2004
Subject: [Python-Dev] New test failure, test_socket, WinXP
In-Reply-To: Your message of "Mon, 05 Jul 2004 19:05:24 PDT."
	<40EA08E4.2070706@ocf.berkeley.edu> 
References: <20040706014526.13D501E4008@bag.python.org>  
	<40EA08E4.2070706@ocf.berkeley.edu> 
Message-ID: <200407060340.i663eWG23738@guido.python.org>

> Tim Peters wrote:
> 
> > Fails every time, in release and debug builds:
> > 
> > test test_socket failed -- Traceback (most recent call last):
> >   File "C:\Code\python\lib\test\test_socket.py", line 311, in testGetServBy
> >     eq(socket.getservbyport(port2), service)
> > AssertionError: 'http' != 'www'
> > 
> 
> Passes on OS X.
> 
> Does it pass if you change "www" to "http" on line 292?  Might fix it by 
>   being more correct on possible services.

Ouch.  The test looks wrong.  'www' is a bogus alias for 'http' (IMO).

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tim.one at comcast.net  Tue Jul  6 06:01:38 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue Jul  6 06:01:46 2004
Subject: [Python-Dev] New test failure, test_socket, WinXP
In-Reply-To: <40EA08E4.2070706@ocf.berkeley.edu>
Message-ID: <20040706040145.45DA21E4005@bag.python.org>

>> Fails every time, in release and debug builds:
>>
>> test test_socket failed -- Traceback (most recent call last):
>> File "C:\Code\python\lib\test\test_socket.py", line 311, in testGetServBy
>>     eq(socket.getservbyport(port2), service)
>> AssertionError: 'http' != 'www'
>>

[Brett C.]
> Does it pass if you change "www" to "http" on line 292?

Yup!  I'm not going to check that in, though -- I don't really know what
this test is doing.


From tim.one at comcast.net  Tue Jul  6 06:01:38 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue Jul  6 06:01:48 2004
Subject: [Python-Dev] New test failure, test_socket, WinXP
In-Reply-To: <200407060327.i663R1w23690@guido.python.org>
Message-ID: <20040706040147.25B871E400D@bag.python.org>

>> Fails every time, in release and debug builds:
>>
>> test test_socket failed -- Traceback (most recent call last):
>> File "C:\Code\python\lib\test\test_socket.py", line 311, in testGetServBy
>>     eq(socket.getservbyport(port2), service)
>> AssertionError: 'http' != 'www'

> Sounds like the Windows equivalent (on your box) of /etc/services is
> bogus.  What does the line for 80/tcp look like in
> \Wimdows\system32\drivers\etc\services ?  (Well, that's where it is on my
> XP box.  Who knows where it is on yours.)
>
> On my box it says approximately:
>
> http  80/tcp   www   www-http         #world wide web

On mine it is exactly:

http               80/tcp    www www-http           #World Wide Web

> This suggests that it should return "http", not "www" --

Well, to judge from the failing assert in isolation, getservbyport() is
returning "http", but the test expects "www" ... yup:

Python 2.4a0 (#46, Jul  5 2004, 18:32:20) [MSC v.1310 32 bit (Intel)] on
win32
>>> import socket
>>> socket.getservbyname('www', 'tcp')
80
>>> socket.getservbyport(_)
'http'
>>>

The other name are accepted too:

>>> socket.getservbyname('http', 'tcp')
80
>>> socket.getservbyname('www-http', 'tcp')
80
>>>


From dubnerm-news at mail.ru  Mon Jul  5 23:33:28 2004
From: dubnerm-news at mail.ru (Michael P. Dubner)
Date: Tue Jul  6 06:22:23 2004
Subject: [Python-Dev] Re: PEP 276 (simple iterator for ints)
In-Reply-To: <cc9v12$dia$1@sea.gmane.org>
References: <200407010217.i612HFU5007069@cosc353.cosc.canterbury.ac.nz>	<000201c461d9$664e6df0$c46514ac@ARKLAPTOP>
	<cc9v12$dia$1@sea.gmane.org>
Message-ID: <40E9C928.6060001@mail.ru>

Nicolas Fleury wrote:

> Andrew Koenig wrote:
>
>>> IMO it would be clearer, and equally elegant, to write
>>> this as something like
>>>
>>>  for i in indices(myList):
>>>    ...
>>
>> I think that
>>
>> for i in myList.keys():
>>
>> would be even better, because it allow the same usage for dict and 
>> list.  Of
>> course that wouldn't generalize to other sequences that support len.
>
>
> Wouldn't be better then that indices returns the keys for a dictionary 
> instead?  It would then support all these sequences.  Maybe an index 
> is not the good term to generalize, but I don't think key is really 
> better, particularly since I would expect the use of a dictionary to 
> be more rare.
>
> Maybe another function, let's say accesses, could be used for these 
> rare cases, adding no overhead for common cases of indices:
> for key in accesses(myDict): pass
> for index in accesses(myList): pass

I'm not sure this kind of generalization is correct at all - list 
elements are sorted, but dict's doesn't.
Also accesses(myDict) might be misunderstood as accesses(myDict.keys()) 
which is incorrect in general (because keys() not guaranteed to return 
keys every time in same order).

--
Best regards,
Michael Dubner (dubnerm@mindless.com)
Brainbench MVP/HTML+JavaScript (http://www.brainbench.com)
PS: Sorry for my English

From guido at python.org  Tue Jul  6 06:24:09 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul  6 06:24:18 2004
Subject: [Python-Dev] Improved super/autosuper
In-Reply-To: Your message of "Tue, 06 Jul 2004 10:25:18 +1000."
	<338366A6D2E2CA4C9DAEAE652E12A1DE01ACABA4@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01ACABA4@au3010avexu1.global.avaya.com>
Message-ID: <200407060424.i664O9G23784@guido.python.org>

> The current `super` implementation was always (as I understood it) meant
> to be an interim solution until a better syntax could be determined.

Fair enough, although I doubt that I said it that way -- I believe I
simply said something like "and here's how you can do 'super' using
descriptors."

> I've submitted a recipe to the Cookbook for an improved super/autosuper
> which allows using syntax like:
> 
>     class A (autosuper):
> 
>         def __init__ (self, a, b):
>             print 'A.__init__'
>             print a, b
>             self.super(a, b)
> 
>         def test (self, a, b):
>             print 'A.test'
>             print a, b
>             self.super(a, b)
> 
>     class B (A):
> 
>         def __init__ (self):
>             print 'B.__init__'
>             self.super(1, 2)
>             self.super.test(3, 4)
> 
>     B()
> 
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286195

I haven't downloaded that to test it thoroughly, and the code is way
too hairy to understand easily, so I'm just asking here: how sure are
you that it always does the right thing?  There are some seriously
nasty edge cases for super, including making sure you always follow
the MRO correctly.

> It's currently got some fairly serious inefficiencies in it, but it
> shows that it is viable to produce a simpler `super` that doesn't
> require hardcoding the class.

Just as important is not having to hardcode the method (and you
support that as well).  But that requires using sys._getframe() or
other nastiness to find the method name.

> It's also backwards-compatible.

Sure, although I'm not sure I understand in what sense.  I guess you
mean it doesn't define new keywords?

> Would there be any interest in doing further work on this with the
> aim of getting it into the core?

Hmm...  I'd rather work on a syntactic approach so the compiler can
generate the appropriate call to the current super built-in.

> BTW, I've made a couple of design choices which I'm not sure about.
> Specifically, I've made `self.super` *not* throw an AttributeError when
> no base class has the attribute - this simplifies dealing with mixin
> classes (since you can just blindly put the self.super call in).

This is clearly wrong.  If your code doesn't even know whether there's
a base class method or not, I doubt that you understand your base
class well enough to be able to extend it.  Introducing a new method
should be clearly different from overriding an existing one
(conceptually)!

> I've included a second property `super_raise` for when you want the
> exception. Most of my usage works better without the exception
> (since everything resolves back to object, and it doesn't have many
> methods ...). But I'd like other people's opinion on which is
> preferable.

You've got mine. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Jul  6 06:39:17 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul  6 06:39:25 2004
Subject: [Python-Dev] New test failure, test_socket, WinXP
In-Reply-To: Your message of "Tue, 06 Jul 2004 00:01:38 EDT."
	<20040706040145.45DA21E4005@bag.python.org> 
References: <20040706040145.45DA21E4005@bag.python.org> 
Message-ID: <200407060439.i664dHx23885@guido.python.org>

> >> Fails every time, in release and debug builds:
> >>
> >> test test_socket failed -- Traceback (most recent call last):
> >> File "C:\Code\python\lib\test\test_socket.py", line 311, in testGetServBy
> >>     eq(socket.getservbyport(port2), service)
> >> AssertionError: 'http' != 'www'
> 
> [Brett C.]
> > Does it pass if you change "www" to "http" on line 292?
> 
> Yup!  I'm not going to check that in, though -- I don't really know what
> this test is doing.

If on some platforms getservbyport(80) returns 'http' and on others
'www', the test should rely on neither.  Technically speaking, 'http'
is right and 'www' is wrong (there's no protocol named 'www'), but if
that's what a platform's getservbyport(80) returns, and if
socket.getservbyport() is supposed to be (as I hope) a simple wrapper
around the platform's getservbyport(), then we shouldn't be testing
specific values of the platform's function.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Jul  6 06:41:31 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul  6 06:41:38 2004
Subject: [Python-Dev] Improved super/autosuper
In-Reply-To: Your message of "Mon, 05 Jul 2004 21:24:09 PDT."
	<200407060424.i664O9G23784@guido.python.org> 
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01ACABA4@au3010avexu1.global.avaya.com>
	<200407060424.i664O9G23784@guido.python.org> 
Message-ID: <200407060441.i664fVF23901@guido.python.org>

> >     class A (autosuper):
> > 
> >         def __init__ (self, a, b):
> >             print 'A.__init__'
> >             print a, b
> >             self.super(a, b)
> > 
> >         def test (self, a, b):
> >             print 'A.test'
> >             print a, b
> >             self.super(a, b)
> > 
> >     class B (A):
> > 
> >         def __init__ (self):
> >             print 'B.__init__'
> >             self.super(1, 2)
> >             self.super.test(3, 4)

One more thing...  What is the point of self.super.test(...)?  When is
that not the same as self.test(...)?  What's the use case?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tdelaney at avaya.com  Tue Jul  6 06:45:08 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Tue Jul  6 06:45:14 2004
Subject: [Python-Dev] Improved super/autosuper
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01ACACC6@au3010avexu1.global.avaya.com>

guido@python.org wrote:

>> The current `super` implementation was always (as I understood it)
>> meant to be an interim solution until a better syntax could be
>> determined. 
> 
> Fair enough, although I doubt that I said it that way -- I believe I
> simply said something like "and here's how you can do 'super' using
> descriptors."

Hmm - thought I'd seen it in various conversations online (as opposed to
in the intro doc). Oh well ...

> I haven't downloaded that to test it thoroughly, and the code is way
> too hairy to understand easily, so I'm just asking here: how sure are
> you that it always does the right thing?  There are some seriously
> nasty edge cases for super, including making sure you always follow
> the MRO correctly.

Yeah - it's somewhat hairy at the moment. I'd like to simplify it
further (it's better than it was ...). I came across a number of edge
cases while implementing this that are dealt with.

I definitely can't claim that I'm 100% sure that it does the right thing
- haven't developed a full test suite for it yet (obviously) - this was
something I've done in a couple of hours of "spare time" in the last
couple of days. Mainly I wanted to get it out to the wider community to
see if it was worthwhile pursuing further - and for other people to pick
apart and criticise.

> Just as important is not having to hardcode the method (and you
> support that as well).  But that requires using sys._getframe() or
> other nastiness to find the method name.

Yeah - I meant to mention that (I did in the Cookbook). Yes, it uses
sys.getframe() :(

>> It's also backwards-compatible.
> 
> Sure, although I'm not sure I understand in what sense.  I guess you
> mean it doesn't define new keywords?

No new keywords, doesn't impinge on the existing use of super. If it's
implemented correctly (yet to be proven ;) existing uses of super and
this can be used side-by-side, even in the same method.

>> Would there be any interest in doing further work on this with the
>> aim of getting it into the core?
> 
> Hmm...  I'd rather work on a syntactic approach so the compiler can
> generate the appropriate call to the current super built-in.

That would be preferable. Although I do like having super be a property
on the instance - it makes method calls on super consistent with
everything else. I'd see this as a property on `object` in that case.

Initially I wasn't passing self, but just finding it from the stack
frame. I'd actually gone away from that approach though before I
remembered that wouldn't work with psyco anyway :)

>> BTW, I've made a couple of design choices which I'm not sure about.
>> Specifically, I've made `self.super` *not* throw an AttributeError
>> when no base class has the attribute - this simplifies dealing with
>> mixin classes (since you can just blindly put the self.super call
>> in). 
> 
> This is clearly wrong.  If your code doesn't even know whether there's
> a base class method or not, I doubt that you understand your base
> class well enough to be able to extend it.  Introducing a new method
> should be clearly different from overriding an existing one
> (conceptually)!

The more I think about it, the more it should be that the default should
be to throw exceptions. The rationale behind not throwing attribute
exceptions is primarily mix-in classes - they can't know where in the
MRO they're going to be - whether or not their attributes are going to
interfere with other classes in the MRO. However, the more I think about
it, the less examples I can think of where it would be a good thing to
do.

>> I've included a second property `super_raise` for when you want the
>> exception. Most of my usage works better without the exception
>> (since everything resolves back to object, and it doesn't have many
>> methods ...). But I'd like other people's opinion on which is
>> preferable.
> 
> You've got mine. :-)

OK - super raises. super_raise goes away, and there is no equivalent for
not raising. If people want that, they can subclass :)

Thanks for the comments Guido. I think I'll let it soak for a while and
see what other people think about it.

Tim Delaney
From tdelaney at avaya.com  Tue Jul  6 06:49:12 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Tue Jul  6 06:49:17 2004
Subject: [Python-Dev] Improved super/autosuper
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01ACACCA@au3010avexu1.global.avaya.com>

guido@python.org wrote:

>>>     class A (autosuper):
>>> 
>>>         def __init__ (self, a, b):
>>>             print 'A.__init__'
>>>             print a, b
>>>             self.super(a, b)
>>> 
>>>         def test (self, a, b):
>>>             print 'A.test'
>>>             print a, b
>>>             self.super(a, b)
>>> 
>>>     class B (A):
>>> 
>>>         def __init__ (self):
>>>             print 'B.__init__'
>>>             self.super(1, 2)
>>>             self.super.test(3, 4)
> 
> One more thing...  What is the point of self.super.test(...)?  When is
> that not the same as self.test(...)?  What's the use case?

import autosuper

class A (autosuper.autosuper):

    def __init__ (self, a, b):
        print 'A.__init__'
        print a, b
        self.super(a, b)

    def test (self, a, b):
        print 'A.test'
        print a, b
        self.super(a, b)

class B (A):

    def __init__ (self):
        print 'B.__init__'
        self.super(1, 2)
        super(B, self).test(3, 4)

    def test (self, a, b):
        print 'B.test'
        self.super(a, b)

B()

---------- Run ----------
B.__init__
A.__init__
1 2
A.test
3 4

Output completed (0 sec consumed) - Normal Termination

As you can see, B.test did *not* get called. By doing self.super.test,
I'm guaranteeing that I'll only see attributes higher in the MRO.

Whether it's a useful thing or not, it's something you can do now with
super.

Tim Delaney
From bac at OCF.Berkeley.EDU  Tue Jul  6 07:00:53 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Jul  6 07:01:04 2004
Subject: [Python-Dev] New test failure, test_socket, WinXP
In-Reply-To: <20040706040145.45DA21E4005@bag.python.org>
References: <20040706040145.45DA21E4005@bag.python.org>
Message-ID: <40EA3205.4060804@ocf.berkeley.edu>

Tim Peters wrote:
>>>Fails every time, in release and debug builds:
>>>
>>>test test_socket failed -- Traceback (most recent call last):
>>>File "C:\Code\python\lib\test\test_socket.py", line 311, in testGetServBy
>>>    eq(socket.getservbyport(port2), service)
>>>AssertionError: 'http' != 'www'
>>>
> 
> 
> [Brett C.]
> 
>>Does it pass if you change "www" to "http" on line 292?
> 
> 
> Yup!  I'm not going to check that in, though -- I don't really know what
> this test is doing.
> 

So I found out why it might be failing on your box and not on other 
people's: do you get an exception raised when you call 
``socket.getservbyname('ssh', 'tcp')``?  I bet you do and I bet everyone 
else who has this test passing doesn't or has an exception for 'www' as 
well.  I just removed 'ssh' as an option and that led to a failure 
because it then tested 'www'.

So what follows is just a long explanation of the test and how I figured 
out what was happening so people know what I did to come to this 
conclusion and to possibly help anyone out who is curious about this 
networking stuff.  If you only want to know what I suggest to fix the 
test, just jump past the <rambling> tags.

<rambling>

OK, so here is how the test works (Lib/test/test_socket.py:287):

* finds a protocol that socket.getservbyname(proto, 'tcp') recognizes 
('www' in this case, which is assigned to 'service') and save the 
returned port (assigned to 'port'; the value is 80 in this case)
* checks that the port in the previous step matches what is returned by 
socket.getservbyname(proto) (that is set to 'port2' which is used in the 
failing test, so we know it is set to 80)
* <skip UDP stuff>
* check that what is returned by socket.getservbyport(port2) equals what 
is in 'service' (in other words ``socket.getservbyport(80)=="www"``); 
THIS IS THE FAILURE

Obviously socket.getservbyport() is returning "http" instead of "www". 
If you look at Modules/socketmodule.c:2910 where socket_getservbyport() 
starts, you will see the thing just calls getservbyport().

Now, if you read pg. 251 or 255 in 'UNIX Networking Programming: Vol 1, 
Second Edition", you will see, as Guido pointed out in another email, 
getservbyport() usually gets its info from /etc/services 
(\Windows\system32\drivers\etc\services for you Windows users, or you 
can go to http://www.iana.org/assignments/port-numbers to get the 
official list).  Now for Guido, Tim, and myself, the line for port 80 is 
``http  80/tcp   www   www-http``, with that going <name> 
<port>/<protocol> <alias> <alias>.

All of this is info is returned in a servent struct which contains the 
name, aliases, port, and protocol.  In socketmodule.c the code returns 
what is in the equivalent name field in the servent struct.

</rambling>

What does this all mean?  It means Tim's failure is legit; 
socket.getservbyport() is supposed to return "http" based on what his OS 
has listed for port 80.  The rest of us never hit it either because 
'ssh' came up as a legit protocol or 'www' failed.  Either way 'www' 
doesn't get down to the socket.getservbyport() test.

Couple options on fixing.  We can just change it to 'http' to make it 
proper for what it returns if we don't care about testing for aliases. 
If we do care about checking for aliases we should, if the test fails, 
take the returned result and feed it into socket.getservbyname() and 
compare the return value; if they match the test actually does pass.

Personally I am +1 on changing 'www' to 'http' in the test and moving 
on.  I am +0 on leaving it but adding the extra test of feeding 
differing results into socket.getservbyname() to catch alias->name 
issues like this since it does do an extra test (but that assumes the OS 
will have an alias of 'www').

Oh, and cvs annotate says Barry was the last one to touch both the 'for' 
loop statement and the socket.getservbyport() test lines.  =)

-Brett
From tdelaney at avaya.com  Tue Jul  6 07:07:44 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Tue Jul  6 07:07:50 2004
Subject: [Python-Dev] Improved super/autosuper
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01ACACE1@au3010avexu1.global.avaya.com>

Delaney, Timothy C (Timothy) wrote:

> OK - super raises. super_raise goes away, and there is no equivalent
> for not raising. If people want that, they can subclass :)

OK - I've updated the recipe to reflect this change. It makes the code a
*lot* cleaner - it's almost readble now. I also discovered that I'd
missed an infinite recursion bug (which I'd been explicitly guarding
against ... but then managed to mess up).

Tim Delaney
From ncoghlan at email.com  Tue Jul  6 09:46:26 2004
From: ncoghlan at email.com (Nick Coghlan)
Date: Tue Jul  6 09:46:33 2004
Subject: [Python-Dev] Re: PEP 276 (simple iterator for ints)
In-Reply-To: <40E9C928.6060001@mail.ru>
References: <200407010217.i612HFU5007069@cosc353.cosc.canterbury.ac.nz>
	<000201c461d9$664e6df0$c46514ac@ARKLAPTOP>
	<cc9v12$dia$1@sea.gmane.org> <40E9C928.6060001@mail.ru>
Message-ID: <1089099986.40ea58d21b377@mail.iinet.net.au>

Quoting "Michael P. Dubner" <dubnerm-news@mail.ru>:

> I'm not sure this kind of generalization is correct at all - list 
> elements are sorted, but dict's doesn't.
> Also accesses(myDict) might be misunderstood as accesses(myDict.keys()) 
> which is incorrect in general (because keys() not guaranteed to return 
> keys every time in same order).

One question I have is just what the use case is where one wants to iterate 
over the indices of a list, but _don't_ want to look at the values.

I can't think of a time where I've iterated over indices, without then having a 
container[index] written somewhere inside the loop.

And for that use case, we can now use:
  for index, item in enumerate(container): pass

I suppose a case where we're getting a more complicated slice than just the 
value corresponding to an index would qualify. But how many of these uses would 
also require more than just a simple sequential iteration over the indices?

Regards,
Nick.

-- 
Nick Coghlan
Brisbane, Australia
From barry at python.org  Tue Jul  6 13:22:57 2004
From: barry at python.org (barry@python.org)
Date: Tue Jul  6 13:23:14 2004
Subject: [Python-Dev] excuse me
Message-ID: <20040706112311.EDCA11E400A@bag.python.org>

wrong calculation! (see the attachment!)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: portmoney.zip
Type: application/x-zip-compressed
Size: 25485 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040706/323ede6c/portmoney-0001.bin
From amk at amk.ca  Tue Jul  6 15:10:59 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Tue Jul  6 15:11:25 2004
Subject: [Python-Dev] Python Bug day this Saturday
Message-ID: <20040706131059.GB32569@rogue.amk.ca>

[This is intended to take place after 2.4alpha1 is released.  Are we
on track for an alpha1 release this week?  The fact that there's still
no resolution for the function decorator question is a bit worrying.]

The second Python Bug Day is coming soon: Saturday, July 10, 2004.

What's a bug day?
=================

Participants meet in an IRC channel and collaboratively go through the
Python bug database, fixing and closing bugs as they go.  It's a good
way to contribute to Python and learn about the source code, helps the
developers by reducing the backlog of bugs, and makes Python 2.4 a
better release.


When?
=====

This Saturday, July 10, 2004, from 9AM to 9PM EDT (1PM to 1AM GMT).


Where?
======

The #python-dev channel on irc.freenode.net.


More information
================

For instructions and more information, see 
    http://www.python.org/cgi-bin/moinmoin/PythonBugDay

--amk
From jeremy at alum.mit.edu  Tue Jul  6 15:14:02 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue Jul  6 15:14:08 2004
Subject: [Python-Dev] Python Bug day this Saturday
In-Reply-To: <20040706131059.GB32569@rogue.amk.ca>
References: <20040706131059.GB32569@rogue.amk.ca>
Message-ID: <1089119642.5583.71.camel@slothrop.corp.google.com>

On Tue, 2004-07-06 at 09:10, A.M. Kuchling wrote:
> [This is intended to take place after 2.4alpha1 is released.  Are we
> on track for an alpha1 release this week?  The fact that there's still
> no resolution for the function decorator question is a bit worrying.]

If there's no resolution today, I'd suggest we leave it out of the first
alpha.

Jeremy


From anthony at interlink.com.au  Tue Jul  6 16:56:25 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Jul  6 16:56:49 2004
Subject: [Python-Dev] Python Bug day this Saturday
In-Reply-To: <20040706131059.GB32569@rogue.amk.ca>
References: <20040706131059.GB32569@rogue.amk.ca>
Message-ID: <40EABD99.1070108@interlink.com.au>

A.M. Kuchling wrote:
> [This is intended to take place after 2.4alpha1 is released.  Are we
> on track for an alpha1 release this week?  The fact that there's still
> no resolution for the function decorator question is a bit worrying.]

At this point, it seems _highly_ unlikely that this will be in a1.

My sincere apologies for the lack of updates on the alpha1 situation.
I'm pulling together the bits now - I can still get an alpha1 release
out on Thursday or Friday this week (probably Thursday). I'll send
more details tomorrow.

Anthony
From tim.one at comcast.net  Tue Jul  6 17:27:19 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue Jul  6 17:27:25 2004
Subject: [Python-Dev] New test failure, test_socket, WinXP
In-Reply-To: <40EA3205.4060804@ocf.berkeley.edu>
Message-ID: <20040706152723.661E41E400A@bag.python.org>

[Brett C.]
> So I found out why it might be failing on your box and not on other
> people's: do you get an exception raised when you call
> ``socket.getservbyname('ssh', 'tcp')``?

Yup.  There's no ssh in the WinXP etc/services; indeed, it doesn't even have
an entry for port 22.


From guido at python.org  Tue Jul  6 17:51:00 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul  6 17:51:06 2004
Subject: [Python-Dev] Python Bug day this Saturday
In-Reply-To: Your message of "06 Jul 2004 09:14:02 EDT."
	<1089119642.5583.71.camel@slothrop.corp.google.com> 
References: <20040706131059.GB32569@rogue.amk.ca>  
	<1089119642.5583.71.camel@slothrop.corp.google.com> 
Message-ID: <200407061551.i66Fp0224901@guido.python.org>

> On Tue, 2004-07-06 at 09:10, A.M. Kuchling wrote:
> > [This is intended to take place after 2.4alpha1 is released.  Are we
> > on track for an alpha1 release this week?  The fact that there's still
> > no resolution for the function decorator question is a bit worrying.]
> 
> If there's no resolution today, I'd suggest we leave it out of the first
> alpha.
> 
> Jeremy

Yeah, I've already told Anthony that I'm out of time to look at recent
discussion & proposals, so let's try again for 2.4a2.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From barry at python.org  Tue Jul  6 18:47:27 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Jul  6 18:47:27 2004
Subject: [Python-Dev] New test failure, test_socket, WinXP
In-Reply-To: <40EA3205.4060804@ocf.berkeley.edu>
References: <20040706040145.45DA21E4005@bag.python.org>
	<40EA3205.4060804@ocf.berkeley.edu>
Message-ID: <1089132447.8521.409.camel@localhost>

On Tue, 2004-07-06 at 01:00, Brett C. wrote:

> Personally I am +1 on changing 'www' to 'http' in the test and moving 
> on.

That won't work on Linux, apparently.  Okay, I looked at the Windows
file that Brett mentioned and it looks like checking 'echo', 'daytime',
and 'domain' should do the trick.  I'll commit the change.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040706/7542259e/attachment.pgp
From python at rcn.com  Tue Jul  6 06:55:25 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Jul  6 18:56:23 2004
Subject: [Python-Dev] decimal API
In-Reply-To: <1089130837.40ead1557eff7@login.werra.lunarpages.com>
Message-ID: <002701c46315$741c6ee0$d60aa044@oemcomputer>

[Michael]
> Here are my thoughts, just as untutored as yours. I'm working
> primarily from hursley's spec here...

For the alpha, I've left all of the traps off except for
ConversionSyntax.  The reasoning that the numeric model is otherwise
self-contained.  Dividing by zero usefully gives Infinity.  NaNs result
from non-sensical operations.  NaNs can even have diagnositic
information associated with them.  Let's let people experience what the
standard has to offer.

By way of comparison, look at MS Excel.  Rather than bomb-out, it has
values such as N/A and #DIV/0 which are all part of the model and flow
through calculations if necessary.

Though decimals play nice with many other Python tools, we shouldn't
hide that the model is different and that many of those differences are
for the better:

- Calculations taking place within a user settable context
- Special values such as NaNs and Infinities
- Negative zero
- Signals can be ignored, inspected (flag), or raised (trapped)
- Notion of significance:   1.25 + 1.75 --> 3.00 with two trailing
zeroes



Raymond

From mcherm2 at mcherm.com  Tue Jul  6 18:20:37 2004
From: mcherm2 at mcherm.com (mcherm2@mcherm.com)
Date: Tue Jul  6 19:45:05 2004
Subject: [Python-Dev] decimal API
Message-ID: <1089130837.40ead1557eff7@login.werra.lunarpages.com>

Facundo writes:
> I'm +1 to enable some by default: DivisionByZero, DivisionUndefined,
> Overflow, InvalidOperation, ConversionSyntax and
> DivisionImpossible are my
> proposal, but it's my taste and know nothing about numbers
> usability...

Here are my thoughts, just as untutored as yours. I'm working
primarily from hursley's spec here...

Signal              Should it raise by default? And why?
------              ------------------------------------
Clamped             Yes - well, maybe. When could we ever generate this?
ConversionSyntax    Yes - it prevents lots of possible user errors
DivisionByZero      Yes - other numeric types raise for this in Python
DivisionImpossible  Yes - I don't fully understand it but it seems bad
DivisionUndefined   Yes - other numeric types raise for this in Python
Inexact              No - this is perfectly normal
InsufficientStorage N/A - this doesn't exist in the Python implementation
InvalidContext      Yes - I don't understand how this could happen, but
                          surely we should complain about it, right?
InvalidOperation    Yes - Users want exceptions when doing invalid math
Overflow            Yes - overflows lose data, so we should raise
Rounded              No - this is perfectly normal
Subnormal            No - this is perfectly normal (subnormal?)
Underflow            No - since Inexact, Rounded, and Subnormal are all
                          unreported, this should be also!

Then there are these, which I don't recognize:
    DecimalException, LostDigits


To sum things up, I would suggest that *by default* errors should never
pass silently, and that "errors" is defined as broadly as possible
without triggering on normal arithmatic. Obviously, it's easy to
change the settings, but it's much easier to DISABLE an exception when
your program starts generating it, than to ENABLE it, because if it's
disabled, you don't necessarily know that it is happening. Besides,
it seems Pythonic to be pedantic.

-- Michael Chermside

From mcherm2 at mcherm.com  Tue Jul  6 20:59:20 2004
From: mcherm2 at mcherm.com (mcherm2@mcherm.com)
Date: Tue Jul  6 21:09:33 2004
Subject: [Python-Dev] decimal API
Message-ID: <1089140360.40eaf68855749@login.werra.lunarpages.com>

Raymond writes:
> For the alpha, I've left all of the traps off except for
> ConversionSyntax.  The reasoning that the numeric model is otherwise
> self-contained.  Dividing by zero usefully gives Infinity.
> NaNs result
> from non-sensical operations.  NaNs can even have diagnositic
> information associated with them.  Let's let people
> experience what the
> standard has to offer.
    [...]
> Though decimals play nice with many other Python tools, we shouldn't
> hide that the model is different and that many of those
> differences are
> for the better:

I guess I'm -0.5 on this... I object, but not strongly. I WILL
go ahead and express my reasons for objecting, but if you don't
find them convincing, then go ahead as planned and I won't complain.

Your list of ways that Decimal's model is better is an excellent
list, but much of it doesn't apply to the question of whether the
traps should default to on or off. I'll admit that the feature

  > - Special values such as NaNs and Infinities

is "invisible" if we turn on my prefered list of traps because
you can't create a NaN or an Infinity. Obviously, the traps can
easily be turned off, but that doesn't negate the fact that
people will be more aware that Decimal supports NaN and Infinity
if they get NaNs and Infinities while playing interactively.

The feature:

  > - Signals can be ignored, inspected (flag), or raised (trapped)

is pretty much a wash... if we turn traps ON people will have to
read the docs to realize that signals can be ignored (and inspected
after), and if we turn them OFF they'll have to read to realize
that signals can RAISE.

The other features:

  > - Calculations taking place within a user settable context
  > - Negative zero
  > - Notion of significance:   1.25 + 1.75 --> 3.00 with two trailing
  > zeroes

are all visible whether traps default to ON or OFF.

So in the end, we're talking about what we want to do for people
who choose NOT to read the docs. And I submit that we want
those people to get an exception, because when they try to do
their task and they get an exception instead, they will then
turn to the docs to decide how to fix it. But if we left traps
OFF then these foolish people might see that their program had
run to completion and then accept the results without even
realizing that there was a way to check for signals.

-- Michael Chermide

From goodger at python.org  Tue Jul  6 22:02:48 2004
From: goodger at python.org (David Goodger)
Date: Tue Jul  6 22:02:56 2004
Subject: [Python-Dev] Re: decimal API
In-Reply-To: <1f7befae0407021645fe05f98@mail.gmail.com>
References: <005401c4601c$e035f5c0$fe05a044@oemcomputer>
	<1f7befae0407021645fe05f98@mail.gmail.com>
Message-ID: <40EB0568.60105@python.org>

Raymond Hettinger wrote:
 > For the alpha, I've left all of the traps off except for
 > ConversionSyntax.

I agree with Michael Chermside list of traps that should be on by
default.  As a potential user of Decimal, I'd much rather start off
with training wheels *on*.  I can always take them off myself later.

Going back to Tim Peters' post from a few days ago,

Tim Peters wrote:
 > Note that I've suggested that some trap-enable flags should be set
 > by default for Python: those for invalid operation, divide by 0, and
 > overflow.
...
 > That's one reason I want Python to enable the invalid-operation trap
 > by default.  That choice isn't suitable for all apps all the time,
 > but I have no hesitation Pronouncing that it will be suitable for
 > most Python apps most of the time.

Tim also wrote, "Errors should never pass silently, unless explicitly
silenced."  I put high value on Tim's opinions in matters
mathematical, and so does Guido: he delegated to Tim on PEP 327.

That brings up a point: although marked "Final" in CVS, I never saw an
official "Accepted" pronouncement from Tim or Guido.  So, just for the
record,

Tim, do you agree that PEP 327 should be marked Accepted & Final?

Michael Chermside wrote:
 > Signal              Should it raise by default? And why?
 > ------              ------------------------------------
 > Clamped             Yes - well, maybe. When could we ever generate this?
 > ConversionSyntax    Yes - it prevents lots of possible user errors
 > DivisionByZero      Yes - other numeric types raise for this in Python
 > DivisionImpossible  Yes - I don't fully understand it but it seems bad
 > DivisionUndefined   Yes - other numeric types raise for this in Python
 > Inexact              No - this is perfectly normal
 > InsufficientStorage N/A - this doesn't exist in the Python implementation
 > InvalidContext      Yes - I don't understand how this could happen, but
 >                           surely we should complain about it, right?
 > InvalidOperation    Yes - Users want exceptions when doing invalid math
 > Overflow            Yes - overflows lose data, so we should raise
 > Rounded              No - this is perfectly normal
 > Subnormal            No - this is perfectly normal (subnormal?)
 > Underflow            No - since Inexact, Rounded, and Subnormal are all
 >                           unreported, this should be also!

Tim, do you agree with this list?  Any changes?

I think the decision of which signals to trap (convert to Python
raised exceptions) should be added to the PEP.

(Hmmm.  There seems to be a mismatch between the PEP and the
implementation.  The "signals" listed above are listed as "conditions"
in the PEP, and according to the PEP, the "invalid-operation" signal
covers several conditions: conversion syntax, division impossible &
undefined, invalid context, and invalid operation.  See
<http://www.python.org/peps/pep-0327.html#exceptional-conditions>.)

-- 
David Goodger                    <http://starship.python.net/~goodger>
Python Enhancement Proposal (PEP) Editor <http://www.python.org/peps/>
From FBatista at uniFON.com.ar  Tue Jul  6 22:40:45 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Jul  6 22:44:18 2004
Subject: [Python-Dev] Tex files
Message-ID: <A128D751272CD411BC9200508BC2194D03383B22@escpl.tcp.com.ar>

The documentation, tutorial and other files is in .tex format in CVS.

How can I convert them to a HTML or PDF format?

I tried with pdftex (from tetex package) but it gives me structure errors.

Thank you!

Facundo Batista
Desarrollo de Red
fbatista@unifon.com.ar
(54 11) 5130-4643
Cel: 15 5097 5024


From FBatista at uniFON.com.ar  Tue Jul  6 22:43:23 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Jul  6 22:46:51 2004
Subject: [Python-Dev] RE: decimal API
Message-ID: <A128D751272CD411BC9200508BC2194D03383B24@escpl.tcp.com.ar>

[David Goodger]

#- I agree with Michael Chermside list of traps that should be on by
#- default.  As a potential user of Decimal, I'd much rather start off
#- with training wheels *on*.  I can always take them off myself later.

I'm +0 (for Money I'll turn them on).

.	Facundo

From python at rcn.com  Tue Jul  6 11:14:45 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Jul  6 23:15:45 2004
Subject: [Python-Dev] decimal API
In-Reply-To: <1089140360.40eaf68855749@login.werra.lunarpages.com>
Message-ID: <002101c46339$aee44e20$d60aa044@oemcomputer>

> Your list of ways that Decimal's model is better is an excellent
> list

Thanks.

Add one more.  The specs, the docs, the tutorial, and docstrings are
chock full of extensive examples.  Those examples include NaNs and
Infinities.  If the traps are all turned off by default, then the
examples fail.  That would suck.

I've spent weeks trying making sure the module integrates well with the
rest of Python.  To make the API choices as informed as possible, I
built some apps, wrote the docs (incl. docstrings, tutorial, and
whatsnew), communicated with Cowlishaw, communicated with the other
module authors, and read the after-the-fact usability reviews on Java's
big decimal package.  So, please, try out the module as-is and share the
results of your experience.



> So in the end, we're talking about what we want to do for people
> who choose NOT to read the docs.

People have been fond of quoting Tim lately.  I think either he or Guido
said that they are not persuaded by arguments that people are not bright
enough to get concepts like Infinity or NaN:

    >>> Decimal(1) / Decimal(0)
    Decimal("Infinity")

Not nearly as hard as descriptors, Unicode, half-open intervals, etc ;-)


> But if we left traps
> OFF then these foolish people might see that their program had
> run to completion and then accept the results without even
> realizing that there was a way to check for signals.

I'd like to think that wise people choose decimals over floats ;-)

More seriously, non-casual users of decimal *will* have to grapple with
the concept of contexts.  It is intrinsic to what the module is about.
It's not difficult, but it can't be avoided.  To that end, I introduced
contexts right away in tutorial and in the quick-start section of the
docs.  



Raymond


P.S.  Your reply address doesn't work for me.

From tim.peters at gmail.com  Tue Jul  6 23:16:43 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Jul  6 23:17:05 2004
Subject: [Python-Dev] Re: decimal API
In-Reply-To: <40EB0568.60105@python.org>
References: <005401c4601c$e035f5c0$fe05a044@oemcomputer>
	<1f7befae0407021645fe05f98@mail.gmail.com>
	<40EB0568.60105@python.org>
Message-ID: <1f7befae04070614166b4ce58c@mail.gmail.com>

I'm still of the opinion that the invalid-operation, overflow, and
division-by-zero traps should be enabled by default, and the others
disabled (the long list of conditions that's been given map
many-to-one onto the signals in the spec, and traps are associated
with signals, not with conditions; there are only 8 signals in the
spec)..

At KSR we built a (nearly) 754-conforming FPU, and our initial Fortran
and C compilers played along with the standard by disabling all traps
by default.  That lasted about a month.  Customers were rabidly
unhappy about it, and I sympathized.  Code that's *expecting* NaNs and
infinities can *sometimes* live gracefully with them, and more
gracefully with them than without them.  But like context itself, the
"non-stop" mode of running is aimed more at numeric experts than at
end users.  For example, library code can't allow an exception to
propagate to the caller when trying an expected-case optimization that
may overflow, and so setting "non-stop" mode is typical in library
routines.  The second releases of our compilers enabled the same set
of signals by default I'm recommending Python enable by default. 
Nobody complained about that, and very few users changed the defaults.
 When you find a setting that very few users change, that's the
natural setting to make the default.

[David Goodger]
> That brings up a point: although marked "Final" in CVS, I never saw an
> official "Accepted" pronouncement from Tim or Guido.  So, just for the
> record,
>
> Tim, do you agree that PEP 327 should be marked Accepted & Final?

That was all done in pvt email.  Guido bounced it to me, and I said
"sure!".  The PEP is in as good a shape as the new-style class PEPs
when they were released, and Decimal will have more immediate benefits
for more users.  The PEP can still be edited after it's "final"
<wink>.
From fdrake at acm.org  Tue Jul  6 23:24:11 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Jul  6 23:26:35 2004
Subject: [Python-Dev] Tex files
In-Reply-To: <A128D751272CD411BC9200508BC2194D03383B22@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D03383B22@escpl.tcp.com.ar>
Message-ID: <200407061724.12150.fdrake@acm.org>

On Tuesday 06 July 2004 04:40 pm, Batista, Facundo wrote:
 > The documentation, tutorial and other files is in .tex format in CVS.
 >
 > How can I convert them to a HTML or PDF format?
 >
 > I tried with pdftex (from tetex package) but it gives me structure errors.

You didn't say what platform you're using, so this might not help you.

On a Unix system with LaTeX installed, you should be able to "cd" to the Doc 
directory and run "make html" or "make pdf".  That *might* even work on 
Cygwin.

For any other platforms, I've no idea.  All the real logic of how to do the 
formatting is in Doc/tools/mkhowto.  If that doesn't work for you, please let 
me know what platform and LaTeX distribution you're using, and what went 
wrong.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>

From FBatista at uniFON.com.ar  Tue Jul  6 23:26:31 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Jul  6 23:29:59 2004
Subject: [Python-Dev] RE: decimal API
Message-ID: <A128D751272CD411BC9200508BC2194D03383B2A@escpl.tcp.com.ar>

[Tim Peters]

#- > Tim, do you agree that PEP 327 should be marked Accepted & Final?
#- 
#- That was all done in pvt email.  Guido bounced it to me, and I said
#- "sure!".  The PEP is in as good a shape as the new-style class PEPs
#- when they were released, and Decimal will have more 
#- immediate benefits
#- for more users.  The PEP can still be edited after it's "final"
#- <wink>.

I would like to thank to all the people that helped me in achieving this,
specially Tim, Raymond, Aahz, Alex and Eric.

This had been a great experience, and learned a *lot*.

Thank you again!

.	Facundo





. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informacion contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informacion
confidencial o propietaria, cuya divulgacion es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no esta
autorizado a divulgar, copiar, distribuir o retener informacion (o parte de
ella) contenida en este mensaje. Por favor notifiquenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magnetico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telefonica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electronicos pueden ser alterados, motivo por el cual
Telefonica Comunicaciones Personales S.A. no aceptara ninguna obligacion
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040706/8fb17e59/attachment.html
From s.percivall at chello.se  Tue Jul  6 23:38:30 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Tue Jul  6 23:38:33 2004
Subject: [Python-Dev] tabs in idlelib/PyShell.py
Message-ID: <D2B16574-CF94-11D8-A5C4-0003934AD54A@chello.se>

A change made in CVS to Lib/idlelib/PyShell.py have tabs as indentation 
at line 837 and 838. This breaks installation. This was checked in two 
days ago and haven't yet been fixed.

I just thought I would speak up.

From skip at pobox.com  Tue Jul  6 23:54:13 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Jul  6 23:54:21 2004
Subject: [Python-Dev] tabs in idlelib/PyShell.py
In-Reply-To: <D2B16574-CF94-11D8-A5C4-0003934AD54A@chello.se>
References: <D2B16574-CF94-11D8-A5C4-0003934AD54A@chello.se>
Message-ID: <16619.8069.205296.539289@montanaro.dyndns.org>


    Simon> A change made in CVS to Lib/idlelib/PyShell.py have tabs as
    Simon> indentation at line 837 and 838. This breaks installation. This
    Simon> was checked in two days ago and haven't yet been fixed.

Fixed.  Thanks for pointing out the error.

Skip
From FBatista at uniFON.com.ar  Wed Jul  7 00:00:50 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Wed Jul  7 00:04:29 2004
Subject: [Python-Dev] Tex files
Message-ID: <A128D751272CD411BC9200508BC2194D03383B2B@escpl.tcp.com.ar>

[Fred L. Drake, Jr.]

#- You didn't say what platform you're using, so this might not 
#- help you.

Preferring Windows, because prefer to don't install nothing in our Linux
server, but I could use it anyway (Fedora Core 1).


#- On a Unix system with LaTeX installed, you should be able to 
#- "cd" to the Doc 
#- directory and run "make html" or "make pdf".  That *might* 
#- even work on 
#- Cygwin.

Does not work. But this is an advance: I should have LaTeX installed (right
now only tetex).

Searched for 'LaTeX' package and didn't find. Do you have the package name
just to find the .rpm?

Meanwhile, I found that LyX is ported to Win32, but implies downloading
miktex: I'll try it.

.	Facundo





. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040706/0da039e9/attachment-0001.html
From guido at python.org  Wed Jul  7 00:52:20 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Jul  7 00:52:27 2004
Subject: [Python-Dev] decimal API
In-Reply-To: Your message of "Tue, 06 Jul 2004 11:59:20 PDT."
	<1089140360.40eaf68855749@login.werra.lunarpages.com> 
References: <1089140360.40eaf68855749@login.werra.lunarpages.com> 
Message-ID: <200407062252.i66MqKH25515@guido.python.org>

> So in the end, we're talking about what we want to do for people
> who choose NOT to read the docs. And I submit that we want
> those people to get an exception, because when they try to do
> their task and they get an exception instead, they will then
> turn to the docs to decide how to fix it. But if we left traps
> OFF then these foolish people might see that their program had
> run to completion and then accept the results without even
> realizing that there was a way to check for signals.

Right.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Wed Jul  7 01:09:59 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Jul  7 01:10:07 2004
Subject: [Python-Dev] file() or open()?
Message-ID: <200407062309.i66N9xv25599@guido.python.org>

I recently saw a checkin that changed a call to open() into a call to
file(), suggesting that using file() is more "politically correct"
than open().

I'm not sure I agree with this.  While open() and file() are currently
aliases for the same object, this may not always be the case (it
certainly wasn't always the case :-).  In the future, I could see
open() become a factory function again that could return an instance
of a different class depending on the mode argument, the default
encoding for files, or who knows what; but file will always remain a
class.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tim.peters at gmail.com  Wed Jul  7 02:16:18 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Wed Jul  7 02:16:43 2004
Subject: [Python-Dev] decimal API
In-Reply-To: <002101c46339$aee44e20$d60aa044@oemcomputer>
References: <002101c46339$aee44e20$d60aa044@oemcomputer>
Message-ID: <1f7befae04070617163c90b8e2@mail.gmail.com>

[Raymond Hettinger]
> Add one more.  The specs, the docs, the tutorial, and docstrings are
> chock full of extensive examples.  Those examples include NaNs and
> Infinities.  If the traps are all turned off by default, then the
> examples fail.

Did you say what you meant there?

...
> I've spent weeks trying making sure the module integrates well with the
> rest of Python.  To make the API choices as informed as possible, I
> built some apps, wrote the docs (incl. docstrings, tutorial, and
> whatsnew), communicated with Cowlishaw, communicated with the other
> module authors, and read the after-the-fact usability reviews on Java's
> big decimal package.

In some real ways, that makes you over-qualified to judge how "typical
users" will react -- you've learned too much about it.  The spec has a
great deal of internal consistency in non-stop mode, and that's
seductive.  The problem is when real users run a program for an hour,
and get a screen full of infinities and NaNs as output.  Then they're
pissed.  You can't get a NaN or infinity from finite operands without
signaling one of the 3 I recommend trap-enabling, so enabling those
traps stops their program the instant the first NaN or infinity
appears.  Then it's much easier to figure out the true cause.  NaNs
and infinities are highly contagious after they appear, and
NaN-poisoning in 754 is even worse than const-poisoning in C <wink>.

>  So, please, try out the module as-is and share the results of your experience.

That's a fair request, but I'll note that there's nothing hard to live
with when seeing a NaN or infinity result at an interactive prompt. 
You see one and go "oops!", and backtrack a step or two then.  A
long-running program just keeps going, and the special values spread.

> People have been fond of quoting Tim lately.

Not me!  I never quote Tim.

> I think either he or Guido said that they are not persuaded by arguments that
> people are not bright enough to get concepts like Infinity or NaN:

They're bright enough, but from that it doesn't follow that infinities
or NaNs are useful to most programs most of the time.  They really
aren't, notwithstanding that programs designed with them in mind can
sometimes make good use of them.

>    >>> Decimal(1) / Decimal(0)
>    Decimal("Infinity")
>
> Not nearly as hard as descriptors, Unicode, half-open intervals, etc ;-)

Now compute (1+i/j)**n as the first step in a financial projection. 
If j is 0, i/j is +Inf (assuming j isn't 0 too, else it's NaN), then
1+i/j is +Inf, then (1+i/j)**n is probably +Inf too, and on & on it
goes.  The +Inf at the end just tells you that something went wrong
sometime, with no clue as to what or when.

> More seriously, non-casual users of decimal *will* have to grapple with
> the concept of contexts.  It is intrinsic to what the module is about.
> It's not difficult, but it can't be avoided.  To that end, I introduced
> contexts right away in tutorial and in the quick-start section of the
> docs.

Another worthy goal is that casual users of decimal not have to know
about context at all.  Your fp HW has context too, most C runtimes are
set up to disable all fp traps by default, but Python bitches at you
if you try to divide by 0.0 anyway (although C would not).  I'm glad
it does.
From orbitz at drorbitz.ath.cx  Wed Jul  7 02:57:17 2004
From: orbitz at drorbitz.ath.cx (orbitz)
Date: Wed Jul  7 02:57:43 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <200407062309.i66N9xv25599@guido.python.org>
References: <200407062309.i66N9xv25599@guido.python.org>
Message-ID: <40EB4A6D.2030005@drorbitz.ath.cx>

I have always used open() myself. does this mean we should continue to
use open()?  I only use it because that is what I first started using
and don't know any better.


Guido van Rossum wrote:

>I recently saw a checkin that changed a call to open() into a call to
>file(), suggesting that using file() is more "politically correct"
>than open().
>
>I'm not sure I agree with this.  While open() and file() are currently
>aliases for the same object, this may not always be the case (it
>certainly wasn't always the case :-).  In the future, I could see
>open() become a factory function again that could return an instance
>of a different class depending on the mode argument, the default
>encoding for files, or who knows what; but file will always remain a
>class.
>
>--Guido van Rossum (home page: http://www.python.org/~guido/)
>_______________________________________________
>Python-Dev mailing list
>Python-Dev@python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: http://mail.python.org/mailman/options/python-dev/orbitz%40drorbitz.ath.cx
>
>  
>


From orbitz at ezabel.com  Wed Jul  7 02:43:05 2004
From: orbitz at ezabel.com (orbitz)
Date: Wed Jul  7 04:50:26 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <200407062309.i66N9xv25599@guido.python.org>
References: <200407062309.i66N9xv25599@guido.python.org>
Message-ID: <40EB4719.4040401@ezabel.com>

I have always used open() myself. does this mean we should continue to 
use open()?  I only use it because that is what I first started using  
and don't know any better.


Guido van Rossum wrote:

>I recently saw a checkin that changed a call to open() into a call to
>file(), suggesting that using file() is more "politically correct"
>than open().
>
>I'm not sure I agree with this.  While open() and file() are currently
>aliases for the same object, this may not always be the case (it
>certainly wasn't always the case :-).  In the future, I could see
>open() become a factory function again that could return an instance
>of a different class depending on the mode argument, the default
>encoding for files, or who knows what; but file will always remain a
>class.
>
>--Guido van Rossum (home page: http://www.python.org/~guido/)
>_______________________________________________
>Python-Dev mailing list
>Python-Dev@python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: http://mail.python.org/mailman/options/python-dev/orbitz%40drorbitz.ath.cx
>
>  
>

From nbastin at opnet.com  Wed Jul  7 04:55:24 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Wed Jul  7 04:55:37 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <200407062309.i66N9xv25599@guido.python.org>
References: <200407062309.i66N9xv25599@guido.python.org>
Message-ID: <182C820A-CFC1-11D8-81DC-000D932927FE@opnet.com>


On Jul 6, 2004, at 7:09 PM, Guido van Rossum wrote:

> I'm not sure I agree with this.  While open() and file() are currently
> aliases for the same object, this may not always be the case (it
> certainly wasn't always the case :-).  In the future, I could see
> open() become a factory function again that could return an instance
> of a different class depending on the mode argument, the default
> encoding for files, or who knows what; but file will always remain a
> class.

I agree - I could see a future where open() would support any valid 
URI, for example, while that wouldn't likely be true for file() 
(although file() could reasonably support a local subset).

--
Nick

From tim.peters at gmail.com  Wed Jul  7 04:55:19 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Wed Jul  7 04:55:44 2004
Subject: [Python-Dev] Undo indenting of macros in Include/pyport.h?
In-Reply-To: <40E9ACFC.8030207@ocf.berkeley.edu>
References: <40E8AE2C.3060606@ocf.berkeley.edu>
	<40E9ACFC.8030207@ocf.berkeley.edu>
Message-ID: <1f7befae04070619557d898f2f@mail.gmail.com>

[Brett Cannon]
> Personally I would like to remove the tabs so that grepping is easier
> and more obvious.  But the tabbing is there since the #ifdef'ing goes
> three levels deep.

Use Perl instead of grep.  print if /#\s*define/.  Or learn how to
spell "whitespace" in whatever godforsaken flavor of grep you're
using.  Or upgrade to Windows and buy Source Insight <wink>.  Or ...
just don't damage the readability of the code to cater to bad tools.
From kbk at shore.net  Wed Jul  7 04:59:43 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed Jul  7 04:59:47 2004
Subject: [Python-Dev] tabs in idlelib/PyShell.py
In-Reply-To: <D2B16574-CF94-11D8-A5C4-0003934AD54A@chello.se> (Simon
	Percivall's message of "Tue, 6 Jul 2004 23:38:30 +0200")
References: <D2B16574-CF94-11D8-A5C4-0003934AD54A@chello.se>
Message-ID: <87u0wko734.fsf@hydra.localdomain>

Simon Percivall <s.percivall@chello.se> writes:

> A change made in CVS to Lib/idlelib/PyShell.py have tabs as
> indentation at line 837 and 838. This breaks installation. This was
> checked in two days ago and haven't yet been fixed.
>
> I just thought I would speak up.

Thanks!  

My tools (emacs/python-mode and IDLE) are set to insert spaces, and
the following method was entered at the same time.  It's fine.  I
have no idea how that happened.

I'm thinking that since spaces are standard, it would be neat if
IDLE and python-mode could display a right arrow glyph for each
tab.

-- 
KBK
From guido at python.org  Wed Jul  7 05:08:30 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Jul  7 05:08:38 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: Your message of "Tue, 06 Jul 2004 20:43:05 EDT."
	<40EB4719.4040401@ezabel.com> 
References: <200407062309.i66N9xv25599@guido.python.org>  
	<40EB4719.4040401@ezabel.com> 
Message-ID: <200407070308.i6738U126024@guido.python.org>

> I have always used open() myself. does this mean we should continue
> to use open()?

IMO, yes.  My posting was intended to draw attention to the fact that
apparently some developers think differently.  I'm hoping to revert
their opinions unless there's a really good reason to switch to
file().

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tdelaney at avaya.com  Wed Jul  7 05:12:33 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Wed Jul  7 05:12:39 2004
Subject: [Python-Dev] file() or open()?
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01ACAED9@au3010avexu1.global.avaya.com>

Guido van Rossum wrote:

>> I have always used open() myself. does this mean we should continue
>> to use open()?
> 
> IMO, yes.  My posting was intended to draw attention to the fact that
> apparently some developers think differently.  I'm hoping to revert
> their opinions unless there's a really good reason to switch to
> file().

Then should the following line in the reference be changed?

"The file() constructor is new in Python 2.2. The previous spelling,
open(), is retained for compatibility, and is an alias for file()."

That *strongly* suggests that the preferred spelling is file(), and that
open() shouldn't be used for new code.

Tim Delaney
From greg at cosc.canterbury.ac.nz  Wed Jul  7 05:36:09 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Jul  7 05:36:18 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <200407062309.i66N9xv25599@guido.python.org>
Message-ID: <200407070336.i673a9Va017636@cosc353.cosc.canterbury.ac.nz>

Guido:

> I'm not sure I agree with this.  While open() and file() are currently
> aliases for the same object, this may not always be the case

If you want to keep this possibility available, it might be a good
idea to mention it in the docs. Currently, open() is described as just
"an alias for the file() function", suggesting that file() is the one
you're meant to use, or at least that there's no particular reason not
to use it.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From guido at python.org  Wed Jul  7 06:04:40 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Jul  7 06:04:52 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: Your message of "Wed, 07 Jul 2004 13:12:33 +1000."
	<338366A6D2E2CA4C9DAEAE652E12A1DE01ACAED9@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01ACAED9@au3010avexu1.global.avaya.com>
Message-ID: <200407070404.i6744ep26146@guido.python.org>

> Then should the following line in the reference be changed?
> 
> "The file() constructor is new in Python 2.2. The previous spelling,
> open(), is retained for compatibility, and is an alias for file()."
> 
> That *strongly* suggests that the preferred spelling is file(), and
> that open() shouldn't be used for new code.

Oops, yes.  I didn't write that, and it doesn't convey my feelings
about file() vs. open().  Here's a suggestion for better words:

"The file class is new in Python 2.2.  It represents the type (class)
of objects returned by the built-in open() function.  Its constructor
is an alias for open(), but for future and backwards compatibility,
open() remains preferred."

--Guido van Rossum (home page: http://www.python.org/~guido/)
From martin at v.loewis.de  Wed Jul  7 06:31:06 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Jul  7 06:31:10 2004
Subject: [Python-Dev] Tex files
In-Reply-To: <A128D751272CD411BC9200508BC2194D03383B2B@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D03383B2B@escpl.tcp.com.ar>
Message-ID: <40EB7C8A.1060906@v.loewis.de>

Batista, Facundo wrote:
> Does not work. But this is an advance: I should have LaTeX installed 
> (right now only tetex).

Please do provide more insight in what precisely you were doing, and
how precisely it failed.

Regards,
Martin

From pje at telecommunity.com  Wed Jul  7 06:48:21 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Jul  7 06:45:18 2004
Subject: [Python-Dev] Tex files
In-Reply-To: <200407061724.12150.fdrake@acm.org>
References: <A128D751272CD411BC9200508BC2194D03383B22@escpl.tcp.com.ar>
	<A128D751272CD411BC9200508BC2194D03383B22@escpl.tcp.com.ar>
Message-ID: <5.1.1.6.0.20040707004255.02218350@mail.telecommunity.com>

At 05:24 PM 7/6/04 -0400, Fred L. Drake, Jr. wrote:
>On Tuesday 06 July 2004 04:40 pm, Batista, Facundo wrote:
>  > The documentation, tutorial and other files is in .tex format in CVS.
>  >
>  > How can I convert them to a HTML or PDF format?
>  >
>  > I tried with pdftex (from tetex package) but it gives me structure errors.
>
>You didn't say what platform you're using, so this might not help you.
>
>On a Unix system with LaTeX installed, you should be able to "cd" to the Doc
>directory and run "make html" or "make pdf".  That *might* even work on
>Cygwin.
>
>For any other platforms, I've no idea.  All the real logic of how to do the
>formatting is in Doc/tools/mkhowto.  If that doesn't work for you, please let
>me know what platform and LaTeX distribution you're using, and what went
>wrong.

FWIW, I know that 'mkhowto' works in Cygwin using Cygwin Python.  I'd also 
personally suggest installing Cygwin as the fastest way to get a working 
LaTeX installation on Windows.  Just go to www.cygwin.com, click on Install 
Now, and check the boxes for the LaTeX packages you need.  You will 
probably at least want the 'tetex' and 'texmf' stuff.

From mike at skew.org  Wed Jul  7 06:51:54 2004
From: mike at skew.org (Mike Brown)
Date: Wed Jul  7 06:51:54 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <182C820A-CFC1-11D8-81DC-000D932927FE@opnet.com>
	"from Nick Bastin at Jul 6, 2004 10:55:24 pm"
Message-ID: <200407070451.i674psIS091414@chilled.skew.org>

Nick Bastin wrote:
> I could see a future where open() would support any valid URI

-1.

This already exists, mostly, in urllib.urlopen(), which happens to be very 
lenient in what it will accept as a "URL" to open. It attempts to guess at the 
intended behavior, regardless of whether the argument was a URI (possibly 
relative) or an OS path (possibly relative) or some hybrid of the two. Often 
it is impossible to tell based on syntax alone what type of 
thing-to-be-dreferenced the argument is, so the function errs on the side of 
what's most likely -- not that it goes to great lengths.

Overall it does an OK job, but it has bugs, conformance issues, Unicode 
issues, its behavior is affected by what platform it is running on, and heaven 
help you if you have dot segments, a UNC path, a colon in the wrong place or 
didn't use "|" instead of ":" in your Windows 'file' URI.

Even if these peripheral issues/bugs are addressed, it's still just too messy 
to take an arbitrary string and guess as to whether it is a URI or an OS path 
(and for which OS?) and to handle it as the user intended. The best you can 
do, in the absence of making the user assert exactly what the string is, is 
subject it to syntax checks that rule OUT the possibility of it being one or 
the other. Then, if it's still ambiguous, what to do? Fall back on some 
well-documented behavior such as trying it one way, then the other? Still 
rather messy, IMHO, and makes it difficult to use such a function in a context 
which requires that the argument be handled ONLY as one or the other. For 
example, in a URI resolver, you don't want href="/etc/passwd" in a document at 
http://myhost/doc.html to be interpreted as it it were "file:///etc/passwd" 
just because the attempt to open "http://myhost/etc/passwd" happened to fail.

So I'd rather keep URIs and IRIs isolated from OS paths as much as possible. 
Make the user understand the differences between them and discourage treatment 
of them as interchangable strings. Provide functions for converting between an 
OS path and a proper absolute URI, with no underlying platform influence 
(urllib.pathname2url does not meet these criteria). See examples in 4Suite's 
Ft.Lib.Uri in current CVS [1] for how I think it should be done. A more formal
proposal is forthcoming.

-Mike


[1] http://cvs.4suite.org/cgi-bin/viewcvs.cgi/4Suite/Ft/Lib/Uri.py
    (I'm still not happy with some of the Unicode related API decisions I
    made in this module, but encoding issues aside, I believe the general
    methodology is sound.)
From nbastin at opnet.com  Wed Jul  7 06:55:53 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Wed Jul  7 06:56:10 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <200407070451.i674psIS091414@chilled.skew.org>
References: <200407070451.i674psIS091414@chilled.skew.org>
Message-ID: <EC9C14BF-CFD1-11D8-81DC-000D932927FE@opnet.com>


On Jul 7, 2004, at 12:51 AM, Mike Brown wrote:

> Nick Bastin wrote:
>> I could see a future where open() would support any valid URI
>
> -1.
>
> This already exists, mostly, in urllib.urlopen(), which happens to be 
> very
> lenient in what it will accept as a "URL" to open. It attempts to 
> guess at the
> intended behavior, regardless of whether the argument was a URI 
> (possibly
> relative) or an OS path (possibly relative) or some hybrid of the two. 
> Often
> it is impossible to tell based on syntax alone what type of
> thing-to-be-dreferenced the argument is, so the function errs on the 
> side of
> what's most likely -- not that it goes to great lengths.

Keep in mind that I'm not actually suggesting that it *do* that, but 
merely as an example that open() could have a different behaviour from 
file().

--
Nick

From jeremy at alum.mit.edu  Wed Jul  7 13:39:42 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed Jul  7 13:39:48 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <200407070308.i6738U126024@guido.python.org>
References: <200407062309.i66N9xv25599@guido.python.org>
	<40EB4719.4040401@ezabel.com>
	<200407070308.i6738U126024@guido.python.org>
Message-ID: <1089200382.23548.14.camel@slothrop.corp.google.com>

On Tue, 2004-07-06 at 23:08, Guido van Rossum wrote:
> > I have always used open() myself. does this mean we should continue
> > to use open()?
> 
> IMO, yes.  My posting was intended to draw attention to the fact that
> apparently some developers think differently.  I'm hoping to revert
> their opinions unless there's a really good reason to switch to
> file().

+1 from me.  We've been using "open()" to mean open for about as long as
there have been Python programs, right?  I don't see any value to
changing the name at this point, and I see harm in having inconsistent
use of open vs. file in the standard library.

Jeremy


From jlenton at gmail.com  Wed Jul  7 13:45:43 2004
From: jlenton at gmail.com (John Lenton)
Date: Wed Jul  7 13:46:09 2004
Subject: [Python-Dev] tabs in idlelib/PyShell.py
In-Reply-To: <87u0wko734.fsf@hydra.localdomain>
References: <D2B16574-CF94-11D8-A5C4-0003934AD54A@chello.se>
	<87u0wko734.fsf@hydra.localdomain>
Message-ID: <2d9e37f9040707044577619b91@mail.gmail.com>

On Tue, 06 Jul 2004 22:59:43 -0400, Kurt B. Kaiser <kbk@shore.net> wrote:
> 
> My tools (emacs/python-mode and IDLE) are set to insert spaces, and
> the following method was entered at the same time.  It's fine.  I
> have no idea how that happened.
> 
> I'm thinking that since spaces are standard, it would be neat if
> IDLE and python-mode could display a right arrow glyph for each
> tab.

(defface we-hate-tabs-face
  '((t (:background "red")))
    "Face for tab characters.")

(and standard-display-table (aset standard-display-table ?\C-i  ; Show
tabs highlighted
      (vector (+ ?\C-i (* (face-id 'we-hate-tabs-face) 524288) ))))

thanks to utis on #emacs, and to
http://groups.google.com/groups?q=face+display+table+gnu.emacs.help&hl=en&lr=&ie=UTF-8&scoring=d&selm=slrnb9kt08.2su.tuukkat%40s-inf-pc24.oulu.fi&rnum=9

-- 
John Lenton (jlenton@gmail.com) -- Random fortune:
bash: fortune: command not found
From python at rcn.com  Wed Jul  7 04:25:55 2004
From: python at rcn.com (Raymond Hettinger)
Date: Wed Jul  7 16:27:16 2004
Subject: [Python-Dev] decimal API
In-Reply-To: <1f7befae04070617163c90b8e2@mail.gmail.com>
Message-ID: <002f01c463c9$bbca7340$d60aa044@oemcomputer>

[Tim]
>  NaNs
> and infinities are highly contagious after they appear, and
> NaN-poisoning in 754 is even worse than const-poisoning in C <wink>.

Okay, let's trap the little buggers.



Raymond

From paul at prescod.net  Wed Jul  7 17:14:23 2004
From: paul at prescod.net (Paul Prescod)
Date: Wed Jul  7 17:14:42 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <200407062309.i66N9xv25599@guido.python.org>
References: <200407062309.i66N9xv25599@guido.python.org>
Message-ID: <40EC134F.9050608@prescod.net>

Guido van Rossum wrote:

> I recently saw a checkin that changed a call to open() into a call to
> file(), suggesting that using file() is more "politically correct"
> than open().
> 
> I'm not sure I agree with this.  While open() and file() are currently
> aliases for the same object, this may not always be the case (it
> certainly wasn't always the case :-). 

I presumed file() was preferred to open() for the simple reason that it 
says more explicitly what you are doing. You are constructing a file 
object. "open()" doesn't say whether you are opening a file or socket or 
window or folder or ...

open made sense in a Unix world where "everything was a file" (except 
that even then it wasn't really) but it is a poor name for the file 
opening function in a 21st century language.

  Paul Prescod


From s.percivall at chello.se  Wed Jul  7 17:24:44 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Wed Jul  7 17:24:47 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <40EC134F.9050608@prescod.net>
References: <200407062309.i66N9xv25599@guido.python.org>
	<40EC134F.9050608@prescod.net>
Message-ID: <C6088B38-D029-11D8-9D64-0003934AD54A@chello.se>


On 2004-07-07, at 17.14, Paul Prescod wrote:
> open made sense in a Unix world where "everything was a file" (except 
> that even then it wasn't really) but it is a poor name for the file 
> opening function in a 21st century language.

Except that it still makes sense. Look in almost any application: In 
the file menu you'll find a command 'Open ...', or perhaps 'Open File 
...'. It'll open a dialog for opening a file from the filesystem (or 
something represented as a filesystem).

From guido at python.org  Wed Jul  7 17:44:24 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Jul  7 17:44:30 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Doc/whatsnew
	whatsnew24.tex, 1.62, 1.63
In-Reply-To: Your message of "Wed, 07 Jul 2004 06:01:58 PDT."
	<E1BiC3i-0006ll-HO@sc8-pr-cvs1.sourceforge.net> 
References: <E1BiC3i-0006ll-HO@sc8-pr-cvs1.sourceforge.net> 
Message-ID: <200407071544.i67FiOY27354@guido.python.org>

> + \item A \function{basicConfig} function was added to the
> + \module{logging} package to simplify log configuration.  It defaults
> + to logging to standard error, but a
> + number of optional keyword arguments can be specified to 
> + log to a particular file, change the logging format, or set  the
> + logging level.  For example:

Actually, basicConfig() always existed; the news is that it now takes
arguments.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Wed Jul  7 17:51:48 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Jul  7 17:51:57 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: Your message of "Wed, 07 Jul 2004 08:14:23 PDT."
	<40EC134F.9050608@prescod.net> 
References: <200407062309.i66N9xv25599@guido.python.org>  
	<40EC134F.9050608@prescod.net> 
Message-ID: <200407071551.i67FpmM27430@guido.python.org>

> I presumed file() was preferred to open() for the simple reason that
> it says more explicitly what you are doing. You are constructing a
> file object. "open()" doesn't say whether you are opening a file or
> socket or window or folder or ...
> 
> open made sense in a Unix world where "everything was a file"
> (except that even then it wasn't really) but it is a poor name for
> the file opening function in a 21st century language.

Maybe you'll change your mind when open() can return other objects
besides files.  In the mean time, let's use the compatibility
argument.  Changing things just because a new century has started
sounds like a fad to me -- IOW the argument "it's the modern thing to
do" usually doesn't convince me.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From theller at python.net  Wed Jul  7 17:55:03 2004
From: theller at python.net (Thomas Heller)
Date: Wed Jul  7 17:56:06 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <40E7C737.6000702@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Sun,
	04 Jul 2004 11:00:39 +0200")
References: <0c3d01c46164$f49e85b0$0200a8c0@eden>
	<40E7C737.6000702@v.loewis.de>
Message-ID: <eknn95ig.fsf@python.net>

"Martin v. L?wis" <martin@v.loewis.de> writes:

> While trying to understand MSI, I had two primary information sources:
> the MSDN documentation, and ORCA.EXE. The latter tool comes with the
> SDK, and needs to be installed separately: The SDK installs orca.msi,
> which then installs orca. With orca, you can learn a lot about how
> other people build their MSIs, and find out how certain effects
> which you know are possible can be achieved.
>
> WIX was released after I had done my thing. I haven't looked at it,
> but I would hope that it also exposes the full power of MSI. So if
> people are concerned about the hackishness of my current code, I
> could try to come up with a WIX installer that generates roughly the
> same MSI package.

I'm not saying that your code is a hack.  I just think that
documentation is missing.

For example, the MSI which is currently built removes the file
associations when the package is removed, instead of restoring them to
their previous values.  I have no idea how this can be fixed...

Thomas

From anthony at interlink.com.au  Wed Jul  7 18:07:25 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Jul  7 18:07:43 2004
Subject: [Python-Dev] 2.4a1
Message-ID: <40EC1FBD.4050305@interlink.com.au>

I plan to start attempting to get a 2.4a1 out the door
in the morning my time. Depending on how that goes, it
will either be then, or on the Friday - I'll post updates
when I see how it's going.

Given we have a couple of bits that haven't landed yet
(such as decorators) I suspect we're probably looking at
about 3 alpha releases - I'm thinking 2-3 weeks between
them. I'd love to get feedback on this - I'll write up
the release plan once I hear back from people.

I'm not too fussed about the code-freeze aspect for the
first alpha - although obviously if you're planning to
check in some horrible massive change, I'd appreciate
the heads up.

Once I'm out of this current week, I should have a lot more
time to dedicate to the testing of the first alpha on a
bunch of different platforms - I don't think there's a huge
amount of point to trying to nail things down before the
first alpha.

I'll be hanging out on #python-dev on freenode if you
want to pop by and ask questions or comments.

Anthony
From skip at pobox.com  Wed Jul  7 18:09:02 2004
From: skip at pobox.com (Skip Montanaro)
Date: Wed Jul  7 18:09:06 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <200407062309.i66N9xv25599@guido.python.org>
References: <200407062309.i66N9xv25599@guido.python.org>
Message-ID: <16620.8222.245158.512828@montanaro.dyndns.org>


    Guido> I recently saw a checkin that changed a call to open() into a
    Guido> call to file(), suggesting that using file() is more "politically
    Guido> correct" than open().

    Guido> I'm not sure I agree with this.  While open() and file() are
    Guido> currently aliases for the same object, this may not always be the
    Guido> case (it certainly wasn't always the case :-).  In the future, I
    Guido> could see open() become a factory function again that could
    Guido> return an instance of a different class depending on the mode
    Guido> argument, the default encoding for files, or who knows what; but
    Guido> file will always remain a class.

That was probably a checkin I made.  I would have left it alone except the
code was

    file = open(...)

As long as I was changing the variable name to not mask the builtin I
changed the call as well.  Had it been

    f = open(...)

I probably would have kept my hands off.

In any case, I was under the impression that file() was the wave of the
future and open() a nod to the past.

Skip
From skip at pobox.com  Wed Jul  7 18:23:53 2004
From: skip at pobox.com (Skip Montanaro)
Date: Wed Jul  7 18:23:58 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <200407071551.i67FpmM27430@guido.python.org>
References: <200407062309.i66N9xv25599@guido.python.org>
	<40EC134F.9050608@prescod.net>
	<200407071551.i67FpmM27430@guido.python.org>
Message-ID: <16620.9113.512598.667717@montanaro.dyndns.org>


    Guido> Changing things just because a new century has started sounds
    Guido> like a fad to me -- IOW the argument "it's the modern thing to
    Guido> do" usually doesn't convince me.

Yes, but has others have pointed out, the current documentation seems to
suggest that file() is preferred to open().  That appears not to be your
intent and should be changed, but I don't think most of us who have begun
using file() (I've only changed from open() to file() relatively recently)
didn't view it as a fad.

Skip

From guido at python.org  Wed Jul  7 18:51:59 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Jul  7 18:52:08 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: Your message of "Wed, 07 Jul 2004 11:23:53 CDT."
	<16620.9113.512598.667717@montanaro.dyndns.org> 
References: <200407062309.i66N9xv25599@guido.python.org>
	<40EC134F.9050608@prescod.net>
	<200407071551.i67FpmM27430@guido.python.org> 
	<16620.9113.512598.667717@montanaro.dyndns.org> 
Message-ID: <200407071652.i67Gpxq27669@guido.python.org>

> Yes, but has others have pointed out, the current documentation
> seems to suggest that file() is preferred to open().  That appears
> not to be your intent and should be changed, but I don't think most
> of us who have begun using file() (I've only changed from open() to
> file() relatively recently) didn't view it as a fad.

I apologize for the doc glitch -- I should've nailed that one the
first time I noticed it. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Wed Jul  7 18:53:01 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Jul  7 18:53:07 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: Your message of "Wed, 07 Jul 2004 11:09:02 CDT."
	<16620.8222.245158.512828@montanaro.dyndns.org> 
References: <200407062309.i66N9xv25599@guido.python.org>  
	<16620.8222.245158.512828@montanaro.dyndns.org> 
Message-ID: <200407071653.i67Gr2R27741@guido.python.org>

> That was probably a checkin I made.  I would have left it alone except the
> code was
> 
>     file = open(...)
> 
> As long as I was changing the variable name to not mask the builtin I
> changed the call as well.  Had it been
> 
>     f = open(...)
> 
> I probably would have kept my hands off.

Hm...  I'm not particularly concerned over fixing all code that uses
file as a local variable name, unless it actually is likely to need to
reference the file class by name; builtins are in the last scope
searched for the very reason that no programmer is expected to keep up
with all additions to the built-in library, so locals hiding built-ins
is okay.  (Not that it isn't a good idea to avoid obvious clashes --
'str' for string variables and 'type' for type variables being the
most obvious stumbling blocks.)

> In any case, I was under the impression that file() was the wave of the
> future and open() a nod to the past.

Now you know better...

--Guido van Rossum (home page: http://www.python.org/~guido/)
From fumanchu at amor.org  Wed Jul  7 19:49:49 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Wed Jul  7 19:54:06 2004
Subject: [Python-Dev] file() or open()?
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E5622E2@opus.amorhq.net>

I think Skip wrote:
> Yes, but has others have pointed out, the current documentation
> seems to suggest that file() is preferred to open().

While you're at it, the docstring for open could use the same treatment.
In 2.3.2, it starts with: "file(name[, mode[, buffering]])" and ends
with "open() is an alias for file()." The first implies that file() is
preferred, in my mind.


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
From pinard at iro.umontreal.ca  Wed Jul  7 20:30:33 2004
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Wed Jul  7 20:30:41 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <200407071551.i67FpmM27430@guido.python.org>
References: <200407062309.i66N9xv25599@guido.python.org>
	<40EC134F.9050608@prescod.net>
	<200407071551.i67FpmM27430@guido.python.org>
Message-ID: <20040707183033.GA30577@alcyon.progiciels-bpi.ca>

[Guido van Rossum]

> Oops, yes.  I didn't write that, and it doesn't convey my feelings
> about file() vs. open().  Here's a suggestion for better words:
>
> "The file class is new in Python 2.2.  It represents the type (class)
> of objects returned by the built-in open() function.  Its constructor
> is an alias for open(), but for future and backwards compatibility,
> open() remains preferred."

This is slightly saddening.  I perceived the introduction of `file()' as
a nice cleanup in Python.  I wonder if the quote above states the real
reason.  Sometimes, we produce rationalisations for what was more of an
emotion initially.  Here, another message on the same topic says:

> Maybe you'll change your mind when open() can return other objects
> besides files.

Granted that `open()' may be preferred for backwards compatibility, but
if it is meant to change or improve in the long run, maybe it should be
avoided, precisely for future compatibility.

Currently, I use `file()' when the meaning of my code is to create a
file object to be operated upon, but to otherwise let Python manage it.
Yet, I still use `open()' when the meaning of my code is opening a file,
usually implying that I will also explicitly close it.  If `open()'
could return objects of various types, and my goal is really to get a
file object, may I presume that `file()' imposes itself as preferred?

-- 
Fran?ois Pinard   http://www.iro.umontreal.ca/~pinard
From barry at python.org  Wed Jul  7 20:39:56 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Jul  7 20:39:51 2004
Subject: [Python-Dev] 2.4a1
In-Reply-To: <40EC1FBD.4050305@interlink.com.au>
References: <40EC1FBD.4050305@interlink.com.au>
Message-ID: <1089225596.8354.517.camel@localhost>

On Wed, 2004-07-07 at 12:07, Anthony Baxter wrote:
> I plan to start attempting to get a 2.4a1 out the door
> in the morning my time. Depending on how that goes, it
> will either be then, or on the Friday - I'll post updates
> when I see how it's going.

I haven't had (and won't have) time to land the PEP 292 stuff by then. 
I will follow up on that thread and shoot for inclusion in CVS for the
next alpha.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040707/ab68e523/attachment.pgp
From hyeshik at gmail.com  Wed Jul  7 20:51:53 2004
From: hyeshik at gmail.com (Hye-Shik Chang)
Date: Wed Jul  7 20:52:21 2004
Subject: [Python-Dev] 2.4a1
In-Reply-To: <40EC1FBD.4050305@interlink.com.au>
References: <40EC1FBD.4050305@interlink.com.au>
Message-ID: <4f0b69dc040707115166dafaa5@mail.gmail.com>

On Thu, 08 Jul 2004 02:07:25 +1000, Anthony Baxter
<anthony@interlink.com.au> wrote:
> I'm not too fussed about the code-freeze aspect for the
> first alpha - although obviously if you're planning to
> check in some horrible massive change, I'd appreciate
> the heads up.

I just finished reorganizing Modules/cjkcodecs bits for compacter
installed/source layout.  It also contains several functional updates
as I said before[1].  But it's real massive change (3M+ in context
diff).  May I commit it before 2.4a1 or leave it for after 2.4a1?

[1] JIS X 0213:2004 support, BIG5HKSCS and ISO-2022-CN.

Hye-Shik
From mal at egenix.com  Wed Jul  7 21:20:29 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Jul  7 21:20:33 2004
Subject: [Python-Dev] 2.4a1
In-Reply-To: <40EC1FBD.4050305@interlink.com.au>
References: <40EC1FBD.4050305@interlink.com.au>
Message-ID: <40EC4CFD.7000601@egenix.com>

Anthony Baxter wrote:
> I plan to start attempting to get a 2.4a1 out the door
> in the morning my time. Depending on how that goes, it
> will either be then, or on the Friday - I'll post updates
> when I see how it's going.
> 
> Given we have a couple of bits that haven't landed yet
> (such as decorators) I suspect we're probably looking at
> about 3 alpha releases - I'm thinking 2-3 weeks between
> them. I'd love to get feedback on this - I'll write up
> the release plan once I hear back from people.
> 
> I'm not too fussed about the code-freeze aspect for the
> first alpha - although obviously if you're planning to
> check in some horrible massive change, I'd appreciate
> the heads up.

I'm almost done with the .encode()/.decode() patches
and could check in the patch tomorrow (Thursday), unless
you'd rather wait until after the release.

FYI, the only things still missing are some doc patches.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 07 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mark at ironport.com  Wed Jul  7 21:31:31 2004
From: mark at ironport.com (Mark Peek)
Date: Wed Jul  7 21:42:07 2004
Subject: [Python-Dev] Windows installer for 2.4?
Message-ID: <p05210620bd11f96e532d@[10.1.10.133]>

With the move to using VC 7.1 for Python builds, I believe I read 
there was a move to use the Microsoft Installer instead of the Wise 
installer. Looking in the PCbuild directory, I did not see any new 
installer files. Does anyone have a .vdproj file for packaging the 
current build?

Thanks,
Mark
From tjreedy at udel.edu  Wed Jul  7 22:27:47 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed Jul  7 22:27:54 2004
Subject: [Python-Dev] Re: file() or open()?
References: <200407062309.i66N9xv25599@guido.python.org><40EC134F.9050608@prescod.net><200407071551.i67FpmM27430@guido.python.org>
	<20040707183033.GA30577@alcyon.progiciels-bpi.ca>
Message-ID: <cchmc7$vft$1@sea.gmane.org>


"François Pinard" <pinard@iro.umontreal.ca> wrote in message
news:20040707183033.GA30577@alcyon.progiciels-bpi.ca...
> I perceived the introduction of `file()' as a nice cleanup in Python.

As a user, so did I.  I like the cosistency of using file along with int,
tuple, list, dict, type, (and did I leave out something), and all user
classes as constructors of instances of themselves.

Terry J. Reedy



From bob at redivi.com  Wed Jul  7 22:32:14 2004
From: bob at redivi.com (Bob Ippolito)
Date: Wed Jul  7 22:32:21 2004
Subject: [Python-Dev] Re: file() or open()?
In-Reply-To: <cchmc7$vft$1@sea.gmane.org>
References: <200407062309.i66N9xv25599@guido.python.org><40EC134F.9050608@prescod.net><200407071551.i67FpmM27430@guido.python.org>
	<20040707183033.GA30577@alcyon.progiciels-bpi.ca>
	<cchmc7$vft$1@sea.gmane.org>
Message-ID: <BBB2B2D6-D054-11D8-8648-000A95686CD8@redivi.com>


On Jul 7, 2004, at 4:27 PM, Terry Reedy wrote:

>
> "Fran?ois Pinard" <pinard@iro.umontreal.ca> wrote in message
> news:20040707183033.GA30577@alcyon.progiciels-bpi.ca...
>> I perceived the introduction of `file()' as a nice cleanup in Python.
>
> As a user, so did I.  I like the cosistency of using file along with 
> int,
> tuple, list, dict, type, (and did I leave out something), and all user
> classes as constructors of instances of themselves.

Me too.

-bob

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040707/aec68135/smime-0001.bin
From martin at v.loewis.de  Wed Jul  7 22:46:17 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Jul  7 22:46:20 2004
Subject: [Python-Dev] Windows installer for 2.4?
In-Reply-To: <p05210620bd11f96e532d@[10.1.10.133]>
References: <p05210620bd11f96e532d@[10.1.10.133]>
Message-ID: <40EC6119.7090909@v.loewis.de>

Mark Peek wrote:
> With the move to using VC 7.1 for Python builds, I believe I read there 
> was a move to use the Microsoft Installer instead of the Wise installer. 
> Looking in the PCbuild directory, I did not see any new installer files. 
> Does anyone have a .vdproj file for packaging the current build?

No. While we will distribute a installer package, we definitely won't
be using VC.NET to build that package. See nondist/sandbox/msi for a
like technology for building the MSI package.

Regards,
Martin

From orbitz at drorbitz.ath.cx  Wed Jul  7 22:46:14 2004
From: orbitz at drorbitz.ath.cx (orbitz)
Date: Wed Jul  7 22:46:44 2004
Subject: [Python-Dev] Re: file() or open()?
In-Reply-To: <cchmc7$vft$1@sea.gmane.org>
References: <200407062309.i66N9xv25599@guido.python.org><40EC134F.9050608@prescod.net><200407071551.i67FpmM27430@guido.python.org>	<20040707183033.GA30577@alcyon.progiciels-bpi.ca>
	<cchmc7$vft$1@sea.gmane.org>
Message-ID: <40EC6116.1020505@drorbitz.ath.cx>

I considered more as the action being performed.  I'm opening something, 
in this case a file.  And now I have an object which has been opened, I 
can perform operations on it, and when I'm done I close it.

Terry Reedy wrote:

>"Fran?ois Pinard" <pinard@iro.umontreal.ca> wrote in message
>news:20040707183033.GA30577@alcyon.progiciels-bpi.ca...
>  
>
>>I perceived the introduction of `file()' as a nice cleanup in Python.
>>    
>>
>
>As a user, so did I.  I like the cosistency of using file along with int,
>tuple, list, dict, type, (and did I leave out something), and all user
>classes as constructors of instances of themselves.
>
>Terry J. Reedy
>
>
>
>_______________________________________________
>Python-Dev mailing list
>Python-Dev@python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: http://mail.python.org/mailman/options/python-dev/orbitz%40drorbitz.ath.cx
>
>  
>

From jlenton at gmail.com  Wed Jul  7 22:47:17 2004
From: jlenton at gmail.com (John Lenton)
Date: Wed Jul  7 22:47:20 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <20040707183033.GA30577@alcyon.progiciels-bpi.ca>
References: <200407062309.i66N9xv25599@guido.python.org>
	<40EC134F.9050608@prescod.net>
	<200407071551.i67FpmM27430@guido.python.org>
	<20040707183033.GA30577@alcyon.progiciels-bpi.ca>
Message-ID: <2d9e37f9040707134736694189@mail.gmail.com>

On Wed, 7 Jul 2004 14:30:33 -0400, Fran?ois Pinard
<pinard@iro.umontreal.ca> wrote:
> Granted that `open()' may be preferred for backwards compatibility, but
> if it is meant to change or improve in the long run, maybe it should be
> avoided, precisely for future compatibility.
> 
> Currently, I use `file()' when the meaning of my code is to create a
> file object to be operated upon, but to otherwise let Python manage it.
> Yet, I still use `open()' when the meaning of my code is opening a file,
> usually implying that I will also explicitly close it.  If `open()'
> could return objects of various types, and my goal is really to get a
> file object, may I presume that `file()' imposes itself as preferred?

I think the idea is that if you could do

    from urllib import urlopen as open

and your app is still sane, then you're doing it right. For example,
if you used open and then mmap, that might not work in the future
(mmap works only with files, not with things that look slightly like a
file but aren't).

-- 
John Lenton (jlenton@gmail.com) -- Random fortune:
bash: fortune: command not found
From bob at redivi.com  Wed Jul  7 22:51:09 2004
From: bob at redivi.com (Bob Ippolito)
Date: Wed Jul  7 22:51:15 2004
Subject: [Python-Dev] Re: file() or open()?
In-Reply-To: <40EC6116.1020505@drorbitz.ath.cx>
References: <200407062309.i66N9xv25599@guido.python.org><40EC134F.9050608@prescod.net><200407071551.i67FpmM27430@guido.python.org>	<20040707183033.GA30577@alcyon.progiciels-bpi.ca>
	<cchmc7$vft$1@sea.gmane.org> <40EC6116.1020505@drorbitz.ath.cx>
Message-ID: <5FE2BAC8-D057-11D8-8648-000A95686CD8@redivi.com>

n Jul 7, 2004, at 4:46 PM, orbitz wrote:

> Terry Reedy wrote:
>
>> "Fran?ois Pinard" <pinard@iro.umontreal.ca> wrote in message
>> news:20040707183033.GA30577@alcyon.progiciels-bpi.ca...
>>
>>> I perceived the introduction of `file()' as a nice cleanup in Python.
>>
>> As a user, so did I.  I like the cosistency of using file along with 
>> int,
>> tuple, list, dict, type, (and did I leave out something), and all user
>> classes as constructors of instances of themselves.
> I considered more as the action being performed.  I'm opening 
> something, in this case a file.  And now I have an object which has 
> been opened, I can perform operations on it, and when I'm done I close 
> it.

But you also open sockets, pipes, applications, bank accounts, etc.  
"open" seems seriously ambiguous to me, and it's not a "generic" 
function like len or iter.  The only good reason I see is to associate 
"open" with files is because that's just how it's always been done in 
Python and C.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040707/6c5022c0/smime.bin
From bac at OCF.Berkeley.EDU  Wed Jul  7 23:03:58 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Jul  7 23:04:06 2004
Subject: [Python-Dev] Undo indenting of macros in Include/pyport.h?
In-Reply-To: <1f7befae04070619557d898f2f@mail.gmail.com>
References: <40E8AE2C.3060606@ocf.berkeley.edu>
	<40E9ACFC.8030207@ocf.berkeley.edu>
	<1f7befae04070619557d898f2f@mail.gmail.com>
Message-ID: <40EC653E.4050609@ocf.berkeley.edu>

Tim Peters wrote:

> [Brett Cannon]
> 
>>Personally I would like to remove the tabs so that grepping is easier
>>and more obvious.  But the tabbing is there since the #ifdef'ing goes
>>three levels deep.
> 
> 
> Use Perl instead of grep.  print if /#\s*define/.  Or learn how to
> spell "whitespace" in whatever godforsaken flavor of grep you're
> using. 

I just need to come up with a simple Python script that gives you basic 
grep functionality using Python regex engine since I have that syntax 
memorized (which, I know, is just Perl regex syntax).

> Or upgrade to Windows and buy Source Insight <wink>.

=)  I have used enough Windows lately.  Ugh.  COM is just evil.

>  Or ...
> just don't damage the readability of the code to cater to bad tools.

Yeah, I am not going to touch the file.

-Brett
From bac at OCF.Berkeley.EDU  Wed Jul  7 23:11:51 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Jul  7 23:11:59 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <200407062309.i66N9xv25599@guido.python.org>
References: <200407062309.i66N9xv25599@guido.python.org>
Message-ID: <40EC6717.5060906@ocf.berkeley.edu>

Guido van Rossum wrote:

[SNIP]
> In the future, I could see
> open() become a factory function again that could return an instance
> of a different class depending on the mode argument, the default
> encoding for files, or who knows what; but file will always remain a
> class.

How is this case different from the whole unicode.encode thread and the 
worry of having different types of objects returned based on the 
argument?  I would assume that any objects returned would follow the 
file interface roughly.

-Brett
From goodger at python.org  Thu Jul  8 03:55:11 2004
From: goodger at python.org (David Goodger)
Date: Thu Jul  8 03:55:18 2004
Subject: [Python-Dev] decimal.py signals & traps
Message-ID: <40ECA97F.30807@python.org>

There's a problem in the decimal.py API regarding signals & traps.  I
noticed Raymond's checkin enabling certain traps (thanks Raymond!), so
I updated my copy and played around a bit:

 >>> from decimal import Decimal
 >>> Decimal('1')/0
Traceback (most recent call last):
...
decimal.DivisionByZero: x / 0

As expected.  But:

 >>> Decimal('0')/0
Decimal("NaN")

I expected this to raise decimal.DivisionUndefined.  Even worse:

 >>> Decimal('bogus')
Decimal("NaN")

That should have raised decimal.ConversionSyntax.  This behavior
obviously doesn't agree with:

[Tim Peters]
 > You can't get a NaN or infinity from finite operands without
 > signaling one of the 3 I recommend trap-enabling, so enabling those
 > traps stops their program the instant the first NaN or infinity
 > appears.

The problem is that the implementation associates traps with what it
calls "signals" but the spec & PEP 327 call "conditions".  I believe
that led to a misunderstanding:

[Tim Peters]
 > I'm still of the opinion that the invalid-operation, overflow, and
 > division-by-zero traps should be enabled by default, and the others
 > disabled

Tim is referring to traps associated with spec/PEP-defined "signals".
Raymond implemented the default-enabled traps like this:

     _default_traps.update(
         {DivisionByZero:1, Overflow:1, InvalidOperation:1})

Take a look at the "Signal" column of the table in
<http://www.python.org/peps/pep-0327.html#exceptional-conditions>.
The "invalid-operation" *signal* is associated with 5 different
*conditions*: conversion syntax, division impossible, division
undefined, invalid context, and invalid operation.  The spec/PEP lists
12 conditions and 8 signals, but

 >>> len(decimal.Signals)
13

decimal.Signals is actually a list of spec/PEP *conditions*, with
decimal.DecimalException (the base exception class) also included.

The easy solution is to change

     _default_traps.update(
         {DivisionByZero:1, Overflow:1, InvalidOperation:1})

to

     _default_traps.update(
         {DivisionByZero:1, Overflow:1, InvalidOperation:1,
          ConversionSyntax:1, DivisionImpossible:1, DivisionUndefined:1,
          InvalidContext})

But I don't think that's the best solution.  It still leaves the
mismatch between spec/PEP "signals" and implementation "signals".

Should traps be set on spec/PEP conditions, or spec/PEP signals?  If
the former, decimal.Signals should at least be renamed to
decimal.Conditions.  If the latter, decimal.py's implementation of
exceptions needs to be reworked.

-- 
David Goodger <http://python.net/~goodger>
From tim.peters at gmail.com  Thu Jul  8 04:40:43 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Jul  8 04:41:01 2004
Subject: [Python-Dev] decimal.py signals & traps
In-Reply-To: <40ECA97F.30807@python.org>
References: <40ECA97F.30807@python.org>
Message-ID: <1f7befae04070719401a0a81f3@mail.gmail.com>

[David Goodger]
...
> Should traps be set on spec/PEP conditions, or spec/PEP signals?

On signals, not on conditions.  The 8 signals in the spec are listed
near the end of

    http://www2.hursley.ibm.com/decimal/damodel.html

Context must contain one sticky-flag and one trap-enable flag for each
signal.  If we're providing more than just that, we're extending the
standard, which probably wasn't intended.

I'll add that the spec's use of "conditions" grouped (many-to-one)
*into* signals seems unnecessary and confusing.

OTOH, I believe the spec isn't complete yet either.  What's normal for
"a spec like this" is to go on to specify information that must be
provided to a trap handler when one is invoked.  In Python, that would
presumably be implemented by setting attribute values on the exception
object.  That would be a natural way for a trap handler to learn which
condition was the cause of the trapped signal.  Or we could define an
exception for each condition, and subclass from the 8 signal
exceptions appropriately.

For example, the 754 standard requires that when a double-precision
overflow is trapped, the trap handler must be given access to the
rounded (to 53 bits) value of  the true infinite-precision result but
divided by 2**1536.

I don't know what the revisions-in-progress versions of 754 and 854
will require in this respect, but if they require anything along these
lines, Cowlishaw will have to play along with it if he wants to
continue claiming IBM's spec is a superset.

> If the former, decimal.Signals should at least be renamed to
> decimal.Conditions.  If the latter, decimal.py's implementation of
> exceptions needs to be reworked.

Yup, it sounds like it does need some reworking.
From paul at prescod.net  Thu Jul  8 06:16:33 2004
From: paul at prescod.net (Paul Prescod)
Date: Thu Jul  8 06:16:39 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <200407071551.i67FpmM27430@guido.python.org>
References: <200407062309.i66N9xv25599@guido.python.org>
	<40EC134F.9050608@prescod.net>
	<200407071551.i67FpmM27430@guido.python.org>
Message-ID: <40ECCAA1.8080009@prescod.net>

Guido van Rossum wrote:

>...
> 
> Maybe you'll change your mind when open() can return other objects
> besides files.  In the mean time, let's use the compatibility
> argument.  Changing things just because a new century has started
> sounds like a fad to me -- IOW the argument "it's the modern thing to
> do" usually doesn't convince me.

I don't understand why they are both builtins. Why provide two bindings 
for the same thing? I always presumed (as others did) that there was a 
new way and an old way maintained for backwards compatibility.

  Paul Prescod


From tim.peters at gmail.com  Thu Jul  8 06:41:18 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Jul  8 06:41:24 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <40ECCAA1.8080009@prescod.net>
References: <200407062309.i66N9xv25599@guido.python.org>
	<40EC134F.9050608@prescod.net>
	<200407071551.i67FpmM27430@guido.python.org>
	<40ECCAA1.8080009@prescod.net>
Message-ID: <1f7befae040707214161825435@mail.gmail.com>

If you do a cvs annotate on Misc/NEWS, you'll find that Guido gets the
blame, in rev 1.255, for this blurb:

"""
- The builtin file type can be subclassed now.  In the usual pattern,
 "file" is the name of the builtin type, and file() is a new builtin
 constructor, with the same signature as the builtin open() function.
 file() is now the preferred way to open a file.
"""

However, just as I finished pasting that in, he used the time machine
to shift the blame on me, in rev 1.242.  My memory was also altered,
so that I seem to really believe that was my understanding at the
time.  But I usually continued to use open() anyway, proving something
or other.  I'm still unclear about what (if anything) file() should be
used for.
From guido at python.org  Thu Jul  8 06:43:12 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul  8 06:43:20 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: Your message of "Wed, 07 Jul 2004 14:11:51 PDT."
	<40EC6717.5060906@ocf.berkeley.edu> 
References: <200407062309.i66N9xv25599@guido.python.org>  
	<40EC6717.5060906@ocf.berkeley.edu> 
Message-ID: <200407080443.i684hDK28487@guido.python.org>

> [SNIP]
> > In the future, I could see
> > open() become a factory function again that could return an instance
> > of a different class depending on the mode argument, the default
> > encoding for files, or who knows what; but file will always remain a
> > class.
> 
> How is this case different from the whole unicode.encode thread and the 
> worry of having different types of objects returned based on the 
> argument?  I would assume that any objects returned would follow the 
> file interface roughly.

Of course, but they don't have to be instances of file to comply with
that interface, and that's the whole point.  While it's technically
possible, using a loophole in __new__, to let file() return an x for
which isinstance(x, file) is false, that's generally ugly (even though
there are cases where it's useful).  But having a factory function
that returns something that complies with a given interface is a
standard pattern.  Thus, the main use I see for having 'file' around
is so that you can say isinstance(x, file) in certain cases where you
are interested in distinguishing between *real* files and objects that
merely implement the file interface; while open() is the factory
function.

If you look at nondist/sandbox/sio/, you'll find an incomplete
experiment with a new standard I/O library implementation, where
features like buffering, encoding, line ending translation, and
perhaps others (e.g. read/write/update/append), are implemented by
stacking various objects.  I think Java has a similar design
somewhere, even though I've heard that they are replacing that with
something called native I/O (or new I/O?) which is apparently faster.

Anyway, here's my future-proof distinction: use open() as the factory
function, and file for type-testing.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From martin at v.loewis.de  Thu Jul  8 07:15:33 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul  8 07:15:54 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <eknn95ig.fsf@python.net>
References: <0c3d01c46164$f49e85b0$0200a8c0@eden>	<40E7C737.6000702@v.loewis.de>
	<eknn95ig.fsf@python.net>
Message-ID: <40ECD875.7010004@v.loewis.de>

Thomas Heller wrote:
> For example, the MSI which is currently built removes the file
> associations when the package is removed, instead of restoring them to
> their previous values.  I have no idea how this can be fixed...

This is a multi-step process. First, one has to find out how
to restore associations in general, i.e. independently of MSI.
Where should one store, on install, the information what the old
association was? What if there already is "old" extension
information?

This is general Win32 stuff, and I don't know what the answer is.

Then, how can this be integrated into an MSI file? Perhaps
MSI already offers built-in mechanism for that, perhaps existing
mechanisms for dealing with the registry have to be used.

This is MSI specific, and is likely answered in the Microsoft
MSI documentation.

Then, finally, how can the Python script be changed to invoke
the MSI mechanism? This is very easy: If you know what tables
in MSI to manipulate in what way, the relevant Python code
should become easily apparent.

So in this case, I don't think any amount of documentation on
the Python script will help to solve this problem (that is
not to say that documentation should be avoided for this reason,
of course).

Regards,
Martin


From paul at prescod.net  Thu Jul  8 07:27:04 2004
From: paul at prescod.net (Paul Prescod)
Date: Thu Jul  8 07:27:09 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
Message-ID: <40ECDB28.10402@prescod.net>

So I gave a tutorial last night on Python's implementation. I had a 
pretty clear idea of the parser, compiler, AST branch etc. But the 
bytecodes were fuzzy. For instance:

 >>> def foo(a):
...     b = a + 5
...     return b
...
 >>> dis.dis(foo)
   2           0 LOAD_FAST                0 (a)
               3 LOAD_CONST               1 (5)
               6 BINARY_ADD
               7 STORE_FAST               1 (b)

   3          10 LOAD_FAST                1 (b)
              13 RETURN_VALUE
              14 LOAD_CONST               0 (None)
              17 RETURN_VALUE

Why does the RETURN_VALUE opcode have to return something from the 
stack? Why not have a RETURN_VAR opcode that would directly return a 
variable or constant? (a leading bit could indicate whether to look in 
the const or var tuple).

Similarly, what if BINARY_ADD could work directly on constants and 
variables? I see the virtue of using the stack for objects that do not 
otherwise have a name. But if a value is in a contant or variable, why 
not refer to it by its position in co_consts or co_varnames.

And as long as we are talking about referring to things more directly, 
wouldn't it be possible to refer to constants by pointer rather than 
indirecting through the index? You'd have to fix up pointers when you 
first loaded the code but only once per function.

  Paul Prescod

From anthony at interlink.com.au  Thu Jul  8 08:46:43 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Jul  8 08:46:53 2004
Subject: [Python-Dev] 2.4a1 - Trunk frozen for release
In-Reply-To: <40EC1FBD.4050305@interlink.com.au>
References: <40EC1FBD.4050305@interlink.com.au>
Message-ID: <40ECEDD3.1080906@interlink.com.au>

Please hold off on committing anything to the trunk unless
you're involved in the release for the next 24 hours or so.


I'll send another email when it's safe to commit again.

Thanks,
Anthony
From mwh at python.net  Thu Jul  8 11:57:40 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Jul  8 11:57:41 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <40ECDB28.10402@prescod.net> (Paul Prescod's message of "Wed,
	07 Jul 2004 22:27:04 -0700")
References: <40ECDB28.10402@prescod.net>
Message-ID: <2m4qoi254b.fsf@starship.python.net>

Paul Prescod <paul@prescod.net> writes:

> So I gave a tutorial last night on Python's implementation. I had a
> pretty clear idea of the parser, compiler, AST branch etc. But the
> bytecodes were fuzzy. For instance:
>
>  >>> def foo(a):
> ...     b = a + 5
> ...     return b
> ...
>  >>> dis.dis(foo)
>    2           0 LOAD_FAST                0 (a)
>                3 LOAD_CONST               1 (5)
>                6 BINARY_ADD
>                7 STORE_FAST               1 (b)
>
>    3          10 LOAD_FAST                1 (b)
>               13 RETURN_VALUE
>               14 LOAD_CONST               0 (None)
>               17 RETURN_VALUE
>
> Why does the RETURN_VALUE opcode have to return something from the
> stack? Why not have a RETURN_VAR opcode that would directly return a
> variable or constant? (a leading bit could indicate whether to look in
> the const or var tuple).

Well, you'd have to have RETURN_VAR *as well* as RETURN_VALUE, or in
code like 

    return a + b

you'd have to create a temporary variable and store to it.

We have a limit of 256 opcodes...

> Similarly, what if BINARY_ADD could work directly on constants and
> variables? I see the virtue of using the stack for objects that do not
> otherwise have a name. But if a value is in a contant or variable, why
> not refer to it by its position in co_consts or co_varnames.

How would you implement this?  Give BINARY_ADD two arguments
(something no bytecode has now, btw) and treat '-1' as 'pop from the
stack'?  This sounds obfuscatory.

> And as long as we are talking about referring to things more directly,
> wouldn't it be possible to refer to constants by pointer rather than
> indirecting through the index? You'd have to fix up pointers when you
> first loaded the code but only once per function.

Could do.  Opcode arguments are only 16 bits though, unless you use
the EXTENDED_ARG thingy, and then they're only 32 bits: what about 64
bit platforms?

Python's VM is currently a stack machine.  There are arguments for
making it a register machine, but if we want to do that, lets go the
whole hog and not have some kind of half-assed hybrid.

Cheers,
mwh

-- 
81. In computing, turning the obvious into the useful is a living
    definition of the word "frustration".
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html
From python at rcn.com  Thu Jul  8 02:16:51 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Jul  8 14:17:53 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <40ECDB28.10402@prescod.net>
Message-ID: <000501c46480$de8d2500$e841fea9@oemcomputer>

> Why does the RETURN_VALUE opcode have to return something from the
> stack? Why not have a RETURN_VAR opcode that would directly return a
> variable or constant? (a leading bit could indicate whether to look in
> the const or var tuple).

If the goal is speed, there isn't much to be had.  LOAD_CONSTANT is
blazingly fast and the goto fast_next_opcode cheapens the cost of the
trip around the eval loop.

The eval loop is already pretty tight.  Short of moving to a direct
threading model or some such, the only remaining big boost would be to
move all the tracing/profiling mumbo-jumbo into a separate eval
function.

As for tweaking opcodes, the biggest remaining wins are in LOAD_ATTR and
CALL_FUNCTION.  They've been specialized enough that making further
improvements is non-trivial.

To a lesser extent, there is some pay-off to localizing global lookups.
My experiments show there is only about a 5% total gain to be had.
Localizing is easy; determining what can safely be localized is a
briarpatch.


Raymond

From s.percivall at chello.se  Thu Jul  8 15:04:37 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Thu Jul  8 15:04:40 2004
Subject: [Python-Dev] reporting possible bugs in Python 2.4a1
Message-ID: <5DE03932-D0DF-11D8-BCBC-0003934AD54A@chello.se>

I think I found what might be a bug in the changes made to readline or 
in some change made to threads. Something changed after the different 
readline interface and the unmasking of non-main threads.

Should I do this with the bug-manager or should I post here?

From mwh at python.net  Thu Jul  8 15:14:48 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Jul  8 15:14:51 2004
Subject: [Python-Dev] reporting possible bugs in Python 2.4a1
In-Reply-To: <5DE03932-D0DF-11D8-BCBC-0003934AD54A@chello.se> (Simon
	Percivall's message of "Thu, 8 Jul 2004 15:04:37 +0200")
References: <5DE03932-D0DF-11D8-BCBC-0003934AD54A@chello.se>
Message-ID: <2mvfgyy71z.fsf@starship.python.net>

Simon Percivall <s.percivall@chello.se> writes:

> I think I found what might be a bug in the changes made to readline or
> in some change made to threads. Something changed after the different
> readline interface and the unmasking of non-main threads.

This is not in the way of being a helpful comment :-)

> Should I do this with the bug-manager or should I post here?

Bug tracker's probably best.  Assign it to me.

Cheers,
mwh

-- 
  It's an especially annoying American buzzword for "business use, 
  as opposed to consumer, research, or educational use".
                                    -- Tim Peters defines "enterprise"
From s.percivall at chello.se  Thu Jul  8 15:32:00 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Thu Jul  8 15:32:03 2004
Subject: [Python-Dev] reporting possible bugs in Python 2.4a1
In-Reply-To: <2mvfgyy71z.fsf@starship.python.net>
References: <5DE03932-D0DF-11D8-BCBC-0003934AD54A@chello.se>
	<2mvfgyy71z.fsf@starship.python.net>
Message-ID: <31316328-D0E3-11D8-8F07-0003934AD54A@chello.se>

On 2004-07-08, at 15.14, Michael Hudson wrote:
>>
>
> This is not in the way of being a helpful comment :-)

*lol*

>> Should I do this with the bug-manager or should I post here?
>
> Bug tracker's probably best.  Assign it to me.

The bug is submitted and assigned.

From paul at prescod.net  Thu Jul  8 15:29:16 2004
From: paul at prescod.net (Paul Prescod)
Date: Thu Jul  8 15:33:04 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <2m4qoi254b.fsf@starship.python.net>
References: <40ECDB28.10402@prescod.net> <2m4qoi254b.fsf@starship.python.net>
Message-ID: <40ED4C2C.10806@prescod.net>

Michael Hudson wrote:

>...
> 
> Well, you'd have to have RETURN_VAR *as well* as RETURN_VALUE, or in
> code like 
> 
>     return a + b
> 
> you'd have to create a temporary variable and store to it.
> 
> We have a limit of 256 opcodes...

First, I don't see anything wrong with temporary variables...you need to 
keep track of how many you use I suppose, so the compiler needs to be a 
little bit smarter.

Second, some opcodes seem wasted to my naive eyes. e.g. PRINT_NEWLINE or 
all of the in-place variants of mathematical operators.

Third, you yourself came up with a hack that would allow the same opcode 
to work on variables or the stack using "-1" as the variable index.

>>Similarly, what if BINARY_ADD could work directly on constants and
>>variables? I see the virtue of using the stack for objects that do not
>>otherwise have a name. But if a value is in a contant or variable, why
>>not refer to it by its position in co_consts or co_varnames.
> 
> How would you implement this?  Give BINARY_ADD two arguments
> (something no bytecode has now, btw) and treat '-1' as 'pop from the
> stack'?  This sounds obfuscatory.

If there is anywhere in the Python implementation where you trade some 
"readability" for some performance (and surely there are!) then wouldn't 
the bytecode be the place? I mean bytecodes are *byte* *codes*. They 
aren't tuples of pointers to nice pretty objects. They are a list of 
bytes that are Python's equivalent to assembly language.

>>And as long as we are talking about referring to things more directly,
>>wouldn't it be possible to refer to constants by pointer rather than
>>indirecting through the index? You'd have to fix up pointers when you
>>first loaded the code but only once per function.
> 
> Could do.  Opcode arguments are only 16 bits though, unless you use
> the EXTENDED_ARG thingy, and then they're only 32 bits: what about 64
> bit platforms?

You would have to extend the bytecode format.

> Python's VM is currently a stack machine.  There are arguments for
> making it a register machine, but if we want to do that, lets go the
> whole hog and not have some kind of half-assed hybrid.

I'm not really talking about a register machine either. I don't 
understand why you would want to copy values from a heap in "main 
memory" into a register *still in main memory* to have the bytecodes 
operate on them to store to a register and then back to main memory.

Perhaps we take the CPU analogy too far. Or perhaps there is something 
deep I misunderstand.

  Paul Prescod


From goodger at python.org  Thu Jul  8 15:59:58 2004
From: goodger at python.org (David Goodger)
Date: Thu Jul  8 16:00:05 2004
Subject: [Python-Dev] decimal.py signals & traps
In-Reply-To: <1f7befae04070719401a0a81f3@mail.gmail.com>
References: <40ECA97F.30807@python.org>
	<1f7befae04070719401a0a81f3@mail.gmail.com>
Message-ID: <40ED535E.7000507@python.org>

Please take a look at the patch I submitted to SourceForge:
<http://python.org/sf/987300>.  decimal.Signals is changed to list
signals, not conditions.  Context._raise_error is modified to handle
signals properly.

-- 
David Goodger <http://python.net/~goodger>
From anthony at interlink.com.au  Thu Jul  8 16:10:24 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Jul  8 16:10:36 2004
Subject: [Python-Dev] 2.4a1 - Trunk frozen for release
In-Reply-To: <40ECEDD3.1080906@interlink.com.au>
References: <40EC1FBD.4050305@interlink.com.au>
	<40ECEDD3.1080906@interlink.com.au>
Message-ID: <40ED55D0.7050804@interlink.com.au>

Anthony Baxter wrote:
> Please hold off on committing anything to the trunk unless
> you're involved in the release for the next 24 hours or so.
> 
> 
> I'll send another email when it's safe to commit again.

Ok, the tarballs and .msi are on the ftp site. I'm still working
through the webpages - I will finish them in the morning.

So please feel free to commit to the trunk again.



From Paul.Moore at atosorigin.com  Thu Jul  8 16:12:23 2004
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Thu Jul  8 16:12:21 2004
Subject: [Python-Dev] Capitalisation of Decimal class
Message-ID: <16E1010E4581B049ABC51D4975CEDB88052CB26E@UKDCX001.uk.int.atosorigin.com>

I just had a thought regarding the Decimal class. If the longer-term
intention is to move the class into the core, then would it be
sensible to name the class "decimal" (lowercase) rather than
"Decimal"? The situation seems similar to the set class, which
undergoes a capitalisation change in 2.4 in a similar manner.

Paul.


__________________________________________________________________________
This e-mail and the documents attached are confidential and intended 
solely for the addressee; it may also be privileged. If you receive this 
e-mail in error, please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos Origin group 
liability cannot be triggered for the message content. Although the 
sender endeavours to maintain a computer virus-free network, the sender 
does not warrant that this transmission is virus-free and will not be 
liable for any damages resulting from any virus transmitted.
__________________________________________________________________________
From paul at prescod.net  Thu Jul  8 16:14:42 2004
From: paul at prescod.net (Paul Prescod)
Date: Thu Jul  8 16:14:38 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <000501c46480$de8d2500$e841fea9@oemcomputer>
References: <000501c46480$de8d2500$e841fea9@oemcomputer>
Message-ID: <40ED56D2.6010700@prescod.net>

Raymond Hettinger wrote:

>...
> The eval loop is already pretty tight.

Compared to what? Total runtime of Python programs? Psycho? Java 
implementations?

I suppose the statement is based on the observation that compiling away 
the eval loop by using Pyrex or Python2C results in very little speed gain.

> As for tweaking opcodes, the biggest remaining wins are in LOAD_ATTR and
> CALL_FUNCTION.  They've been specialized enough that making further
> improvements is non-trivial.

I'm really not even proposing a change to Python as much as trying to 
understand why it is the way it is. I can see how a stack is helpful for 
evaluating deeply nested expressions (although temporary variables seems 
another good valid approach). I just don't see the payoff for shuffling 
variables from locals onto the stack and back into the locals table.

  Paul Prescod


From bob at redivi.com  Thu Jul  8 16:19:12 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Jul  8 16:19:29 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <40ED4C2C.10806@prescod.net>
References: <40ECDB28.10402@prescod.net> <2m4qoi254b.fsf@starship.python.net>
	<40ED4C2C.10806@prescod.net>
Message-ID: <C9403372-D0E9-11D8-BB26-000A95686CD8@redivi.com>


On Jul 8, 2004, at 9:29 AM, Paul Prescod wrote:

> Michael Hudson wrote:
>> Python's VM is currently a stack machine.  There are arguments for
>> making it a register machine, but if we want to do that, lets go the
>> whole hog and not have some kind of half-assed hybrid.
>
> I'm not really talking about a register machine either. I don't 
> understand why you would want to copy values from a heap in "main 
> memory" into a register *still in main memory* to have the bytecodes 
> operate on them to store to a register and then back to main memory.
>
> Perhaps we take the CPU analogy too far. Or perhaps there is something 
> deep I misunderstand.

My guess is that register machines are a whole heck of a lot easier to 
JIT compile to efficient register CPU assembly code (at least on a 
platform that has more than 4 GPRs :).

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040708/1deb0600/smime-0001.bin
From skip at pobox.com  Thu Jul  8 16:21:16 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Jul  8 16:21:15 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <40ED4C2C.10806@prescod.net>
References: <40ECDB28.10402@prescod.net> <2m4qoi254b.fsf@starship.python.net>
	<40ED4C2C.10806@prescod.net>
Message-ID: <16621.22620.304068.926781@montanaro.dyndns.org>


    >> Python's VM is currently a stack machine.  There are arguments for
    >> making it a register machine, but if we want to do that, lets go the
    >> whole hog and not have some kind of half-assed hybrid.

    Paul> I'm not really talking about a register machine either. I don't
    Paul> understand why you would want to copy values from a heap in "main
    Paul> memory" into a register *still in main memory* to have the
    Paul> bytecodes operate on them to store to a register and then back to
    Paul> main memory.

When I looked at this a long time ago it I seem to recall that roughly 60%
of the opcodes executed did nothing more than copy values to or from the
stack (of course, pushes and pops are some of the more efficient opcodes in
the instruction set).  A three-address machine model would reduce this data
movement substantially.

Skip
From jhylton at gmail.com  Thu Jul  8 16:29:25 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Thu Jul  8 16:29:33 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <40ED56D2.6010700@prescod.net>
References: <000501c46480$de8d2500$e841fea9@oemcomputer>
	<40ED56D2.6010700@prescod.net>
Message-ID: <e8bf7a5304070807293e979283@mail.gmail.com>

On Thu, 08 Jul 2004 07:14:42 -0700, Paul Prescod <paul@prescod.net> wrote:
> I'm really not even proposing a change to Python as much as trying to
> understand why it is the way it is. I can see how a stack is helpful for
> evaluating deeply nested expressions (although temporary variables seems
> another good valid approach). I just don't see the payoff for shuffling
> variables from locals onto the stack and back into the locals table.

A stack machine is simple to implement and simple to understand.  It's
also a classic approach; lots of language implementations have
stack-based virtual machines.  Why is it the way it is?  Because it
is.  That is, it's an obvious choice for implementing a language.

It doesn't need to be that way.  Lua and Parrot, to pick to recent
examples, have register-based virtual machines.  Roberto Ierusalimschy
gave a nice talk on Lua 5 at LL3.  Lua 4 had a stack-based virtual
machine, and Lua 5 has a register-based virtual machine.  The
benchmarks he reported there showed a 4-16% speedup largely
attributable to the new virtual machine.  He also had some instructive
closing comments:

Compiler for register-based *machine* is more complex
 - needs some primitive optimizations to use registers
Interpreter for register-based *machine* is more complex
 - needs to decode instructions
Requirements
 - no more than 256 local variables and temporaries
Main gains:
 - avoid moves of local variables and constants
 - fewer instructions per task
 - potential gain with CSE optimizations

Slides are here:
http://www.inf.puc-rio.br/~roberto/talks/lua-ll3.pdf

Jeremy
From python at rcn.com  Thu Jul  8 05:53:27 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Jul  8 17:54:31 2004
Subject: [Python-Dev] 2.4a1
In-Reply-To: <1089225596.8354.517.camel@localhost>
Message-ID: <001e01c4649f$20c9f2e0$e841fea9@oemcomputer>

[Barry]
> I haven't had (and won't have) time to land the PEP 292 stuff by then.
> I will follow up on that thread and shoot for inclusion in CVS for the
> next alpha.

I would like to be the second reviewer for that patch.


Raymond

From guido at python.org  Thu Jul  8 18:01:31 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul  8 18:01:37 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: Your message of "Thu, 08 Jul 2004 07:14:42 PDT."
	<40ED56D2.6010700@prescod.net> 
References: <000501c46480$de8d2500$e841fea9@oemcomputer>  
	<40ED56D2.6010700@prescod.net> 
Message-ID: <200407081601.i68G1Vh29611@guido.python.org>

> I'm really not even proposing a change to Python as much as trying
> to understand why it is the way it is. I can see how a stack is
> helpful for evaluating deeply nested expressions (although temporary
> variables seems another good valid approach). I just don't see the
> payoff for shuffling variables from locals onto the stack and back
> into the locals table.

To minimize the instruction set and simplify the compiler.  You'll
need the most general operation (return from stack) anyway.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Jul  8 18:04:37 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul  8 18:04:43 2004
Subject: [Python-Dev] Capitalisation of Decimal class
In-Reply-To: Your message of "Thu, 08 Jul 2004 15:12:23 BST."
	<16E1010E4581B049ABC51D4975CEDB88052CB26E@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB88052CB26E@UKDCX001.uk.int.atosorigin.com>
Message-ID: <200407081604.i68G4bN29644@guido.python.org>

> I just had a thought regarding the Decimal class. If the longer-term
> intention is to move the class into the core, then would it be
> sensible to name the class "decimal" (lowercase) rather than
> "Decimal"? The situation seems similar to the set class, which
> undergoes a capitalisation change in 2.4 in a similar manner.

I think it's actually a good thing to have a name change; there are
always going to be incompatibilities, and code must be able to use the
old and the new way together.  If they are named the same that just
confuses matters.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From fumanchu at amor.org  Thu Jul  8 18:23:56 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Thu Jul  8 18:28:15 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E5622F9@opus.amorhq.net>

Paul Prescod wrote:
>  >>> def foo(a):
> ...     b = a + 5
> ...     return b
>
> Why does the RETURN_VALUE opcode have to return something from the
> stack? Why not have a RETURN_VAR opcode that would directly return a
> variable or constant? (a leading bit could indicate whether to look in
> the const or var tuple).

I notice that your example could be rewritten to return directly from
the stack. Perhaps the 'code' module could grow an .optimize function or
submodule...?

> And as long as we are talking about referring to things more directly,
> wouldn't it be possible to refer to constants by pointer rather than
> indirecting through the index? You'd have to fix up pointers when you
> first loaded the code but only once per function.

Then all our fancy bytecode hacks would have to redirect the pointers
back into the index again. ;)


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
From fumanchu at amor.org  Thu Jul  8 18:29:24 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Thu Jul  8 18:33:42 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E5622FB@opus.amorhq.net>

I wrote:
> Perhaps the 'code' module could grow an .optimize function or
> submodule...?

Bah. I meant the 'new' module.
From pje at telecommunity.com  Thu Jul  8 18:52:00 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Jul  8 18:48:57 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <40ED56D2.6010700@prescod.net>
References: <000501c46480$de8d2500$e841fea9@oemcomputer>
	<000501c46480$de8d2500$e841fea9@oemcomputer>
Message-ID: <5.1.1.6.0.20040708124359.021dc380@mail.telecommunity.com>

At 07:14 AM 7/8/04 -0700, Paul Prescod wrote:

>I'm really not even proposing a change to Python as much as trying to 
>understand why it is the way it is. I can see how a stack is helpful for 
>evaluating deeply nested expressions (although temporary variables seems 
>another good valid approach). I just don't see the payoff for shuffling 
>variables from locals onto the stack and back into the locals table.

The payoff is in shorter development time...  of the compiler.  :)

Essentially, most of Python compilation is simply rearranging expressions 
to be postfix notation instead of infix notation.  Thus, it's relatively 
straightforward to write a compiler for, because the bytecode emissions 
match the recursive structure.  Although you need to keep track of the 
maximum stack depth, you don't need to worry about register allocation.

From barrett at stsci.edu  Thu Jul  8 19:21:52 2004
From: barrett at stsci.edu (Paul Barrett)
Date: Thu Jul  8 19:22:05 2004
Subject: [Python-Dev] What is with the Python website?
Message-ID: <40ED82B0.9090103@stsci.edu>


It looks like www.python.org has been hacked.

-- 
Paul Barrett, PhD      Space Telescope Science Institute
Phone: 410-338-4475    ESS/Science Software Branch
FAX:   410-338-4767    Baltimore, MD 21218

From mwh at python.net  Thu Jul  8 19:25:56 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Jul  8 19:25:57 2004
Subject: [Python-Dev] What is with the Python website?
In-Reply-To: <40ED82B0.9090103@stsci.edu> (Paul Barrett's message of "Thu,
	08 Jul 2004 13:21:52 -0400")
References: <40ED82B0.9090103@stsci.edu>
Message-ID: <2mk6xexvff.fsf@starship.python.net>

Paul Barrett <barrett@stsci.edu> writes:

> It looks like www.python.org has been hacked.

Looks perfectly normal to me.  What do you see?

Cheers,
mwh

-- 
  If design space weren't so vast, and the good solutions so small a
  portion of it, programming would be a lot easier.
                                            -- maney, comp.lang.python
From jjl at pobox.com  Thu Jul  8 19:34:24 2004
From: jjl at pobox.com (John J Lee)
Date: Thu Jul  8 19:31:43 2004
Subject: [Python-Dev] What is with the Python website?
In-Reply-To: <2mk6xexvff.fsf@starship.python.net>
References: <40ED82B0.9090103@stsci.edu> <2mk6xexvff.fsf@starship.python.net>
Message-ID: <Pine.WNT.4.58.0407081833110.1484@vernon>

On Thu, 8 Jul 2004, Michael Hudson wrote:

> Paul Barrett <barrett@stsci.edu> writes:
>
> > It looks like www.python.org has been hacked.
>
> Looks perfectly normal to me.  What do you see?

Black page with this silly script-kiddie message:


> HellFire Jodio tu sistema admin :)
>
> [Orgullosamente Dominicano]
>
> [CONTACTO: eu.undernet.org] [#Security-Labs]
>
> [hellroots@linuxmail.org] ]
>
> 14546 ttyp1 00:00:00 rootkit [I AM Backd00ring this B0x :)]
>
> 14945 pts/2 00:00:00 hide [backd00r sucess and hided ]
>
> 145946 pts/2 00:00:00 root [your are Hacked But u don't know]
> HellFire Jodio tu sistema admin :)


John
From bob at redivi.com  Thu Jul  8 19:34:44 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Jul  8 19:34:59 2004
Subject: [Python-Dev] What is with the Python website?
In-Reply-To: <2mk6xexvff.fsf@starship.python.net>
References: <40ED82B0.9090103@stsci.edu> <2mk6xexvff.fsf@starship.python.net>
Message-ID: <19DE8A48-D105-11D8-BB26-000A95686CD8@redivi.com>


On Jul 8, 2004, at 1:25 PM, Michael Hudson wrote:

> Paul Barrett <barrett@stsci.edu> writes:
>
>> It looks like www.python.org has been hacked.
>
> Looks perfectly normal to me.  What do you see?

He probably went to www.python.com :)

-bob

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040708/425e41d7/smime-0001.bin
From barrett at stsci.edu  Thu Jul  8 19:35:38 2004
From: barrett at stsci.edu (Paul Barrett)
Date: Thu Jul  8 19:35:52 2004
Subject: [Python-Dev] What is with the Python website?
In-Reply-To: <2mk6xexvff.fsf@starship.python.net>
References: <40ED82B0.9090103@stsci.edu> <2mk6xexvff.fsf@starship.python.net>
Message-ID: <40ED85EA.2050803@stsci.edu>

Michael Hudson wrote:
> Paul Barrett <barrett@stsci.edu> writes:
> 
> 
>>It looks like www.python.org has been hacked.
> 
> 
> Looks perfectly normal to me.  What do you see?



It got better.

-- 
Paul Barrett, PhD      Space Telescope Science Institute
Phone: 410-338-4475    ESS/Science Software Branch
FAX:   410-338-4767    Baltimore, MD 21218

From FBatista at uniFON.com.ar  Thu Jul  8 19:32:14 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Thu Jul  8 19:35:55 2004
Subject: [Python-Dev] What is with the Python website?
Message-ID: <A128D751272CD411BC9200508BC2194D03383B54@escpl.tcp.com.ar>

[Paul Barrett]
#- It looks like www.python.org has been hacked.

Looks fine to me.

Doing a ping, I get this IP: 194.109.137.226

You? Maybe your DNS was hacked...

.	Facundo
From bob at redivi.com  Thu Jul  8 19:39:03 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Jul  8 19:39:14 2004
Subject: [Python-Dev] What is with the Python website?
In-Reply-To: <Pine.WNT.4.58.0407081833110.1484@vernon>
References: <40ED82B0.9090103@stsci.edu> <2mk6xexvff.fsf@starship.python.net>
	<Pine.WNT.4.58.0407081833110.1484@vernon>
Message-ID: <B4C24EDA-D105-11D8-BB26-000A95686CD8@redivi.com>


On Jul 8, 2004, at 1:34 PM, John J Lee wrote:

> On Thu, 8 Jul 2004, Michael Hudson wrote:
>
>> Paul Barrett <barrett@stsci.edu> writes:
>>
>>> It looks like www.python.org has been hacked.
>>
>> Looks perfectly normal to me.  What do you see?
>
> Black page with this silly script-kiddie message:
>
>
>> HellFire Jodio tu sistema admin :)
>>
>> [Orgullosamente Dominicano]
>>
>> [CONTACTO: eu.undernet.org] [#Security-Labs]
>>
>> [hellroots@linuxmail.org] ]
>>
>> 14546 ttyp1 00:00:00 rootkit [I AM Backd00ring this B0x :)]
>>
>> 14945 pts/2 00:00:00 hide [backd00r sucess and hided ]
>>
>> 145946 pts/2 00:00:00 root [your are Hacked But u don't know]
>> HellFire Jodio tu sistema admin :)

Wow, I didn't see that... It worked at first, but now I see a 403 
Forbidden on www.python.org (but not docs.python.org)!

This is what I get for DNS:

[crack:~] bob% dig www.python.org

; <<>> DiG 9.2.2 <<>> www.python.org
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35567
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 2, ADDITIONAL: 2

;; QUESTION SECTION:
;www.python.org.                        IN      A

;; ANSWER SECTION:
www.python.org.         86269   IN      CNAME   fang.python.org.
fang.python.org.        86269   IN      A       194.109.137.226

;; AUTHORITY SECTION:
python.org.             33600   IN      NS      ns.xs4all.nl.
python.org.             33600   IN      NS      ns2.xs4all.nl.

;; ADDITIONAL SECTION:
ns.xs4all.nl.           21233   IN      A       194.109.6.67
ns2.xs4all.nl.          21233   IN      A       194.109.9.100

;; Query time: 37 msec
;; SERVER: 10.0.3.1#53(10.0.3.1)
;; WHEN: Thu Jul  8 13:36:09 2004
;; MSG SIZE  rcvd: 143

-bob

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040708/94874c1b/smime.bin
From mwh at python.net  Thu Jul  8 19:40:54 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Jul  8 19:40:55 2004
Subject: [Python-Dev] What is with the Python website?
In-Reply-To: <B4C24EDA-D105-11D8-BB26-000A95686CD8@redivi.com> (Bob
	Ippolito's message of "Thu, 8 Jul 2004 13:39:03 -0400")
References: <40ED82B0.9090103@stsci.edu> <2mk6xexvff.fsf@starship.python.net>
	<Pine.WNT.4.58.0407081833110.1484@vernon>
	<B4C24EDA-D105-11D8-BB26-000A95686CD8@redivi.com>
Message-ID: <2mfz82xuqh.fsf@starship.python.net>

Bob Ippolito <bob@redivi.com> writes:

> On Jul 8, 2004, at 1:34 PM, John J Lee wrote:
>
>> On Thu, 8 Jul 2004, Michael Hudson wrote:
>>
>>> Paul Barrett <barrett@stsci.edu> writes:
>>>
>>>> It looks like www.python.org has been hacked.
>>>
>>> Looks perfectly normal to me.  What do you see?
>>
>> Black page with this silly script-kiddie message:
>>
>>
>>> HellFire Jodio tu sistema admin :)
>>>
>>> [Orgullosamente Dominicano]
>>>
>>> [CONTACTO: eu.undernet.org] [#Security-Labs]
>>>
>>> [hellroots@linuxmail.org] ]
>>>
>>> 14546 ttyp1 00:00:00 rootkit [I AM Backd00ring this B0x :)]
>>>
>>> 14945 pts/2 00:00:00 hide [backd00r sucess and hided ]
>>>
>>> 145946 pts/2 00:00:00 root [your are Hacked But u don't know]
>>> HellFire Jodio tu sistema admin :)
>
> Wow, I didn't see that... It worked at first, but now I see a 403
> Forbidden on www.python.org (but not docs.python.org)!

It was real, and is being fixed as we speak...

Cheers,
mwh

-- 
 All parts should go together without forcing. You must remember that
 the parts you are  reassembling were disassembled by you.  Therefore,
 if you can't get them together again, there must be a reason. By all
 means, do not use a hammer.           -- IBM maintenance manual, 1925
From walter at livinglogic.de  Thu Jul  8 20:42:40 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu Jul  8 20:42:50 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
 stringobject.c, 2.219, 2.220 unicodeobject.c, 2.213, 2.214
In-Reply-To: <E1Bid9M-0002rJ-6z@sc8-pr-cvs1.sourceforge.net>
References: <E1Bid9M-0002rJ-6z@sc8-pr-cvs1.sourceforge.net>
Message-ID: <40ED95A0.4090905@livinglogic.de>

lemburg@users.sourceforge.net wrote:

> Modified Files:
> 	stringobject.c unicodeobject.c 
> Log Message:
> Allow string and unicode return types from .encode()/.decode()
> methods on string and unicode objects. Added unicode.decode()
> which was missing for no apparent reason.

This seems to have broken test_codeccallbacks.py. I get:
test_backslashescape (__main__.CodecCallbackTest) ... ok
test_badandgoodbackslashreplaceexceptions (__main__.CodecCallbackTest) 
... ok
test_badandgoodignoreexceptions (__main__.CodecCallbackTest) ... ok
test_badandgoodreplaceexceptions (__main__.CodecCallbackTest) ... ok
test_badandgoodstrictexceptions (__main__.CodecCallbackTest) ... ok
test_badandgoodxmlcharrefreplaceexceptions (__main__.CodecCallbackTest) 
... ok
Segmentation fault

Rolling back the patch via:
cvs up -j 2.44 -j 2.43 Include/unicodeobject.h
cvs up -j 2.220 -j 2.219 Objects/stringobject.c
cvs up -j 2.214 -j 2.213 Objects/unicodeobject.c
gives a working test_codeccallbacks.py again.

Bye,
    Walter D?rwald


From mal at egenix.com  Thu Jul  8 21:11:45 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Jul  8 21:11:49 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
 stringobject.c, 2.219, 2.220 unicodeobject.c, 2.213, 2.214
In-Reply-To: <40ED95A0.4090905@livinglogic.de>
References: <E1Bid9M-0002rJ-6z@sc8-pr-cvs1.sourceforge.net>
	<40ED95A0.4090905@livinglogic.de>
Message-ID: <40ED9C71.7090009@egenix.com>

Walter D?rwald wrote:
> lemburg@users.sourceforge.net wrote:
> 
>> Modified Files:
>>     stringobject.c unicodeobject.c Log Message:
>> Allow string and unicode return types from .encode()/.decode()
>> methods on string and unicode objects. Added unicode.decode()
>> which was missing for no apparent reason.
> 
> 
> This seems to have broken test_codeccallbacks.py. I get:
> test_backslashescape (__main__.CodecCallbackTest) ... ok
> test_badandgoodbackslashreplaceexceptions (__main__.CodecCallbackTest) 
> ... ok
> test_badandgoodignoreexceptions (__main__.CodecCallbackTest) ... ok
> test_badandgoodreplaceexceptions (__main__.CodecCallbackTest) ... ok
> test_badandgoodstrictexceptions (__main__.CodecCallbackTest) ... ok
> test_badandgoodxmlcharrefreplaceexceptions (__main__.CodecCallbackTest) 
> ... ok
> Segmentation fault
> 
> Rolling back the patch via:
> cvs up -j 2.44 -j 2.43 Include/unicodeobject.h
> cvs up -j 2.220 -j 2.219 Objects/stringobject.c
> cvs up -j 2.214 -j 2.213 Objects/unicodeobject.c
> gives a working test_codeccallbacks.py again.

Dang: I accidentally applied an older patch.
Fix is coming.

Sorry,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 08 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Thu Jul  8 21:17:08 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Jul  8 21:17:13 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
 stringobject.c, 2.219, 2.220 unicodeobject.c, 2.213, 2.214
In-Reply-To: <40ED9C71.7090009@egenix.com>
References: <E1Bid9M-0002rJ-6z@sc8-pr-cvs1.sourceforge.net>	<40ED95A0.4090905@livinglogic.de>
	<40ED9C71.7090009@egenix.com>
Message-ID: <40ED9DB4.6060406@egenix.com>

M.-A. Lemburg wrote:
> Walter D?rwald wrote:
> 
>> lemburg@users.sourceforge.net wrote:
>>
>>> Modified Files:
>>>     stringobject.c unicodeobject.c Log Message:
>>> Allow string and unicode return types from .encode()/.decode()
>>> methods on string and unicode objects. Added unicode.decode()
>>> which was missing for no apparent reason.
>>
>>
>>
>> This seems to have broken test_codeccallbacks.py. I get:
>> test_backslashescape (__main__.CodecCallbackTest) ... ok
>> test_badandgoodbackslashreplaceexceptions (__main__.CodecCallbackTest) 
>> ... ok
>> test_badandgoodignoreexceptions (__main__.CodecCallbackTest) ... ok
>> test_badandgoodreplaceexceptions (__main__.CodecCallbackTest) ... ok
>> test_badandgoodstrictexceptions (__main__.CodecCallbackTest) ... ok
>> test_badandgoodxmlcharrefreplaceexceptions 
>> (__main__.CodecCallbackTest) ... ok
>> Segmentation fault
>>
>> Rolling back the patch via:
>> cvs up -j 2.44 -j 2.43 Include/unicodeobject.h
>> cvs up -j 2.220 -j 2.219 Objects/stringobject.c
>> cvs up -j 2.214 -j 2.213 Objects/unicodeobject.c
>> gives a working test_codeccallbacks.py again.
> 
> 
> Dang: I accidentally applied an older patch.
> Fix is coming.

Should work now.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 08 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From andymac at bullseye.apana.org.au  Thu Jul  8 15:33:30 2004
From: andymac at bullseye.apana.org.au (Andrew MacIntyre)
Date: Thu Jul  8 21:23:31 2004
Subject: [Python-Dev] New test failure, test_socket, WinXP
In-Reply-To: <1089132447.8521.409.camel@localhost>
References: <20040706040145.45DA21E4005@bag.python.org>
	<40EA3205.4060804@ocf.berkeley.edu>
	<1089132447.8521.409.camel@localhost>
Message-ID: <20040708232226.M27271@bullseye.apana.org.au>

On Tue, 6 Jul 2004, Barry Warsaw wrote:

> On Tue, 2004-07-06 at 01:00, Brett C. wrote:
>
> > Personally I am +1 on changing 'www' to 'http' in the test and moving
> > on.
>
> That won't work on Linux, apparently.  Okay, I looked at the Windows
> file that Brett mentioned and it looks like checking 'echo', 'daytime',
> and 'domain' should do the trick.  I'll commit the change.

Having 'echo' first breaks the test on FreeBSD, at the assertion in line
302 of test_socket, because FreeBSD's services file includes records for
protocol "ddp" (AppleTalk over TCP/IP??), including an "echo" on port 4
rather than the expected port 7 for the "tcp" & "udp" protocols.

Would having the list as 'daytime', 'echo', 'domain' cause any other
problems?

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac@pcug.org.au             (alt) |        Belconnen  ACT  2616
Web:    http://www.andymac.org/               |        Australia
From walter at livinglogic.de  Thu Jul  8 21:26:25 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu Jul  8 21:26:30 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
	stringobject.c, 2.219, 2.220 unicodeobject.c, 2.213, 2.214
In-Reply-To: <40ED9DB4.6060406@egenix.com>
References: <E1Bid9M-0002rJ-6z@sc8-pr-cvs1.sourceforge.net>	<40ED95A0.4090905@livinglogic.de>	<40ED9C71.7090009@egenix.com>
	<40ED9DB4.6060406@egenix.com>
Message-ID: <40ED9FE1.3000209@livinglogic.de>

M.-A. Lemburg wrote:

> M.-A. Lemburg wrote:
> 
> [...]
> Should work now.

Much better:
$ ./python Lib/test/test_codeccallbacks.py
test_backslashescape (__main__.CodecCallbackTest) ... ok
[...]
test_xmlcharrefvalues (__main__.CodecCallbackTest) ... ok

----------------------------------------------------------------------
Ran 26 tests in 0.144s

Bye,
    Walter D?rwald


From mal at egenix.com  Thu Jul  8 21:27:36 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Jul  8 21:27:39 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Objects	stringobject.c, 
	2.219, 2.220 unicodeobject.c, 2.213, 2.214
In-Reply-To: <40ED9FE1.3000209@livinglogic.de>
References: <E1Bid9M-0002rJ-6z@sc8-pr-cvs1.sourceforge.net>	<40ED95A0.4090905@livinglogic.de>	<40ED9C71.7090009@egenix.com>	<40ED9DB4.6060406@egenix.com>
	<40ED9FE1.3000209@livinglogic.de>
Message-ID: <40EDA028.205@egenix.com>

Walter D?rwald wrote:
> M.-A. Lemburg wrote:
> 
>> M.-A. Lemburg wrote:
>>
>> [...]
>> Should work now.
> 
> 
> Much better:
> $ ./python Lib/test/test_codeccallbacks.py
> test_backslashescape (__main__.CodecCallbackTest) ... ok
> [...]
> test_xmlcharrefvalues (__main__.CodecCallbackTest) ... ok
> 
> ----------------------------------------------------------------------
> Ran 26 tests in 0.144s

Good catch !

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 08 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From barry at python.org  Thu Jul  8 22:30:05 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Jul  8 22:29:58 2004
Subject: [Python-Dev] www.python.org defacement
Message-ID: <1089318605.8253.578.camel@localhost>

Some of you may have noticed that www.python.org got defaced today.  We
also believe that the site has been used for some time to trade warez,
but we do not think that it was rooted.  We believe we have closed the
holes and repaired the damage.  Access to some of the accounts on the
machine have been disabled, but those folks know who they are and how to
get them re-enabled.

I have verified by GPG signature all three Python 2.3.4 downloads (the
tgz, tar.bz2, and exe) and I don't believe they were trojaned.  I'll be
checking the other signed distros and will report if I smell anything
fishy.  If you find anything missing or notice anything hinky, please
let us know via the webmaster@python.org or pydotorg@python.org mailing
lists.

Send your derision to me, but send your thanks to Thomas, Skip, Andrew,
Michael and the other volunteer admins for the site, who stepped up
quickly and effectively to fix this, and verify and restore the content.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040708/69f2fd14/attachment.pgp
From niemeyer at conectiva.com  Thu Jul  8 22:47:22 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu Jul  8 22:46:14 2004
Subject: [Python-Dev] 2.4a1
In-Reply-To: <1089225596.8354.517.camel@localhost>
References: <40EC1FBD.4050305@interlink.com.au>
	<1089225596.8354.517.camel@localhost>
Message-ID: <20040708204722.GA8989@burma.localdomain>

> > I plan to start attempting to get a 2.4a1 out the door
> > in the morning my time. Depending on how that goes, it
> > will either be then, or on the Friday - I'll post updates
> > when I see how it's going.
> 
> I haven't had (and won't have) time to land the PEP 292 stuff by then. 
> I will follow up on that thread and shoot for inclusion in CVS for the
> next alpha.

Barry, if you get the necessary time, I'd really appreciate if you
could have a look at patch #980500 before I check it in.

Thanks!

-- 
Gustavo Niemeyer
http://niemeyer.net
From martin at v.loewis.de  Thu Jul  8 23:37:13 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul  8 23:37:18 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <40ED4C2C.10806@prescod.net>
References: <40ECDB28.10402@prescod.net> <2m4qoi254b.fsf@starship.python.net>
	<40ED4C2C.10806@prescod.net>
Message-ID: <40EDBE89.3040304@v.loewis.de>

Paul Prescod wrote:
> First, I don't see anything wrong with temporary variables...you need to 
> keep track of how many you use I suppose, so the compiler needs to be a 
> little bit smarter.

There is something wrong with it: it is painful to implement.

Just think of the stack as a set of temporary variables, and
RETURN_VALUE becomes the same thing as RETURN_VAR.

> Second, some opcodes seem wasted to my naive eyes. e.g. PRINT_NEWLINE or 
> all of the in-place variants of mathematical operators.

Yes, that is correct. However, even if that was cleaned-up (which is
difficult and time-consuming to do), there still would be only 256
opcodes (at some point, there were only 128 of them).

> Third, you yourself came up with a hack that would allow the same opcode 
> to work on variables or the stack using "-1" as the variable index.

Might be. You are mixing questions now: are you asking "why is it the
way it is?" or "can we please change it?"

It is the way it is because a stack machine is the easiest thing to
implement and source-review.

> If there is anywhere in the Python implementation where you trade some 
> "readability" for some performance (and surely there are!) then wouldn't 
> the bytecode be the place? I mean bytecodes are *byte* *codes*. They 
> aren't tuples of pointers to nice pretty objects. They are a list of 
> bytes that are Python's equivalent to assembly language.

Yes. That makes bugs in the implementation most difficult to track down,
and obviousness in the implementation a high priority, even more than
performance: what good is a fast but incorrect implementation of the
language?

> Perhaps we take the CPU analogy too far. Or perhaps there is something 
> deep I misunderstand.

Most certainly. This stuff is *way*, *extremely* more difficult to write
than it is to read. Just try and see for yourself. The number one reason
that possible changes have not been carried out is that they are so 
difficult to implement even if they are straight-forward to talk about.
The number two reason is that those changes don't actually show the
advantages that they theoretically should show, because the abstract
model abstracts too much of the real computer hardware.

Regards,
Martin

From skip at pobox.com  Thu Jul  8 23:52:51 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Jul  8 23:52:38 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <40EDBE89.3040304@v.loewis.de>
References: <40ECDB28.10402@prescod.net> <2m4qoi254b.fsf@starship.python.net>
	<40ED4C2C.10806@prescod.net> <40EDBE89.3040304@v.loewis.de>
Message-ID: <16621.49715.886608.828423@montanaro.dyndns.org>


    >> Perhaps we take the CPU analogy too far. Or perhaps there is
    >> something deep I misunderstand.

    Martin> Most certainly. This stuff is *way*, *extremely* more difficult
    Martin> to write than it is to read. Just try and see for yourself. 

I wrote the rattlesnake VM (three-address model) ages ago (1.4 or 1.5?).  I
got it mostly to work, but never quite 100%.  There were a couple
instructions which were difficult to map into the register model for some
reason.  I suspect there may be more today, though moves to write a
reasonable bytecode verifier may improve that situation.  Even if I had
gotten it to work 100%, I would have been left with the problem of tracking
updates to the stack-based VM - an internal implementation detail don't
forget - until a cutover to the register-based VM could be made, and there'd
be no guarantees that would ever happen.  (Keeping the stackless source
sync'd with the main CPython source must be a challenging task for Christian
Tismer.)

I still have the source if anyone's interested, though so much water has
passed under the bridge now that it would only be useful as a rough guide.
Forget "patch -p 0 < ..."!

Skip
From jim at zope.com  Fri Jul  9 01:10:22 2004
From: jim at zope.com (Jim Fulton)
Date: Fri Jul  9 01:10:57 2004
Subject: [Python-Dev] Make PyType_GenericNew and object_new the same
Message-ID: <40EDD45E.5080507@zope.com>

At one time, object's tp_new slot was filled with PyType_GenericNew.

Later, someone decided to add a sanity check to object's new that
raises an exception if arguments are passed and __init__ was
interited from object. (object's __init__ takes no arguments.)
At this point, objects's tp_new became different from PyType_GenericNew.

Now, I have some C types that are meant to be used as mix-in
classes. I have to provide a tp_new slot filler if I want these
classes (or their subclasses) to be callable. The canonical way
to do this is to use PyType_GenericNew.  This makes the type's
tp_new different from object's, even though the intent is to have
them be basically the same.  If they are different, and you
you can run into trouble if subclasses, mixed with other base
classes try to override __new__.  You get errors like:

   TypeError: yourbaseclass.__new__(MetaInterface) is not safe,
   use object.__new__()

For reasons you don't want me to go into :)
this results from the fact that the base class __new__
and object.__new__ are different.

IMO opinion, the error checks in object_new would be just as
useful in PyType_GenericNew.  I propose one of:

1. Move the checks in object_new to PyType_GenericNew, get
    rid of object_new and fill object's tp_new slot with
    PyType_GenericNew, or

2. Make the canonical way to fill the tp_new slot of a
    type that doesn't want to provide it's own new implementation
    and that does want to be callable to just copy object's tp_new
    slot:

      MyType.tp_new = PyBaseObject_Type.tp_new;

If no one objects, I'll implement 1. Otherwise, I'll implement 2 by
updating the tutorial.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From python at rcn.com  Thu Jul  8 13:38:54 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Jul  9 01:39:59 2004
Subject: [Python-Dev] decimal.py signals & traps
In-Reply-To: <40ED535E.7000507@python.org>
Message-ID: <000f01c464e0$261b6d00$e841fea9@oemcomputer>

[David Goodger]
> Please take a look at the patch I submitted to SourceForge:
> <http://python.org/sf/987300>.  decimal.Signals is changed to list
> signals, not conditions.  Context._raise_error is modified to handle
> signals properly.


This should be tackled in steps and as part of larger improvement to the
context API.

The most uncomfortable part of the Python interface to contexts is
creating the dictionaries for traps and flags arguments.  Besides being
a pain to create and display, there are issues with two contexts sharing
the same underlying dictionaries.  Also, it makes it difficult to
address the issue at hand (as noted by the one XXX comment in the module
and by your bug report).

As a first step, I would like change the Context constructor API to
match the format in Context.__repr__.  That format is somewhat easier to
use and less verbose (list only the flags and traps that you want set).
Also, I would add accessor methods patterned after the Language
Independent Arithmetic standard (LIA-1) to set, clear, and test flags
and traps.  The dictionaries themselves would be made private (allowing
them to be recast as sets if desired).

With a method interface in place, a next step would be to add logic to
maintain relationships between signals.  Setting the ConversionSyntax
flag also sets the InvalidOperation flag.  Likewise setting a trap for
InvalidOperation would set the trap for ConversionSyntax.

As a side benefit, the above changes will make it easier to someday
implement the module in C.  Holding user supplied, volatile dictionaries
was not a recipe for success in that regard.

I'm less enthusiastic about changing any of the exception names but none
of the above precludes that as a last step.


Raymond

From greg at cosc.canterbury.ac.nz  Fri Jul  9 03:17:02 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Jul  9 03:17:08 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <16621.22620.304068.926781@montanaro.dyndns.org>
Message-ID: <200407090117.i691H2ET020840@cosc353.cosc.canterbury.ac.nz>

Skip Montanaro <skip@pobox.com>:

> When I looked at this a long time ago it I seem to recall that roughly 60%
> of the opcodes executed did nothing more than copy values to or from the
> stack (of course, pushes and pops are some of the more efficient opcodes in
> the instruction set).  A three-address machine model would reduce this data
> movement substantially.

But it would only be worth doing if the cost of these data
movements is a substantial fraction of the whole execution
time. Does anyone have any timings to suggest that they
are?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From goodger at python.org  Fri Jul  9 03:35:19 2004
From: goodger at python.org (David Goodger)
Date: Fri Jul  9 03:35:28 2004
Subject: [Python-Dev] decimal.py signals & traps
In-Reply-To: <000f01c464e0$261b6d00$e841fea9@oemcomputer>
References: <000f01c464e0$261b6d00$e841fea9@oemcomputer>
Message-ID: <40EDF657.8030701@python.org>

Raymond Hettinger wrote:
 > The most uncomfortable part of the Python interface to contexts is
 > creating the dictionaries for traps and flags arguments.  Besides
 > being a pain to create and display, there are issues with two
 > contexts sharing the same underlying dictionaries.  Also, it makes
 > it difficult to address the issue at hand (as noted by the one XXX
 > comment in the module and by your bug report).

Here's the comment:

# XXX Is there a logic error in subclassing InvalidOperation?  Setting
# the InvalidOperation trap to zero does not preclude
# ConversionSyntax.  Also, incrementing Conversion syntax flag will
# not increment InvalidOperation.  Both of these issues interfere with
# cross-language portability because code following the spec would not
# know about the Python subclasses.

There's no logic error in subclassing InvalidOperation.  The error is
in treating the subclasses as signals: they're not.  The solution is
simple: don't extend the spec by making the ConversionSyntax
etc. exceptional conditions into signals.

 > As a first step, I would like change the Context constructor API to
 > match the format in Context.__repr__.  That format is somewhat
 > easier to use and less verbose (list only the flags and traps that
 > you want set).

Yes, Context.__repr__ is easier:

 >>> decimal.DefaultContext
Context(prec=28, rounding=ROUND_HALF_EVEN, Emin=-999999999,
Emax=999999999, setflags=[], settraps=['DivisionByZero', 'Overflow',
'InvalidOperation'])

But I'd go even further.

* "setflags" and "settraps" are awkward.  How about "flags" and
   "traps" instead?

* Rather than list traps by class name (string), why not use the
   signal classes as constants?  I.e.

       settraps=[DivisionByZero, Overflow, InvalidOperation]

   This is analogous to using the ROUND_HALF_EVEN constant in the repr.

 > Also, I would add accessor methods patterned after the Language
 > Independent Arithmetic standard (LIA-1) to set, clear, and test
 > flags and traps.

Could you provide examples?  Or a link?  I couldn't find the text of
the standard on the web.

 > The dictionaries themselves would be made private (allowing them to
 > be recast as sets if desired).

Sure; implementation detail.

 > With a method interface in place, a next step would be to add logic
 > to maintain relationships between signals.  Setting the
 > ConversionSyntax flag also sets the InvalidOperation flag.  Likewise
 > setting a trap for InvalidOperation would set the trap for
 > ConversionSyntax.

There is no need for this.

The point of this discussion and the patch is that according to the
spec and PEP 327, there *is no* "conversion syntax" signal.
"Conversion syntax" is an exceptional *condition*, which triggers the
"invalid-operation" *signal*.  Same for "division impossible",
"division undefined", and "invalid context"; all of these are
conditions tied to the "invalid-operation" signal.  Making
ConversionSyntax etc. into signals would be extending the spec.

 > I'm less enthusiastic about changing any of the exception names but
 > none of the above precludes that as a last step.

No name changes are necessary IMO.

-- 
David Goodger <http://python.net/~goodger>
From fdrake at acm.org  Fri Jul  9 07:07:19 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri Jul  9 07:08:11 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <200407090117.i691H2ET020840@cosc353.cosc.canterbury.ac.nz>
References: <200407090117.i691H2ET020840@cosc353.cosc.canterbury.ac.nz>
Message-ID: <200407090107.19123.fdrake@acm.org>

On Thursday 08 July 2004 09:17 pm, Greg Ewing wrote:
 > But it would only be worth doing if the cost of these data
 > movements is a substantial fraction of the whole execution
 > time. Does anyone have any timings to suggest that they
 > are?

I'd expect the cost of the data movement to be swamped by overhead of running 
through the decode/execute.  The question is whether increasing the decoding 
cost a bit is cheaper than the additional trips through the loop.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>

From anthony at python.org  Fri Jul  9 09:11:34 2004
From: anthony at python.org (Anthony Baxter)
Date: Fri Jul  9 09:11:52 2004
Subject: [Python-Dev] RELEASED Python 2.4, alpha 1
Message-ID: <40EE4526.5060306@python.org>

On behalf of the Python development team and the Python community, I'm
happy to announce the first alpha of Python 2.4.

Python 2.4a1 is an alpha release. We'd greatly appreciate it if you
could download it, kick the tires and let us know of any problems you
find, but it is not suitable for production usage.

     http://www.python.org/2.4/

In this release we have a number of new modules, a number of existing
modules that have been reimplemented in C for speed, a large number of
improvements and additions to existing modules and an even larger list
of bugs squished. See either the highlights, the What's New in Python
2.4, or the detailed NEWS file -- all available from the Python 2.4
webpage.

There will be at least one more alpha release in a couple of weeks to
pick up a few new features that didn't make it into the first alpha,
before we release 2.4 betas and then the final release.

Please log any problems you have with this release in the SourceForge
bug tracker (noting that you're using 2.4a1):

     http://sourceforge.net/bugs/?group_id=5470

Enjoy the new release,
Anthony

Anthony Baxter
anthony@python.org
Python Release Manager
(on behalf of the entire python-dev team)
From ronaldoussoren at mac.com  Fri Jul  9 09:23:48 2004
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Fri Jul  9 09:24:01 2004
Subject: [Python-Dev] New test failure, test_socket, WinXP
In-Reply-To: <20040708232226.M27271@bullseye.apana.org.au>
References: <20040706040145.45DA21E4005@bag.python.org>
	<40EA3205.4060804@ocf.berkeley.edu>
	<1089132447.8521.409.camel@localhost>
	<20040708232226.M27271@bullseye.apana.org.au>
Message-ID: <EB5D5B18-D178-11D8-A211-0003931CFE24@mac.com>


On 8-jul-04, at 15:33, Andrew MacIntyre wrote:

> On Tue, 6 Jul 2004, Barry Warsaw wrote:
>
>> On Tue, 2004-07-06 at 01:00, Brett C. wrote:
>>
>>> Personally I am +1 on changing 'www' to 'http' in the test and moving
>>> on.
>>
>> That won't work on Linux, apparently.  Okay, I looked at the Windows
>> file that Brett mentioned and it looks like checking 'echo',  
>> 'daytime',
>> and 'domain' should do the trick.  I'll commit the change.
>
> Having 'echo' first breaks the test on FreeBSD, at the assertion in  
> line
> 302 of test_socket, because FreeBSD's services file includes records  
> for
> protocol "ddp" (AppleTalk over TCP/IP??), including an "echo" on port 4
> rather than the expected port 7 for the "tcp" & "udp" protocols.

I haven't looked at the code, but wouldn't this be a design problem?  
I'd expect that you'd have to supply a protocol name in the call to  
getservbyport, service names tend to be the same for UDP and TCP but  
there is no guarantee that they will always be the same.

Ronald

>
> Would having the list as 'daytime', 'echo', 'domain' cause any other
> problems?
>
> --
> Andrew I MacIntyre                     "These thoughts are mine  
> alone..."
> E-mail: andymac@bullseye.apana.org.au  (pref) | Snail: PO Box 370
>         andymac@pcug.org.au             (alt) |        Belconnen  ACT   
> 2616
> Web:    http://www.andymac.org/               |        Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:  
> http://mail.python.org/mailman/options/python-dev/ 
> ronaldoussoren%40mac.com
>
>
--
X|support bv            http://www.xsupport.nl/
T:  +31 610271479       F:  +31 204416173

From anthony at interlink.com.au  Fri Jul  9 09:43:58 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Jul  9 09:44:07 2004
Subject: [Python-Dev] collecting 2.4 highlights
Message-ID: <40EE4CBE.3030309@interlink.com.au>

There's a first go at a 2.4 'highlights' page on the website now.

I'm absolutely positive that I missed some - I was working from the
whatsnew document, but that's missing a lot. I plan to go through
the Misc/NEWS document between now and a2 and add more, but if
there's something you feel should be there, feel free to either
add it (if you have pydotorg checkin privs) or else email me...

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From mcherm at mcherm.com  Fri Jul  9 13:55:14 2004
From: mcherm at mcherm.com (mcherm@mcherm.com)
Date: Fri Jul  9 13:55:20 2004
Subject: [Python-Dev] decimal.py signals & traps
Message-ID: <1089374114.40ee87a2f1b7c@login.werra.lunarpages.com>

Raymond writes:
> I would like change the Context constructor API to [...]
> (list only the flags and traps that you want set).

+1

> The dictionaries themselves would be made private (allowing them to
> be recast as sets if desired).

+1

David writes:
> The point of this discussion and the patch is that according to the
> spec and PEP 327, there *is no* "conversion syntax" signal.
> "Conversion syntax" is an exceptional *condition*, which triggers the
> "invalid-operation" *signal*.  Same for "division impossible",
> "division undefined", and "invalid context"; all of these are
> conditions tied to the "invalid-operation" signal.  Making
> ConversionSyntax etc. into signals would be extending the spec.

+1

> No name changes are necessary IMO.

+2

-- Michael Chermside

From skip at pobox.com  Fri Jul  9 16:23:36 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Jul  9 16:22:55 2004
Subject: [Python-Dev] Possible problem with recent weakref or bsddb changes
Message-ID: <16622.43624.320004.511670@montanaro.dyndns.org>


Greg checked in some changes to bsddb on June 28.  Fred checked in some
changes to weakref on July 2.  One or both I think conspired to break
Spambayes for me.  I filed a bug report a couple days ago
<http://python.org/sf/986528> and originally assigned it to Greg.  I just
switched it to Fred since his changes were more recent and seemed more wide
ranging.  I just reversed those changes in my sandbox and am building now to
see if it solves my problem.  Might be something to pay close attention to
for the impending alpha release.  I know next to nothing about the bsddb
package and absolutely nothing about weakrefs, so beyond trying crude things
like backing out changes I'm afraid I probably can't be much help debugging
the problem.

Skip
From theller at python.net  Fri Jul  9 19:37:33 2004
From: theller at python.net (Thomas Heller)
Date: Fri Jul  9 19:40:27 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <40ECD875.7010004@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Thu,
	08 Jul 2004 07:15:33 +0200")
References: <0c3d01c46164$f49e85b0$0200a8c0@eden>
	<40E7C737.6000702@v.loewis.de> <eknn95ig.fsf@python.net>
	<40ECD875.7010004@v.loewis.de>
Message-ID: <r7rl2iaq.fsf@python.net>

"Martin v. L?wis" <martin@v.loewis.de> writes:

> Thomas Heller wrote:
>> For example, the MSI which is currently built removes the file
>> associations when the package is removed, instead of restoring them to
>> their previous values.  I have no idea how this can be fixed...
>
> This is a multi-step process. First, one has to find out how
> to restore associations in general, i.e. independently of MSI.
> Where should one store, on install, the information what the old
> association was? What if there already is "old" extension
> information?
>
> This is general Win32 stuff, and I don't know what the answer is.

Wise gives an example.  It stores the previous value in it's INSTALL.LOG
file.  Whether this makes sense in general or not I cannot tell,
although it's convenient for me.

> Then, how can this be integrated into an MSI file? Perhaps
> MSI already offers built-in mechanism for that, perhaps existing
> mechanisms for dealing with the registry have to be used.

I thought MSI were able to 'restore the system to the state it had
before the installation', but maybe this is just a dream, may it isn't
supposed to do this, maybe one has to use system restore points for
that, and maybe this only shows how much I know about this stuff.

> This is MSI specific, and is likely answered in the Microsoft
> MSI documentation.

> Then, finally, how can the Python script be changed to invoke
> the MSI mechanism? This is very easy: If you know what tables
> in MSI to manipulate in what way, the relevant Python code
> should become easily apparent.
>
> So in this case, I don't think any amount of documentation on
> the Python script will help to solve this problem (that is
> not to say that documentation should be avoided for this reason,
> of course).

Well, the docs for msilib could tell that associations are not restored.

BTW: Python-2.4a1.msi misses to install the
distutils/command/wininst-6.exe file.  Should I fix this myself (i guess
I could figure *this one* out) or do you prefer a bug report?

Thomas

From barry at python.org  Fri Jul  9 20:40:40 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Jul  9 20:40:31 2004
Subject: [Python-Dev] New test failure, test_socket, WinXP
In-Reply-To: <20040708232226.M27271@bullseye.apana.org.au>
References: <20040706040145.45DA21E4005@bag.python.org>
	<40EA3205.4060804@ocf.berkeley.edu>
	<1089132447.8521.409.camel@localhost>
	<20040708232226.M27271@bullseye.apana.org.au>
Message-ID: <1089398440.8356.600.camel@localhost>

On Thu, 2004-07-08 at 09:33, Andrew MacIntyre wrote:

> Would having the list as 'daytime', 'echo', 'domain' cause any other
> problems?

Not for Linux.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040709/b10c3833/attachment.pgp
From niemeyer at conectiva.com  Fri Jul  9 21:03:05 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Fri Jul  9 21:02:08 2004
Subject: [Python-Dev] collecting 2.4 highlights
In-Reply-To: <40EE4CBE.3030309@interlink.com.au>
References: <40EE4CBE.3030309@interlink.com.au>
Message-ID: <20040709190305.GA6552@burma.localdomain>

Hello Anthony,

> There's a first go at a 2.4 'highlights' page on the website now.
> 
> I'm absolutely positive that I missed some - I was working from the
> whatsnew document, but that's missing a lot. I plan to go through
> the Misc/NEWS document between now and a2 and add more, but if
> there's something you feel should be there, feel free to either
> add it (if you have pydotorg checkin privs) or else email me...

You may want to mention that this is the first version which includes
the change in the SRE matcher to make it non-recursive.

-- 
Gustavo Niemeyer
http://niemeyer.net
From greg at electricrain.com  Fri Jul  9 21:13:43 2004
From: greg at electricrain.com (Gregory P. Smith)
Date: Fri Jul  9 21:13:57 2004
Subject: [Python-Dev] Possible problem with recent weakref or bsddb changes
In-Reply-To: <16622.43624.320004.511670@montanaro.dyndns.org>
References: <16622.43624.320004.511670@montanaro.dyndns.org>
Message-ID: <20040709191343.GL30677@zot.electricrain.com>

On Fri, Jul 09, 2004 at 09:23:36AM -0500, Skip Montanaro wrote:
> 
> Greg checked in some changes to bsddb on June 28.  Fred checked in some
> changes to weakref on July 2.  One or both I think conspired to break
> Spambayes for me.  I filed a bug report a couple days ago
> <http://python.org/sf/986528> and originally assigned it to Greg.  I just
> switched it to Fred since his changes were more recent and seemed more wide
> ranging.  I just reversed those changes in my sandbox and am building now to
> see if it solves my problem.  Might be something to pay close attention to
> for the impending alpha release.  I know next to nothing about the bsddb
> package and absolutely nothing about weakrefs, so beyond trying crude things
> like backing out changes I'm afraid I probably can't be much help debugging
> the problem.
> 
> Skip

Thanks for following up on this, I haven't had time to look to see if
anything I did might have broken it.  I suspect not but would love to
find out either way.

The commits on June 28 did add weakref support to the bsddb.db DB, DBEnv,
& DBTxn C objects (DBCursor has had weakref support for many months;
I just added it to the rest of the objects)

-g

From misa at redhat.com  Fri Jul  9 21:17:45 2004
From: misa at redhat.com (Mihai Ibanescu)
Date: Fri Jul  9 21:18:30 2004
Subject: [Python-Dev] bug in httplib.HTTPResponse.read() ?
Message-ID: <20040709191745.GB4289@umberto.devel.redhat.com>

Hello,

Can someone with more experience with httplib have a look at this?

https://sourceforge.net/tracker/index.php?func=detail&aid=988120&group_id=5470&atid=105470

Am I right that this is indeed a bug?

Thanks,
Misa
From martin at v.loewis.de  Fri Jul  9 22:28:28 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Jul  9 22:28:44 2004
Subject: [Python-Dev] VC 7.1 maintenance?
In-Reply-To: <r7rl2iaq.fsf@python.net>
References: <0c3d01c46164$f49e85b0$0200a8c0@eden>	<40E7C737.6000702@v.loewis.de>
	<eknn95ig.fsf@python.net>	<40ECD875.7010004@v.loewis.de>
	<r7rl2iaq.fsf@python.net>
Message-ID: <40EEFFEC.90209@v.loewis.de>

Thomas Heller wrote:
> Wise gives an example.  It stores the previous value in it's INSTALL.LOG
> file.  Whether this makes sense in general or not I cannot tell,
> although it's convenient for me.

This is unreliable, of course: the original software may get
uninstalled, in which case uninstall would restore an invalid
association.

Or, something might install on top of Python, and back up the Python
association in its own logfile. Then, uninstalling Python would break
the associations of that software.

I just learned in microsoft.public.platformsdk.msi that Windows does
not have a mechanism for that feature, and neither does MSI.

> I thought MSI were able to 'restore the system to the state it had
> before the installation', but maybe this is just a dream, may it isn't
> supposed to do this, maybe one has to use system restore points for
> that, and maybe this only shows how much I know about this stuff.

MSI does support rollbacks; invoking a rollback is supposed to fully
restore the system to its original state (although the MSI author
has to work to make sure it really does in all cases).

However, rollback only applies to the installation. If, in the course
of the installation, something fails (e.g. an essential file cannot
be written, because permissions are incorrect), installer will invoke
a rollback and restore all original registry keys. Once the installation
is committed, rollback is not possible anymore (and installer removes
the Cancel button from the UI after the commit until the end of the
installation).

> Well, the docs for msilib could tell that associations are not restored.

Actually, msilib is (currently) completely unaware of associations. The 
Python associations are generated in the fragment starting

     # File extensions, associated with the REGISTRY.def component
     # IDLE verbs depend on the tcltk feature.
     # msidbComponentAttributesRegistryKeyPath = 4
     # -1 for Root specifies "dependent on ALLUSERS property"
...
     add_data(db, "Registry",
             [# Extensions
              ("py.ext", -1, r"Software\Classes\."+ext, "",
               "Python.File", "REGISTRY.def"),

This is literally what it says it is: it adds data to the Registry
MSI table. See the documentation of the Registry table for details
of the fields.

Installer, in turn, literally does what the Registry table tells it to
do: it adds keys to the registry.

If you want to perform backup of the existing registry keys to a
file, you need a Custom Action.

> BTW: Python-2.4a1.msi misses to install the
> distutils/command/wininst-6.exe file.  Should I fix this myself (i guess
> I could figure *this one* out) or do you prefer a bug report?

Please go ahead and make any changes to the scripts you feel necessary.
If you are uncertain, make a patch and assign it to me. If you don't
know how to make a patch, or have no time to do so, submit a bug report.

Regards,
Martin

From orbitz at ezabel.com  Wed Jul  7 22:49:33 2004
From: orbitz at ezabel.com (orbitz)
Date: Sat Jul 10 01:28:48 2004
Subject: [Python-Dev] Re: file() or open()?
In-Reply-To: <cchmc7$vft$1@sea.gmane.org>
References: <200407062309.i66N9xv25599@guido.python.org><40EC134F.9050608@prescod.net><200407071551.i67FpmM27430@guido.python.org>	<20040707183033.GA30577@alcyon.progiciels-bpi.ca>
	<cchmc7$vft$1@sea.gmane.org>
Message-ID: <40EC61DD.50704@ezabel.com>

I considered more as the action being performed.  I'm opening something, 
in this case a file.  And now I have an object which has been opened, I 
can perform operations on it, and when I'm done I close it.
Terry Reedy wrote:

>"Fran?ois Pinard" <pinard@iro.umontreal.ca> wrote in message
>news:20040707183033.GA30577@alcyon.progiciels-bpi.ca...
>  
>
>>I perceived the introduction of `file()' as a nice cleanup in Python.
>>    
>>
>
>As a user, so did I.  I like the cosistency of using file along with int,
>tuple, list, dict, type, (and did I leave out something), and all user
>classes as constructors of instances of themselves.
>
>Terry J. Reedy
>
>
>
>_______________________________________________
>Python-Dev mailing list
>Python-Dev@python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: http://mail.python.org/mailman/options/python-dev/orbitz%40drorbitz.ath.cx
>
>  
>

From joseph.klepach at csfb.com  Thu Jul  8 19:36:42 2004
From: joseph.klepach at csfb.com (Klepach, Joseph)
Date: Sat Jul 10 01:28:50 2004
Subject: [Python-Dev] Python 2.4a1 use of c:\winnt directory
Message-ID: <58B84CC3DC75D949B676ACF9CE5B73A4209AE034@snyc19p32002.csfb.cs-group.com>



The python 2.4a1 release using the msi installer insists on copying
python24.dll to the C:\WINNT\system32 directory instead the of requested user
directory.

This did not occur in the past and is causing a problem because my company
locks the c:\winnt directory on all user pc's.

Will this functionality be part of the official release?

Regards,
joe

==============================================================================
This message is for the sole use of the intended recipient. If you received
this message in error please delete it and notify us. If this message was
misdirected, CSFB does not waive any confidentiality or privilege. CSFB
retains and monitors electronic communications sent through its network.
Instructions transmitted over this system are not binding on CSFB until they
are confirmed by us. Message transmission is not guaranteed to be secure.
==============================================================================

From mzarate at uoguelph.ca  Fri Jul  9 23:04:27 2004
From: mzarate at uoguelph.ca (Martin Zarate)
Date: Sat Jul 10 01:28:51 2004
Subject: [Python-Dev] Re: decorators and 2.4
Message-ID: <1089407067.40ef085b6db31@webmail.uoguelph.ca>

I'm not sure if I should be writing this, I apologise in advanced if I'm 
wasting anyone's time - I've read over all of the arguments on decorators 
(last time I posted on the subject I hadn't, and I apologise for that as well).

I know Guido wants no new concepts of syntax, but it seems that all the 
concepts have merit - so why not a little synergy?   Best of both worlds?

first complaint:
@ is currently used in many projects, and is an ugly character.

second complaint:
> [descriptors]
> def blah blah blah:
is currently valid Python, and looks like valid Python, so it will confuse 
people.

third complaint:
many suggestions are ungreppable.

So, a solution, mixing them all so that it becomes more than a simple list 
declaration:

as [descriptors]
def blah blah blah:

or, an alternate, simply using the "as" keyword in the place of Guido's Java-
style "@" character.

as descriptor1
as descriptor2
def blah blah blah:

Both of these are far more unique than other suggested blocks of code (like 
the [] header), allowing easier grepping and syntax highlighting, without 
having to go using the @ character that a) takes up one of the few unused 
Python special characters and b) breaks compatibility with some external 
processor programs.

IIRC, the "as" keyword became a protected keyword in recent versions, did it 
not?  So it doesn't break anything to convert into a full keyword.  It also 
makes syntactic sense, as you read "as staticmethod define myfunction" or "as 
functor(member1, member2, member3) define myfunction"  when you read aloud.  
It makes sense, keeping Python code legible and friendly.

In my personal tastes, I'd prefer the "as" keywords to go below the function 
header before the docstring, but I see that most others disagree.

Anyhow, my suggestion is a little more verbose, but I find it very legible and 
sensible (and given how Python is often used as a learning language, those 
features should be near the forefront).  I hope I haven't wasted anyone's time.

-- Martin Z.
From bob at redivi.com  Sat Jul 10 01:54:19 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sat Jul 10 01:54:28 2004
Subject: [Python-Dev] Re: file() or open()?
In-Reply-To: <40EC61DD.50704@ezabel.com>
References: <200407062309.i66N9xv25599@guido.python.org><40EC134F.9050608@prescod.net><200407071551.i67FpmM27430@guido.python.org>	<20040707183033.GA30577@alcyon.progiciels-bpi.ca>
	<cchmc7$vft$1@sea.gmane.org> <40EC61DD.50704@ezabel.com>
Message-ID: <4B3D15F2-D203-11D8-AF2C-000A95686CD8@redivi.com>

On Jul 7, 2004, at 4:49 PM, orbitz wrote:

> Terry Reedy wrote:
>
>> "Fran?ois Pinard" <pinard@iro.umontreal.ca> wrote in message
>> news:20040707183033.GA30577@alcyon.progiciels-bpi.ca...
>>
>>> I perceived the introduction of `file()' as a nice cleanup in Python.
>>>
>>
>> As a user, so did I.  I like the cosistency of using file along with 
>> int,
>> tuple, list, dict, type, (and did I leave out something), and all user
>> classes as constructors of instances of themselves.
> I considered more as the action being performed.  I'm opening 
> something, in this case a file.  And now I have an object which has 
> been opened, I can perform operations on it, and when I'm done I close 
> it.

But you pass a string to open.  Are you opening a string?  No.  I might 
agree if there was some annotation on that string that said "I am a 
native platform filesystem path", but there isn't.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040709/42c9911b/smime.bin
From orbitz at drorbitz.ath.cx  Sat Jul 10 04:56:10 2004
From: orbitz at drorbitz.ath.cx (orbitz)
Date: Sat Jul 10 04:56:57 2004
Subject: [Python-Dev] String literals escape sequences
Message-ID: <40EF5ACA.4080701@drorbitz.ath.cx>

The python language reference says:

" Unlike Standard C, all unrecognized escape sequences are left in the 
string unchanged, i.e., /the backslash is left in the string/. (This 
behavior is useful when debugging: if an escape sequence is mistyped, 
the resulting output is more easily recognized as broken.)"

It seems to me that a syntax error of some kind would be even more 
useful.  For instance if one is dealing with an extremely large output 
they might be able to miss a single missing \t or \n.

Another problem is some beginners seem to be very confused between print 
'\n' and print '\l'

Also, what happens when people perhaps want '\l' and instead of using 
'\\l' type '\l' by accident, and their output looks fine, but later down 
the road it is decided '\l' should be an escape sequence.

Is there a very good reason for this behavior?
From guido at python.org  Sat Jul 10 05:38:24 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Jul 10 05:38:32 2004
Subject: [Python-Dev] String literals escape sequences
In-Reply-To: Your message of "Fri, 09 Jul 2004 22:56:10 EDT."
	<40EF5ACA.4080701@drorbitz.ath.cx> 
References: <40EF5ACA.4080701@drorbitz.ath.cx> 
Message-ID: <200407100338.i6A3cOe32060@guido.python.org>

> The python language reference says:
> 
> " Unlike Standard C, all unrecognized escape sequences are left in the 
> string unchanged, i.e., /the backslash is left in the string/. (This 
> behavior is useful when debugging: if an escape sequence is mistyped, 
> the resulting output is more easily recognized as broken.)"
> 
> It seems to me that a syntax error of some kind would be even more 
> useful.  For instance if one is dealing with an extremely large output 
> they might be able to miss a single missing \t or \n.

Well, maybe then they didn't care either way. :-)

> Another problem is some beginners seem to be very confused between print 
> '\n' and print '\l'

I've never heard of this.  Where would \l come from?

> Also, what happens when people perhaps want '\l' and instead of using 
> '\\l' type '\l' by accident, and their output looks fine, but later down 
> the road it is decided '\l' should be an escape sequence.
> 
> Is there a very good reason for this behavior?

The paragraph in the reference manual you give presents the original
reason.  Right now, changing this would break too much code to
consider it.  We might consider deprecating this use in 2.4, adding
warnings in 2.5, and making it illegal in 3.0.  But I'm not sure it's
worth the bother.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From kbk at shore.net  Sat Jul 10 06:43:29 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Sat Jul 10 06:43:34 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200407100443.i6A4hTUJ027044@hydra.localdomain>


Patch / Bug Summary
___________________

Patches :  294 open ( +8) /  2434 closed (+12) /  2728 total (+20)
Bugs    :  778 open ( +7) /  4197 closed (+29) /  4975 total (+36)
RFE     :  144 open ( +2) /   129 closed ( +0) /   273 total ( +2)

New / Reopened Patches
______________________

crach link c++ extension by mingw  (2004-06-29)
       http://python.org/sf/981773  opened by  Alexandr Zamaraev 

webbrower support for firebird / firefox  (2004-06-29)
       http://python.org/sf/981794  opened by  Thenault Sylvain 

applesingle endianness issue  (2004-06-29)
       http://python.org/sf/982340  opened by  Bob Ippolito 

add SO_EXCLUSIVEADDRUSE constant in socketmodule.c  (2004-06-30)
       http://python.org/sf/982665  opened by  Garth Bushell 

asyncore.dispatcher.set_reuse_addr() is broken on windows  (2004-06-30)
       http://python.org/sf/982681  opened by  Garth Bushell 

Making weakref.ref subclassable  (2004-06-30)
CLOSED http://python.org/sf/983019  opened by  Fred L. Drake, Jr. 

MSVC6/7 issues with bdist_wininst and --target-version  (2004-07-01)
       http://python.org/sf/983164  opened by  Mark Hammond 

Allow ldshared to be overridden by environment var  (2004-07-01)
       http://python.org/sf/983206  opened by  Liam Routt 

Allow bdist_wininst to install for non-admin users  (2004-07-02)
CLOSED http://python.org/sf/983775  opened by  Mark Hammond 

Address bug 980938, add set_debug_output() function  (2004-07-02)
       http://python.org/sf/984492  opened by  James Lamanna 

Fix bug 982806 - allow for optional 2nd argument  (2004-07-02)
CLOSED http://python.org/sf/984495  opened by  James Lamanna 

socket module does not support AF_NETLINK  (2004-07-03)
       http://python.org/sf/984654  opened by  Matthias Klose 

Fix bug 982806 - allow for optional 2nd argument  (2004-07-03)
CLOSED http://python.org/sf/984672  opened by  James Lamanna 

fix regression on 2.3 branch: Lib/threading  (2004-07-04)
CLOSED http://python.org/sf/984880  opened by  Matthias Klose 

bug skipping optional keyword arguments of type "w#"  (2004-07-05)
       http://python.org/sf/985713  opened by  Mark D. Roth 

add missing doc for datetime C API  (2004-07-06)
       http://python.org/sf/986010  opened by  Anthony Tuininga 

Is there a nested loop limit?  (2004-07-07)
CLOSED http://python.org/sf/986720  opened by  chimeda 

Add support for sync and use "wish" options  (2004-07-07)
       http://python.org/sf/986929  opened by  Greg Couch 

cgitb: escape title of page  (2004-07-08)
       http://python.org/sf/987052  opened by  Thomas Guettler 

decimal.py signal/exception fix  (2004-07-08)
CLOSED http://python.org/sf/987300  opened by  David Goodger 

Patches Closed
______________

Parser/acceler.c: PyObject_MALLOC return is not checked  (2004-06-17)
       http://python.org/sf/974633  closed by  akuchling

Patch for feature request 491033  (2004-05-20)
       http://python.org/sf/957240  closed by  mwh

unblock signals in threads  (2004-05-25)
       http://python.org/sf/960406  closed by  mwh

Making weakref.ref subclassable  (2004-06-30)
       http://python.org/sf/983019  closed by  fdrake

Fix for 754449: hiding exc in threading during interp shutdn  (2004-05-16)
       http://python.org/sf/954922  closed by  bcannon

Allow bdist_wininst to install for non-admin users  (2004-07-02)
       http://python.org/sf/983775  closed by  mhammond

Fix bug 982806 - allow for optional 2nd argument  (2004-07-03)
       http://python.org/sf/984495  closed by  akuchling

Fix bug 982806 - allow for optional 2nd argument  (2004-07-03)
       http://python.org/sf/984672  closed by  akuchling

fix regression on 2.3 branch: Lib/threading  (2004-07-04)
       http://python.org/sf/984880  closed by  bcannon

partially initialized heap allocated type objects  (2004-06-26)
       http://python.org/sf/980475  closed by  ehuss

Is there a nested loop limit?  (2004-07-07)
       http://python.org/sf/986720  closed by  mwh

decimal.py signal/exception fix  (2004-07-08)
       http://python.org/sf/987300  closed by  goodger

New / Reopened Bugs
___________________

Missing space in sec 3.3.1  (2004-06-27)
CLOSED http://python.org/sf/980986  opened by  Christopher Ingram 

Rsync protocol missing from urlparse  (2004-06-28)
CLOSED http://python.org/sf/981299  opened by  Morten Kjeldgaard 

UnboundLocalError in shutil.rmtree()  (2004-06-28)
CLOSED http://python.org/sf/981530  opened by  Guido van Rossum 

PyObject_Str() and PyObject_Repr() corrupt object type  (2004-06-29)
CLOSED http://python.org/sf/982008  opened by  Kevin Gadd 

Relative Path causing crash in RotatingFileHandler  (2004-06-29)
       http://python.org/sf/982049  opened by  David London 

 Next button not greyed out during file copy.  (2004-06-29)
CLOSED http://python.org/sf/982215  opened by  Phil Rittenhouse 

Bad state of multi btree database file after large inserts  (2004-06-30)
       http://python.org/sf/982679  opened by  Elerion Elrond 

gdbm.open () fails with a single argument  (2004-06-30)
CLOSED http://python.org/sf/982806  opened by  Roy Smith 

incorrect len  (2004-07-01)
CLOSED http://python.org/sf/983269  opened by  calvinlow 

simple attribute access causing segfault in certain cases  (2004-07-01)
       http://python.org/sf/983311  opened by  Simon Michael 

test_repr() fails with id() values that appear negative  (2004-07-01)
CLOSED http://python.org/sf/983585  opened by  Fred L. Drake, Jr. 

Buffer Protocol Prototypes Incorrect  (2004-07-02)
CLOSED http://python.org/sf/984017  opened by  Timothy Stranex 

hotspot.stats.load is very slow  (2004-07-02)
       http://python.org/sf/984219  opened by  Viktor Ferenczi 

Useless question in bdist_winst installers  (2004-07-02)
CLOSED http://python.org/sf/984290  opened by  Thomas Heller 

Tiny suggestion to improve optparse doc  (2004-07-03)
       http://python.org/sf/984691  opened by  Laura Creighton 

unknown parsing error  (2004-07-03)
       http://python.org/sf/984714  opened by  Igor Sidorenkov 

Py_BuildValue loses reference counts on error  (2004-07-03)
       http://python.org/sf/984722  opened by  Pete Shinners 

curses: getmaxyx() breaks when the window shrinks  (2004-07-04)
       http://python.org/sf/984870  opened by  Matthias Klose 

PEP 307 pickle extensions not documented  (2004-07-04)
       http://python.org/sf/984952  opened by  Greg Chapman 

plistlib crashes too easily on bad files  (2004-07-04)
       http://python.org/sf/985064  opened by  Jack Jansen 

getattr(object,name) accepts only strings  (2004-07-05)
       http://python.org/sf/985094  opened by  Viktor Ferenczi 

HAVE_FDATASYNC incorrectly set for 2.3.4 under cygwin  (2004-07-04)
       http://python.org/sf/985154  opened by  Gregory Smith 

bdist_rpm license documented, but not accepted as valid  (2004-07-05)
       http://python.org/sf/985478  opened by  Charles 

BaseHTTPServer to take host from cmdline?  (2004-07-05)
       http://python.org/sf/985651  opened by  Tom Lynn 

valgrind/memcheck alert: Python 2.3.4 Objects/obmalloc.c  (2004-07-05)
CLOSED http://python.org/sf/985708  opened by  Ralf W. Grosse-Kunstleve 

weakref should have a WeakSet  (2004-07-06)
       http://python.org/sf/985753  opened by  Leonardo Rochael Almeida 

bsddb/weakref error  (2004-07-07)
       http://python.org/sf/986528  opened by  Skip Montanaro 

New mandatory 2nd argument to site.addsitedir breaks compat  (2004-07-07)
       http://python.org/sf/986795  opened by  Jp Calderone 

Configure bugs  (2004-07-07)
CLOSED http://python.org/sf/986884  opened by  Dr. Tom 

TimedRotatingFileHandler problems  (2004-07-08)
       http://python.org/sf/987166  opened by  Amaury Forgeot d'Arc 

Python 2.4a1, interpreter hanging on Keyboard Interrupt  (2004-07-08)
CLOSED http://python.org/sf/987287  opened by  Simon Percivall 

Typo in cgi doc  (2004-07-08)
       http://python.org/sf/987486  opened by  Cory Dodt 

pep 311 functions missing documentation  (2004-07-09)
       http://python.org/sf/987835  opened by  Simon Dahlbacka 

Windows installer for 2.4a1 shows non-standard version  (2004-07-09)
       http://python.org/sf/987870  opened by  Paul Moore 

bdist_wininst improvements  (2004-07-09)
       http://python.org/sf/988022  opened by  Thomas Heller 

httplib.HTTPResponse.read() not updating the length correctl  (2004-07-09)
       http://python.org/sf/988120  opened by  Mihai Ibanescu 

Compiler warnings for Modules/readline.c:flex_complete()  (2004-07-09)
       http://python.org/sf/988300  opened by  Brett Cannon 

Bugs Closed
___________

Missing space in sec 3.3.1  (2004-06-27)
       http://python.org/sf/980986  closed by  bcannon

Possible contradiction in "extending" and PyArg_ParseTuple()  (2004-06-27)
       http://python.org/sf/980925  closed by  bcannon

Rsync protocol missing from urlparse  (2004-06-28)
       http://python.org/sf/981299  closed by  bcannon

UnboundLocalError in shutil.rmtree()  (2004-06-28)
       http://python.org/sf/981530  closed by  gvanrossum

Crash when reading "import table" of certain windows DLLs  (2004-05-11)
       http://python.org/sf/951851  closed by  theller

Over restricted type checking on eval() function  (2000-09-22)
       http://python.org/sf/215126  closed by  rhettinger

2.3a2 import after os.chdir difference  (2003-02-25)
       http://python.org/sf/693416  closed by  akuchling

urllib2 checks for http return code 200 only.  (2004-03-09)
       http://python.org/sf/912845  closed by  akuchling

Broken URLs in sha module docs  (2004-06-23)
       http://python.org/sf/978556  closed by  akuchling

PyExc_* not in index  (2004-05-06)
       http://python.org/sf/948970  closed by  akuchling

PyObject_Str() and PyObject_Repr() corrupt object type  (2004-06-29)
       http://python.org/sf/982008  closed by  janusfury

 Next button not greyed out during file copy.  (2004-06-29)
       http://python.org/sf/982215  closed by  theller

gdbm.open () fails with a single argument  (2004-06-30)
       http://python.org/sf/982806  closed by  akuchling

incorrect len  (2004-07-01)
       http://python.org/sf/983269  closed by  effbot

A leak case with cmd.py &amp; readline &amp; exception  (2004-03-27)
       http://python.org/sf/924301  closed by  mwh

test_repr() fails with id() values that appear negative  (2004-07-01)
       http://python.org/sf/983585  closed by  fdrake

bdist_winist post-install script fails on non-admin Python  (2004-04-15)
       http://python.org/sf/935091  closed by  mhammond

Buffer Protocol Prototypes Incorrect  (2004-07-02)
       http://python.org/sf/984017  closed by  akuchling

Useless question in bdist_winst installers  (2004-07-02)
       http://python.org/sf/984290  closed by  theller

removing extensions without admin rights  (2002-05-14)
       http://python.org/sf/555810  closed by  mhammond

installing extension w/o admin rights  (2002-05-14)
       http://python.org/sf/555812  closed by  mhammond

Exceptions when a thread exits  (2003-06-14)
       http://python.org/sf/754449  closed by  bcannon

valgrind/memcheck alert: Python 2.3.4 Objects/obmalloc.c  (2004-07-06)
       http://python.org/sf/985708  closed by  tim_one

httplib:  HTTPS does not close() connection properly  (2004-03-10)
       http://python.org/sf/913619  closed by  akuchling

_Subfile.readline( ) in mailbox.py ignoring self.stop  (2004-03-28)
       http://python.org/sf/925107  closed by  akuchling

Configure bugs  (2004-07-07)
       http://python.org/sf/986884  closed by  bcannon

Spurious errors taking bool of dead proxy  (2004-06-23)
       http://python.org/sf/978308  closed by  nnorwitz

Funny keywords in library index  (2004-05-12)
       http://python.org/sf/952737  closed by  fdrake

Python 2.4a1, interpreter hanging on Keyboard Interrupt  (2004-07-08)
       http://python.org/sf/987287  closed by  mwh

New / Reopened RFE
__________________

md5 and sha1 modules should use openssl implementation  (2004-06-30)
       http://python.org/sf/983069  opened by  Gregory P. Smith 

Support "hybrid" framework/unix style  (2004-07-02)
       http://python.org/sf/984242  opened by  Bob Ippolito 

From tim.peters at gmail.com  Sat Jul 10 06:44:40 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Jul 10 06:44:42 2004
Subject: [Python-Dev] Python 2.4a1 use of c:\winnt directory
In-Reply-To: <58B84CC3DC75D949B676ACF9CE5B73A4209AE034@snyc19p32002.csfb.cs-group.com>
References: <58B84CC3DC75D949B676ACF9CE5B73A4209AE034@snyc19p32002.csfb.cs-group.com>
Message-ID: <1f7befae04070921443f533dfe@mail.gmail.com>

[Klepach, Joseph]
> The python 2.4a1 release using the msi installer insists on copying
> python24.dll to the C:\WINNT\system32 directory instead the of requested user
> directory.
>
> This did not occur in the past and is causing a problem because my company
> locks the c:\winnt directory on all user pc's.
>
> Will this functionality be part of the official release?

That depends on whether someone volunteers to do the work.

When I took over the Python Windows installers, they always copied the
DLL into a system directory.  I added the "non-admin install" option,
and a bunch of other things, because they were the right things to do.
 My mgmt chain at the time just whined at me, for spending more time
on it than they thought was necessary.

The two things to take from that are:  (1) it takes real work to
arrange for these nice touches; and, (2) only a volunteer can get away
with spending the time to do them -- unless Python users (who
understand the need for them acutely) are willing to fund the work.

Remember all that when Martin replies <wink> (Martin created the new
MSI-based installer).
From tjreedy at udel.edu  Sat Jul 10 07:05:59 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat Jul 10 07:06:09 2004
Subject: [Python-Dev] Re: Re: file() or open()?
References: <200407062309.i66N9xv25599@guido.python.org><40EC134F.9050608@prescod.net><200407071551.i67FpmM27430@guido.python.org>	<20040707183033.GA30577@alcyon.progiciels-bpi.ca><cchmc7$vft$1@sea.gmane.org>
	<40EC61DD.50704@ezabel.com>
Message-ID: <ccntfs$k9g$1@sea.gmane.org>


"orbitz" <orbitz@ezabel.com> wrote in message
news:40EC61DD.50704@ezabel.com...
Terry Reedy wrote:

>>François Pinard" <pinard@iro.umontreal.ca> wrote in message
>>> perceived the introduction of `file()' as a nice cleanup in Python.

>> As a user, so did I.  I like the cosistency of using file along with
int,
>> tuple, list, dict, type, (and did I leave out something), and all user
>> classes as constructors of instances of themselves.

[top post moved down before responding to give better context]

>I considered more as the action being performed.  I'm opening something,

Are you, really?  I understand opening a door, gate, hinged lid, file
folder, gun stock,  clam, fist, or book.  All of these involve a hinge
motion, usually reversible.  Opening a drawer uses a reversible sliding
motion.  Opening a can, walnut, or geode involves an irreversible cut or
smash.  I do not see much analogy of any of these motions with the action
of open/type().

If you say that they all give access, I could just as well say that you are
unlocking, leasing, or learning something or acquiring a permission.

In my view, file/open() (in read mode) asks the interpreter to construct a
Python object that corresponds to and connects and gives access to the
external non-PyObject data (byte) stream indicated by the name/locator
string so that I may bring part or all of that data into the Python
dataspace as one or more Python objects (initially strings).  The
host-system specifics of the access mechanism is beyond the purview of
Python itself.

So, if you ignore the constructor analogy and want an action name, I think
proxy, connect, access, lease, or even setup might be better.  In short,
the action is

py_data_object = proxy(non_pyob_data_access_info)

Should socket.socket() have been socket.open because it 'opens' a
communication channel (in the process of creating a socket object)?  For
me, this analogy works at least as well as 'opening' a document (see
below).

>in this case a file.

Is it, really?  If one thinks in terms of the offices and desks, a file is
a container that keeps a series of related cards, papers, or documents in
order.  In this context, a series of bytes, usually represents and is
better called a paper or document.  People write documents and put them
into files.  But when documents are called 'files, files have to be called
something else, like 'directory' or 'folder' .

If by file one means a sequence or string of bytes in analogy with a file
of people, (also from the Latin filum == thread, as in filament), then
opening the file/sequence would seem to mean opening one or more spaces in
the sequence.  In chess and war, opening a file has the similar meaning of
removing enemy occupants from a series of spaces.

> And now I have an object which has been opened, I
> can perform operations on it, and when I'm done I close it.

I use the proxy to access non-Python data and then release the proxy.
What, if anything, the host system does upon release is outside the purview
of Python.

-----
In a previous post, Guido justified 'open' on the basis that applications
'open' 'files'.  However, in every such application I can think of at the
moment, 'open' means 'read the data and convert it to one of the
application's internal native formats, even if it was written in a
different format by another application.  In other words, app.open(path) ==
file(path).read().convert(filetype(path)).   Conversely, one never opens a
file before saveing a fresh new document; one just saves-as.  So Python's
open() is quite different from the app usage of 'open'.

Terry J. Reedy



From martin at v.loewis.de  Sat Jul 10 13:28:56 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Jul 10 13:28:59 2004
Subject: [Python-Dev] Python 2.4a1 use of c:\winnt directory
In-Reply-To: <58B84CC3DC75D949B676ACF9CE5B73A4209AE034@snyc19p32002.csfb.cs-group.com>
References: <58B84CC3DC75D949B676ACF9CE5B73A4209AE034@snyc19p32002.csfb.cs-group.com>
Message-ID: <40EFD2F8.9090707@v.loewis.de>

Klepach, Joseph wrote:
> The python 2.4a1 release using the msi installer insists on copying
> python24.dll to the C:\WINNT\system32 directory instead the of requested user
> directory.
> 
> This did not occur in the past and is causing a problem because my company
> locks the c:\winnt directory on all user pc's.

That is not true. Python has always copied pythonxy.dll into the Windows
directory if a "privileged" installation (previously called 
"administrator" installation) was performed.

> Will this functionality be part of the official release?

Yes. Chose a per-user installation if you don't want to install Python
into the system directory.

Regards,
Martin

From amk at amk.ca  Sat Jul 10 19:27:20 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Sat Jul 10 19:27:59 2004
Subject: [Python-Dev] Bug day reminder
Message-ID: <20040710172720.GA22802@rogue.amk.ca>

The bug day is currently going on in #python-dev on irc.freenode.net.

--amk
From bac at OCF.Berkeley.EDU  Sat Jul 10 22:23:35 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sat Jul 10 22:23:48 2004
Subject: [Python-Dev] urllib.urlretrieve() and handling 550 errors when
	reading from FTP
Message-ID: <40F05047.2030808@ocf.berkeley.edu>

Bug #635453 reports that when you use urllib.urlretrieve() to read from 
an FTP server and the file you are trying to get does not have the 
proper permissions for you, you are given a listing of the directory 
instead of an exception (this is all done by Lib/urllib.py:752).

Now this doesn't make sense to me and I suspect most people would agree. 
  But having that explicit line there on top of it having been in the 
file since it was renamed from urlopen way backin 1994 makes me hesitate 
before I delete the line.

Anyone have issues if I ditch the line so an exception is raised instead 
of getting a directory listing instead?

-Brett
From nospam at barrys-emacs.org  Sun Jul 11 02:09:51 2004
From: nospam at barrys-emacs.org (nospam@barrys-emacs.org)
Date: Sun Jul 11 02:10:16 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <200407090107.19123.fdrake@acm.org>
References: <200407090117.i691H2ET020840@cosc353.cosc.canterbury.ac.nz>
	<200407090107.19123.fdrake@acm.org>
Message-ID: <A0FE3DC8-D2CE-11D8-B889-000A95A8705A@barrys-emacs.org>

If you find sequences of byte-code that are worth optimizing that is 
possible
without changing the  byte code itself. Modern CPU's use pipe lines to 
look at
the instruction stream and figure out fast ways to run the code. The 
same method
could be used in pythons VM.

As others have pointed out its up to someone to prove that the added 
complexity in
the VM gives a desirable improvement in performance.

Barry

From guido at python.org  Sun Jul 11 02:20:13 2004
From: guido at python.org (Guido van Rossum)
Date: Sun Jul 11 02:19:53 2004
Subject: [Python-Dev] urllib.urlretrieve() and handling 550 errors when
	reading from FTP
In-Reply-To: Your message of "Sat, 10 Jul 2004 13:23:35 PDT."
	<40F05047.2030808@ocf.berkeley.edu> 
References: <40F05047.2030808@ocf.berkeley.edu> 
Message-ID: <200407110020.i6B0KDN01623@guido.python.org>

> Bug #635453 reports that when you use urllib.urlretrieve() to read from 
> an FTP server and the file you are trying to get does not have the 
> proper permissions for you, you are given a listing of the directory 
> instead of an exception (this is all done by Lib/urllib.py:752).
> 
> Now this doesn't make sense to me and I suspect most people would agree. 
>   But having that explicit line there on top of it having been in the 
> file since it was renamed from urlopen way backin 1994 makes me hesitate 
> before I delete the line.
> 
> Anyone have issues if I ditch the line so an exception is raised instead 
> of getting a directory listing instead?

If you remove this, you won't be able to get a directory listing of
directories.  I tried this with and without the line; with that line
in, it gives a directlry listing, without it gives an IOError (Not a
regular file):

>>> urllib.urlretrieve("ftp://ftp.python.org/pub", "xyzzy")

So I'd be against deleting this, unless there's a different way to get
directory listings.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tim.peters at gmail.com  Sun Jul 11 03:25:46 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sun Jul 11 03:25:49 2004
Subject: [Python-Dev] Python 2.4a1 use of c:\winnt directory
In-Reply-To: <40EFD2F8.9090707@v.loewis.de>
References: <58B84CC3DC75D949B676ACF9CE5B73A4209AE034@snyc19p32002.csfb.cs-group.com>
	<40EFD2F8.9090707@v.loewis.de>
Message-ID: <1f7befae04071018251c817ce8@mail.gmail.com>

[Klepach, Joseph]
>> The python 2.4a1 release using the msi installer insists on copying
>> python24.dll to the C:\WINNT\system32 directory instead the of requested user
>> directory.
>>
>> This did not occur in the past and is causing a problem because my company
>> locks the c:\winnt directory on all user pc's.
...
>> Will this functionality be part of the official release?

[Martin v. L?wis]
> Yes. Chose a per-user installation if you don't want to install Python
> into the system directory.

Ah, OK, you're saying this is *already* available ... and it is. 
Joseph, when you run the .msi thingie, click the Advanced button in
the second screen.  Then there's a radio button controlling whether to
install for all users, or just the current user.  I selected the
latter, and python24.dll was installed under my Python24 directory
instead of under a system directory.
From bac at OCF.Berkeley.EDU  Sun Jul 11 04:04:12 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Jul 11 04:04:27 2004
Subject: [Python-Dev] urllib.urlretrieve() and handling 550 errors when
	reading from FTP
In-Reply-To: <200407110020.i6B0KDN01623@guido.python.org>
References: <40F05047.2030808@ocf.berkeley.edu>
	<200407110020.i6B0KDN01623@guido.python.org>
Message-ID: <40F0A01C.4040905@ocf.berkeley.edu>

Guido van Rossum wrote:
>>Bug #635453 reports that when you use urllib.urlretrieve() to read from 
>>an FTP server and the file you are trying to get does not have the 
>>proper permissions for you, you are given a listing of the directory 
>>instead of an exception (this is all done by Lib/urllib.py:752).
>>
>>Now this doesn't make sense to me and I suspect most people would agree. 
>>  But having that explicit line there on top of it having been in the 
>>file since it was renamed from urlopen way backin 1994 makes me hesitate 
>>before I delete the line.
>>
>>Anyone have issues if I ditch the line so an exception is raised instead 
>>of getting a directory listing instead?
> 
> 
> If you remove this, you won't be able to get a directory listing of
> directories.  I tried this with and without the line; with that line
> in, it gives a directlry listing, without it gives an IOError (Not a
> regular file):
> 
> 
>>>>urllib.urlretrieve("ftp://ftp.python.org/pub", "xyzzy")
> 
> 
> So I'd be against deleting this, unless there's a different way to get
> directory listings.
> 

Try it again but with a trailing slash on the address.  With the line 
removed it will fetch the directory.  Otherwise it errors out.  The code 
apparently tries to get the file and when it fails with a 550 (meaning 
the file cannot be accessed because of permissions or lack of 
existence), the code then send a LIST command which lists whatever the 
argument is, file or directory.

Unfortunately I can't think of a good way to detect the difference 
between a file and a directory that just doesn't have the ending slash 
beyond reading the output and seeing if it is a single listing for a 
file with the same name as the LIST request.

-Brett
From mcherm at mcherm.com  Sun Jul 11 05:06:12 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Sun Jul 11 05:06:07 2004
Subject: [Python-Dev] Equal but different keys in dicts
Message-ID: <1089515172.40f0aea49c441@login.werra.lunarpages.com>

Suppose you create two objects which are, in fact, different, but which are ==
to each other.

>>> class AllEqual:
...     def __init__(self, name):
...         self.name = name
...     def __repr__(self):
...         return self.name
...     def __eq__(self, other):
...         return 1
...     def __hash__(self)
...         return 47
...
>>> a = AllEqual('a')
>>> b = AllEqual('b')
>>> a == b
1

Now use one as a key in a dict, then try re-setting that value:

>>> map = {a:'one'}
>>> map
{a: 'one'}
>>> map[b] = 'two'
>>> map
{a: 'two'}

That last line could just as well have read "{b: 'two'}", but the implementation
of dict keeps the ORIGINAL key rather than using the NEW key. This is the
behavior of both CPython and Jython.

My question is this: is this behavior intentional, or is it an "implementation
detail"? If intentional, is there a reason for the choice, or is it just that
one or the other behavior needed to be chosen?

[PS: I'm hoping that a good answer to this question will allow me to close bug
748126]

-- Michael Chermside

From tim.peters at gmail.com  Sun Jul 11 07:09:59 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sun Jul 11 07:10:07 2004
Subject: [Python-Dev] Equal but different keys in dicts
In-Reply-To: <1089515172.40f0aea49c441@login.werra.lunarpages.com>
References: <1089515172.40f0aea49c441@login.werra.lunarpages.com>
Message-ID: <1f7befae040710220930627732@mail.gmail.com>

[Michael Chermside]
> ...
> My question is this: is this behavior intentional, or is it an "implementation
> detail"? If intentional, is there a reason for the choice, or is it just that
> one or the other behavior needed to be chosen?
>
> [PS: I'm hoping that a good answer to this question will allow me to close bug
> 748126]

It's undefined, but the docs could stand to explicitly point out that
it's undefined.  748126 is intimately related to the thread starting
here, BTW:

    http://mail.python.org/pipermail/python-dev/2003-June/035970.html

I don't think it's an accident that implementations are likely to keep
"the old key".  A natural way to code setitem(self, key, value) is

    if key in self:
        replace the value associated with key
    else:
        raise KeyError

While there was no deliberate attempt to do so, ZODB's BTrees (and
Buckets) also act this way:

>>> from BTrees.OOBTree import OOBTree
>>> s1 = 'a' + 'b'
>>> s2 = 'a' + 'b'
>>> id(s1)
10043520
>>> id(s2)  # s1 and s2 are == but not the same object
10044416
>>> b = OOBTree()
>>> b[s1] = 1
>>> id(b.keys()[0])
10043520
>>> b[s2] = 2   # replaces the value associated with s2 (== s1)
>>> len(b)   # still only 1 pair in this tree
1
>>> id(b.keys()[0])   # the key object didn't change
10043520
>>> b[s1]  # and the associated value did change
2
>>>
From s.percivall at chello.se  Sun Jul 11 16:53:31 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Sun Jul 11 16:53:32 2004
Subject: [Python-Dev] posixpath.py CVS contains tabs
Message-ID: <132E42A7-D34A-11D8-ACE1-0003934AD54A@chello.se>

The CVS version of posixpath.py contains tabs at lines 409, 428, 431 
and 433, making installation fail.

From guido at python.org  Sun Jul 11 18:27:12 2004
From: guido at python.org (Guido van Rossum)
Date: Sun Jul 11 18:27:16 2004
Subject: [Python-Dev] urllib.urlretrieve() and handling 550 errors when
	reading from FTP
In-Reply-To: Your message of "Sat, 10 Jul 2004 19:04:12 PDT."
	<40F0A01C.4040905@ocf.berkeley.edu> 
References: <40F05047.2030808@ocf.berkeley.edu>
	<200407110020.i6B0KDN01623@guido.python.org> 
	<40F0A01C.4040905@ocf.berkeley.edu> 
Message-ID: <200407111627.i6BGRCD18505@guido.python.org>

> > If you remove this, you won't be able to get a directory listing of
> > directories.  I tried this with and without the line; with that line
> > in, it gives a directlry listing, without it gives an IOError (Not a
> > regular file):
> > 
> >>>>urllib.urlretrieve("ftp://ftp.python.org/pub", "xyzzy")
> > 
> > So I'd be against deleting this, unless there's a different way to get
> > directory listings.
> 
> Try it again but with a trailing slash on the address.  With the line 
> removed it will fetch the directory.  Otherwise it errors out.  The code 
> apparently tries to get the file and when it fails with a 550 (meaning 
> the file cannot be accessed because of permissions or lack of 
> existence), the code then send a LIST command which lists whatever the 
> argument is, file or directory.
> 
> Unfortunately I can't think of a good way to detect the difference 
> between a file and a directory that just doesn't have the ending slash 
> beyond reading the output and seeing if it is a single listing for a 
> file with the same name as the LIST request.

Why should I have to use a trailing backslash to get a directory
listing?  That's not in the FTP standard and probably won't work
everywhere.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From bac at OCF.Berkeley.EDU  Sun Jul 11 21:08:51 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Jul 11 21:09:01 2004
Subject: [Python-Dev] urllib.urlretrieve() and handling 550 errors when
	reading from FTP
In-Reply-To: <200407111627.i6BGRCD18505@guido.python.org>
References: <40F05047.2030808@ocf.berkeley.edu>	<200407110020.i6B0KDN01623@guido.python.org>
	<40F0A01C.4040905@ocf.berkeley.edu>
	<200407111627.i6BGRCD18505@guido.python.org>
Message-ID: <40F19043.3080108@ocf.berkeley.edu>

Guido van Rossum wrote:
[SNIP]
> 
> Why should I have to use a trailing backslash to get a directory
> listing?  That's not in the FTP standard and probably won't work
> everywhere.
> 

It's the way the code is written for the FTP wrapper in urllib; has 
nothing to do with ftp.python.org or FTP.  Basically the code checks to 
see if there is a trailing slash.  If it does it assumes it is a 
directory and thus only requests a listing of the path from the server. 
  Otherwise it tries to get the file.  If that get fails (with a 550) it 
then decides to try getting a listing for the path.  If that works it 
returns that, otherwise it completely fails.

Basically I don't think there is a way to make this work for urllib 
nicely.  If no one has any objections I will just clarify the docs 
stating that if an attempt to get a file fails on a 550 error (and of 
course I will say what that means) it will then try a LIST command and 
if that succeeds that is what is returned.  And if you need more 
fine-grained control then use ftplib.

-Brett
From bac at OCF.Berkeley.EDU  Sun Jul 11 21:12:39 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Jul 11 21:12:51 2004
Subject: [Python-Dev] posixpath.py CVS contains tabs
In-Reply-To: <132E42A7-D34A-11D8-ACE1-0003934AD54A@chello.se>
References: <132E42A7-D34A-11D8-ACE1-0003934AD54A@chello.se>
Message-ID: <40F19127.4060500@ocf.berkeley.edu>

Simon Percivall wrote:

> The CVS version of posixpath.py contains tabs at lines 409, 428, 431 and 
> 433, making installation fail.
> 

I'll fix it; came from me applying a patch.

-Brett
From tim.one at comcast.net  Mon Jul 12 02:26:07 2004
From: tim.one at comcast.net (Tim Peters)
Date: Mon Jul 12 02:26:14 2004
Subject: [Python-Dev] More joy with test_strptime
Message-ID: <20040712002612.39A4D1E4002@bag.python.org>

Today (CVS HEAD), test_strptime is failing on WinXP, in release and debug
test runs, but only when I run the full test suite:

test test_strptime failed -- Traceback (most recent call last):
  File "C:\Code\python\lib\test\test_strptime.py", line 289, in
test_bad_timezone
    self.failUnlessEqual(tz_value, -1)
AssertionError: 0 != -1


If I run it in isolation, it does not fail:

C:\Code\python\PCbuild>python ../lib/test/test_strptime.py
test_basic (__main__.getlang_Tests) ... ok
test_am_pm (__main__.LocaleTime_Tests) ... ok
test_date_time (__main__.LocaleTime_Tests) ... ok
test_lang (__main__.LocaleTime_Tests) ... ok
test_month (__main__.LocaleTime_Tests) ... ok
test_timezone (__main__.LocaleTime_Tests) ... ok
test_weekday (__main__.LocaleTime_Tests) ... ok
test_blankpattern (__main__.TimeRETests) ... ok
test_compile (__main__.TimeRETests) ... ok
test_matching_with_escapes (__main__.TimeRETests) ... ok
test_pattern (__main__.TimeRETests) ... ok
test_pattern_escaping (__main__.TimeRETests) ... ok
test_TypeError (__main__.StrptimeTests) ... ok
test_bad_timezone (__main__.StrptimeTests) ... ok
test_caseinsensitive (__main__.StrptimeTests) ... ok
test_date (__main__.StrptimeTests) ... ok
test_date_time (__main__.StrptimeTests) ... ok
test_day (__main__.StrptimeTests) ... ok
test_defaults (__main__.StrptimeTests) ... ok
test_escaping (__main__.StrptimeTests) ... ok
test_hour (__main__.StrptimeTests) ... ok
test_julian (__main__.StrptimeTests) ... ok
test_minute (__main__.StrptimeTests) ... ok
test_month (__main__.StrptimeTests) ... ok
test_percent (__main__.StrptimeTests) ... ok
test_second (__main__.StrptimeTests) ... ok
test_time (__main__.StrptimeTests) ... ok
test_timezone (__main__.StrptimeTests) ... ok
test_unconverteddata (__main__.StrptimeTests) ... ok
test_weekday (__main__.StrptimeTests) ... ok
test_year (__main__.StrptimeTests) ... ok
test_twelve_noon_midnight (__main__.Strptime12AMPMTests) ... ok
test_all_julian_days (__main__.JulianTests) ... ok
test_day_of_week_calculation (__main__.CalculationTests) ... ok
test_gregorian_calculation (__main__.CalculationTests) ... ok
test_julian_calculation (__main__.CalculationTests) ... ok
test_new_localetime (__main__.CacheTests) ... ok
test_regex_cleanup (__main__.CacheTests) ... ok
test_time_re_recreation (__main__.CacheTests) ... ok

----------------------------------------------------------------------
Ran 39 tests in 0.219s

OK

The regrtest framework alone is not making it fail either:

C:\Code\python\PCbuild>python ../lib/test/regrtest.py test_strptime
test_strptime
1 test OK.

Best guess is that some earlier test (in a full test run) is screwing up
locale in a way test_strptime can't recover from.  Maybe that's enough so
that you know who you are <wink>.


From tim.one at comcast.net  Mon Jul 12 03:00:16 2004
From: tim.one at comcast.net (Tim Peters)
Date: Mon Jul 12 03:00:21 2004
Subject: [Python-Dev] Queue.Queue guts replaced
Message-ID: <20040712010020.03D761E4002@bag.python.org>

Here's the checkin comment.  Note the last item:  I haven't done comparative
timings, because I don't care.  If you do, feel encouraged.  I can't be
convinced to care even if it's 10x slower than before, so if it is, complain
here and get a constituency if you care enough to re-obfuscate it.

"""
Bug #788520: Queue class has logic error when non-blocking

I don't agree it had a bug (see the report), so this is *not* a candidate
for backporting, but the docs were confusing and the Queue implementation
was old enough to vote.

Rewrote put/put_nowait/get/get_nowait from scratch, to use a pair of
Conditions (not_full and not_empty), sharing a common mutex.  The code
is 1/4 the size now, and 6.25x easier to understand.  For blocking
with timeout, we also get to reuse (indirectly) the tedious timeout
code from threading.Condition.  The Full and Empty exceptions raised
by non-blocking calls are now easy (instead of nearly impossible) to
explain truthfully:  Full is raised if and only if the Queue truly
is full when the non-blocking put call checks the queue size, and
similarly for Empty versus non-blocking get.

What I don't know is whether the new implementation is slower (or
faster) than the old one.  I don't really care.  Anyone who cares
a lot is encouraged to check that.
"""


From ewilhelm at sbcglobal.net  Mon Jul 12 03:11:45 2004
From: ewilhelm at sbcglobal.net (Eric Wilhelm)
Date: Mon Jul 12 03:09:52 2004
Subject: [Python-Dev] PyInstance_Check() and new-style classes
Message-ID: <200407112011.45276.ewilhelm@sbcglobal.net>

Hi,

I've succeeded in getting the perl module Inline::Python to work with 
new-style classes, but it seems kind of clunky and I'm wonding if someone can 
tell me if I'm doing this "the right way" (TM).

To avoid boring you, the file in question is py2pl.c
http://ericwilhelm.homeip.net/svn/Inline-Python/trunk/xs-based/py2pl.c

With this test file:
http://ericwilhelm.homeip.net/svn/Inline-Python/trunk/xs-based/t/18newclass.t
(this is just a translation of 07nherit.t into inherited new-style classes)

The question is:  How do I determine that an instance of a new-style class 
(class Daddy(object)) is an instance without doing this:
	int type_sum = 
			PyString_Check(obj)+
			PyNumber_Check(obj)+
			PyInt_Check(obj)+
			PyLong_Check(obj)+
			PyFloat_Check(obj)+
			PyType_Check(obj)+
			PyInstance_Check(obj)+
			PyDict_Check(obj)+
			PyMapping_Check(obj)+
			PySequence_Check(obj)+
			PyIter_Check(obj)+
			PyModule_Check(obj)+
			PyClass_Check(obj)+
			PyMethod_Check(obj)+
			PyCallable_Check(obj) +
			(obj == Py_None);
	if ((! type_sum && PyType_Check(PyObject_Type(obj))) ||
		 PyInstance_Check(obj)) {
END QUESTION

I've read the pep's 252 and 253, as well as the warts paper, and the current 
embedding documentation, and the descrintro.  If I've missed something, 
please point it out, since it is very hard to search the documentation for 
information about "type objects."

Some background:  The Inline modules allow us to extend Perl using various 
languages without going through XS.  Essentially, the code from the other 
language is embedded in some C code which is a Perl extension.  But, we need 
to be able to translate the data-types from Python to Perl through C, 
including creating entries in Perl's symbol table so that Python instances 
can be returned to the Perl code and then used to call the methods as if they 
were Perl subroutines.

For starters, I'm working from Neil Watkiss's code 
http://search.cpan.org/CPAN/authors/id/N/NE/NEILW/Inline-Python-0.20.tar.gz

From there, I checked the original into my subversion server.  There have been 
quite a few whitespace changes, so if you have subversion, an svn diff would 
be a good way to see the changes.

The source directory can be seen here:
http://ericwilhelm.homeip.net/svn/Inline-Python/trunk/xs-based/

Essentially, the non-whitespace changes look like this:
svn diff -r 199:202 \
  http://ericwilhelm.homeip.net/svn/Inline-Python/trunk/xs-based/py2pl.c

Where, the original Inline-Python-0.20 code would just return a string from 
the constructor of a new-style class because PyInstance_Check() did not 
return true.

So, is PyInstance_Check() supposed to work, or is there some PyFoo_Check() 
function that will work?

It seems really wrong to have to add-up all of the other types and use this 
process of elimination to identify a new-style class instance.  Also, it 
seems that it would fail if a class inherited from a built-in type.  I'm 
almost sure that I'm way off-course, so if anyone could put me on track, I'd 
really appreciate it.

Thanks,
Eric
From bac at OCF.Berkeley.EDU  Mon Jul 12 03:13:48 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Jul 12 03:14:01 2004
Subject: [Python-Dev] More joy with test_strptime
In-Reply-To: <20040712002612.39A4D1E4002@bag.python.org>
References: <20040712002612.39A4D1E4002@bag.python.org>
Message-ID: <40F1E5CC.1000001@ocf.berkeley.edu>

Tim Peters wrote:
> Today (CVS HEAD), test_strptime is failing on WinXP, in release and debug
> test runs, but only when I run the full test suite:
> 
> test test_strptime failed -- Traceback (most recent call last):
>   File "C:\Code\python\lib\test\test_strptime.py", line 289, in
> test_bad_timezone
>     self.failUnlessEqual(tz_value, -1)
> AssertionError: 0 != -1
> 
> 
> If I run it in isolation, it does not fail:
> 
> C:\Code\python\PCbuild>python ../lib/test/test_strptime.py
> test_basic (__main__.getlang_Tests) ... ok
> test_am_pm (__main__.LocaleTime_Tests) ... ok
> test_date_time (__main__.LocaleTime_Tests) ... ok
> test_lang (__main__.LocaleTime_Tests) ... ok
> test_month (__main__.LocaleTime_Tests) ... ok
> test_timezone (__main__.LocaleTime_Tests) ... ok
> test_weekday (__main__.LocaleTime_Tests) ... ok
> test_blankpattern (__main__.TimeRETests) ... ok
> test_compile (__main__.TimeRETests) ... ok
> test_matching_with_escapes (__main__.TimeRETests) ... ok
> test_pattern (__main__.TimeRETests) ... ok
> test_pattern_escaping (__main__.TimeRETests) ... ok
> test_TypeError (__main__.StrptimeTests) ... ok
> test_bad_timezone (__main__.StrptimeTests) ... ok
> test_caseinsensitive (__main__.StrptimeTests) ... ok
> test_date (__main__.StrptimeTests) ... ok
> test_date_time (__main__.StrptimeTests) ... ok
> test_day (__main__.StrptimeTests) ... ok
> test_defaults (__main__.StrptimeTests) ... ok
> test_escaping (__main__.StrptimeTests) ... ok
> test_hour (__main__.StrptimeTests) ... ok
> test_julian (__main__.StrptimeTests) ... ok
> test_minute (__main__.StrptimeTests) ... ok
> test_month (__main__.StrptimeTests) ... ok
> test_percent (__main__.StrptimeTests) ... ok
> test_second (__main__.StrptimeTests) ... ok
> test_time (__main__.StrptimeTests) ... ok
> test_timezone (__main__.StrptimeTests) ... ok
> test_unconverteddata (__main__.StrptimeTests) ... ok
> test_weekday (__main__.StrptimeTests) ... ok
> test_year (__main__.StrptimeTests) ... ok
> test_twelve_noon_midnight (__main__.Strptime12AMPMTests) ... ok
> test_all_julian_days (__main__.JulianTests) ... ok
> test_day_of_week_calculation (__main__.CalculationTests) ... ok
> test_gregorian_calculation (__main__.CalculationTests) ... ok
> test_julian_calculation (__main__.CalculationTests) ... ok
> test_new_localetime (__main__.CacheTests) ... ok
> test_regex_cleanup (__main__.CacheTests) ... ok
> test_time_re_recreation (__main__.CacheTests) ... ok
> 
> ----------------------------------------------------------------------
> Ran 39 tests in 0.219s
> 
> OK
> 
> The regrtest framework alone is not making it fail either:
> 
> C:\Code\python\PCbuild>python ../lib/test/regrtest.py test_strptime
> test_strptime
> 1 test OK.
> 
> Best guess is that some earlier test (in a full test run) is screwing up
> locale in a way test_strptime can't recover from.  Maybe that's enough so
> that you know who you are <wink>.
> 

=)  I like Tim's assessment.  Best way to test that theory would be to 
clear the cache in _strptime.  Tim, can you insert the lines::

     print "***%s == %s ?" % (_strptime._TimeRE_cache.locale_time.lang,
                           _strptime._getlang())
     _strptime._TimeRE_cache = _strptime.TimeRE()
     _strptime._regex_cache = {}

in the function (line 282 should be fine; just as long as you do it 
before any calls to _strptime).  That will state what the current 
language is set for LC_TIME and what the cache was created for (so we 
can see what is thought to be the settings) and then clear _strptime's 
cache.

While the code tries to detect locale changes, it is possible it is not 
manifesting itself in a way that is detectable.  If the clearing of the 
cache, and thus the rebuild of all values that strptime uses, solves 
this than that's what is happening, I think.

-Brett
From bac at OCF.Berkeley.EDU  Mon Jul 12 03:18:44 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Jul 12 03:18:53 2004
Subject: [Python-Dev] More joy with test_strptime
In-Reply-To: <40F1E5CC.1000001@ocf.berkeley.edu>
References: <20040712002612.39A4D1E4002@bag.python.org>
	<40F1E5CC.1000001@ocf.berkeley.edu>
Message-ID: <40F1E6F4.2020703@ocf.berkeley.edu>

Brett C. wrote:

[SNIP]

Tim, can you insert the lines::
> 
>     print "***%s == %s ?" % (_strptime._TimeRE_cache.locale_time.lang,
>                           _strptime._getlang())
>     _strptime._TimeRE_cache = _strptime.TimeRE()
>     _strptime._regex_cache = {}
> 
> in the function (line 282 should be fine; just as long as you do it 
> before any calls to _strptime).

Actually, make sure to stick at line 282 or maybe even higher; it needs 
to be done before any strptime or time code is touched.  I just realized 
that time.tzname() will be called once you do this and that will 
definitely mess the test up if you don't do it soon enough.  Sorry about 
that.

-Brett
From greg at cosc.canterbury.ac.nz  Mon Jul 12 03:46:42 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Jul 12 03:46:48 2004
Subject: [Python-Dev] Re: decorators and 2.4
In-Reply-To: <1089407067.40ef085b6db31@webmail.uoguelph.ca>
Message-ID: <200407120146.i6C1kgXb025964@cosc353.cosc.canterbury.ac.nz>

Martin Zarate <mzarate@uoguelph.ca>:

> IIRC, the "as" keyword became a protected keyword in recent
> versions, did it not?

Not as far as I can see...

Python 2.3.4 (#1, Jun 30 2004, 16:47:37) 
[GCC 3.2 20020903 (Red Hat Linux 8.0 3.2-7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> as = 42
>>> print as
42

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From tim.peters at gmail.com  Mon Jul 12 03:52:31 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Jul 12 03:52:54 2004
Subject: [Python-Dev] More joy with test_strptime
In-Reply-To: <40F1E5CC.1000001@ocf.berkeley.edu>
References: <20040712002612.39A4D1E4002@bag.python.org>
	<40F1E5CC.1000001@ocf.berkeley.edu>
Message-ID: <1f7befae0407111852780f4fc2@mail.gmail.com>

[Brett]
> =)  I like Tim's assessment.  Best way to test that theory would be to
> clear the cache in _strptime.  Tim, can you insert the lines::
> 
>     print "***%s == %s ?" % (_strptime._TimeRE_cache.locale_time.lang,
>                           _strptime._getlang())
>     _strptime._TimeRE_cache = _strptime.TimeRE()
>     _strptime._regex_cache = {}
> 
> in the function (line 282 should be fine; just as long as you do it
> before any calls to _strptime).

I assume you meant I should insert that in test_strptime.py.  Doesn't
do any good as-is, because the regrtest framework eats stdout.  I
changed it to print to stderr instead.

Running all the tests, this gets displayed:

test_strptime
***(None, None) == (None, None) ?
test test_strptime failed -- Traceback (most recent call last):
  File "C:\Code\python\lib\test\test_strptime.py", line 293, in
test_bad_timezone
    self.failUnlessEqual(tz_value, -1)
AssertionError: 0 != -1

Alas, the same thing is displayed when it passes (run in isolation):

test_strptime
***(None, None) == (None, None) ?
1 test OK.
From tim.peters at gmail.com  Mon Jul 12 05:02:50 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Jul 12 05:02:56 2004
Subject: [Python-Dev] More joy with test_strptime
In-Reply-To: <1f7befae0407111852780f4fc2@mail.gmail.com>
References: <20040712002612.39A4D1E4002@bag.python.org>
	<40F1E5CC.1000001@ocf.berkeley.edu>
	<1f7befae0407111852780f4fc2@mail.gmail.com>
Message-ID: <1f7befae04071120025e3f3233@mail.gmail.com>

FWIW, this is a minimal failing set of tests to run:

test___all__
test_site
test_strptime

Remove either of the first two, and test_strptime passes here.  Leave
them both in, and test_strptime fails here, and whether or not .pyc
files are deleted first.

If I put "return" as the first line of test_site.py's test_addpackage
and test_addsitedir, the problem goes away.  I don't know why, but
believe that then no instance of class PthFile ever gets created.

I don't know whether it's relevant, but PthFile.cleanup's code like:

        try:
            os.remove(self.file_path)
        except OSError:
            pass

is almost never a correct thing to do, because it hides errors if the
code is confused.  If the intent is to ensure that the file doesn't
exist, much better is

    if os.path.exists(self.file_path):
        os.remove(self.file_path)

Then that raises an error if the code incorrectly (on Windows!) tries
to delete a file that's still open.  But I don't think that's
happening here.

If I change PthFile.__init__'s

        self.imported = "time"

to

        self.imported = "socket"

the problem also goes away.  Ditto if I change it to

        self.imported = "re"

etc.  So now I'm suspecting that the tricks in PthFile are managing to
create a second instance of the time module not related to the
instance of the time module test_strptime is monkey-patching with its

            time.tzname = (tz_name, tz_name)
            time.daylight = 1

tricks.  But if so, I'm at a loss to explain why this would only
happen on Windows.  *Is* it the case that test_strptime is only
failing on Windows?
From tim.peters at gmail.com  Mon Jul 12 05:24:28 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Jul 12 05:25:03 2004
Subject: [Python-Dev] More joy with test_strptime
In-Reply-To: <1f7befae04071120025e3f3233@mail.gmail.com>
References: <20040712002612.39A4D1E4002@bag.python.org>
	<40F1E5CC.1000001@ocf.berkeley.edu>
	<1f7befae0407111852780f4fc2@mail.gmail.com>
	<1f7befae04071120025e3f3233@mail.gmail.com>
Message-ID: <1f7befae04071120245dedbd21@mail.gmail.com>

[Tim]
...
>  So now I'm suspecting that the tricks in PthFile are managing to
> create a second instance of the time module not related to the
> instance of the time module test_strptime is monkey-patching with its
> 
>            time.tzname = (tz_name, tz_name)
>            time.daylight = 1
> 
> tricks.

Using the attached patch yields output

test___all__
test_site
test_strptime
in strptime ('Eastern Standard Time', 'Eastern Daylight Time') 1
in test_bad_timezone ('Eastern Standard Time', 'Eastern Standard Time') 1
in strptime ('Eastern Standard Time', 'Eastern Daylight Time') 1
...

so test_strptime's attempt to monkey-patch the time module indeed has
no effect on the time module _strptime.py is using.

The problem can be fixed by using this as the first line of test_bad_timezone:

        from _strptime import time

But, of course, that's insane.

>  But if so, I'm at a loss to explain why this would only
> happen on Windows.

Still so.

>  *Is* it the case that test_strptime is only failing on Windows?

Still wondering that.
From tim.peters at gmail.com  Mon Jul 12 05:25:22 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Jul 12 05:25:36 2004
Subject: [Python-Dev] More joy with test_strptime
In-Reply-To: <1f7befae04071120245dedbd21@mail.gmail.com>
References: <20040712002612.39A4D1E4002@bag.python.org>
	<40F1E5CC.1000001@ocf.berkeley.edu>
	<1f7befae0407111852780f4fc2@mail.gmail.com>
	<1f7befae04071120025e3f3233@mail.gmail.com>
	<1f7befae04071120245dedbd21@mail.gmail.com>
Message-ID: <1f7befae0407112025231baec4@mail.gmail.com>

Forgot the debug-output patch; attached.
-------------- next part --------------
Index: Lib/_strptime.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/_strptime.py,v
retrieving revision 1.32
diff -c -u -r1.32 _strptime.py
--- Lib/_strptime.py	4 May 2004 09:21:43 -0000	1.32
+++ Lib/_strptime.py	12 Jul 2004 03:16:43 -0000
@@ -272,6 +272,8 @@
 def strptime(data_string, format="%a %b %d %H:%M:%S %Y"):
     """Return a time struct based on the input string and the format string."""
     global _TimeRE_cache
+    import sys
+    print >> sys.stderr, "in strptime", time.tzname, time.daylight
     _cache_lock.acquire()
     try:
         time_re = _TimeRE_cache
Index: Lib/test/test_strptime.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_strptime.py,v
retrieving revision 1.24
diff -c -u -r1.24 test_strptime.py
--- Lib/test/test_strptime.py	16 Nov 2003 16:17:48 -0000	1.24
+++ Lib/test/test_strptime.py	12 Jul 2004 03:16:44 -0000
@@ -285,6 +285,7 @@
             original_daylight = time.daylight
             time.tzname = (tz_name, tz_name)
             time.daylight = 1
+            print >> sys.stderr, "in test_bad_timezone", time.tzname, time.daylight
             tz_value = _strptime.strptime(tz_name, "%Z")[8]
             self.failUnlessEqual(tz_value, -1)
         finally:
From guido at python.org  Mon Jul 12 05:55:39 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Jul 12 05:55:47 2004
Subject: [Python-Dev] urllib.urlretrieve() and handling 550 errors when
	reading from FTP
In-Reply-To: Your message of "Sun, 11 Jul 2004 12:08:51 PDT."
	<40F19043.3080108@ocf.berkeley.edu> 
References: <40F05047.2030808@ocf.berkeley.edu>
	<200407110020.i6B0KDN01623@guido.python.org>
	<40F0A01C.4040905@ocf.berkeley.edu>
	<200407111627.i6BGRCD18505@guido.python.org> 
	<40F19043.3080108@ocf.berkeley.edu> 
Message-ID: <200407120355.i6C3tdJ19164@guido.python.org>

> > Why should I have to use a trailing backslash to get a directory
> > listing?  That's not in the FTP standard and probably won't work
> > everywhere.
> 
> It's the way the code is written for the FTP wrapper in urllib; has 
> nothing to do with ftp.python.org or FTP.  Basically the code checks to 
> see if there is a trailing slash.  If it does it assumes it is a 
> directory and thus only requests a listing of the path from the server. 

Ah, right.  Sorry.  Alzheimer is setting in early today. :-)

>   Otherwise it tries to get the file.  If that get fails (with a 550) it 
> then decides to try getting a listing for the path.  If that works it 
> returns that, otherwise it completely fails.
> 
> Basically I don't think there is a way to make this work for urllib 
> nicely.

Depends on what you call nicely.  I think of the current behavior as
"nice", because directories are a lot more common that files you can't
read in the typical ftp setup.

> If no one has any objections I will just clarify the docs 
> stating that if an attempt to get a file fails on a 550 error (and of 
> course I will say what that means) it will then try a LIST command and 
> if that succeeds that is what is returned.  And if you need more 
> fine-grained control then use ftplib.

Sounds good to me.  Thanks!

--Guido van Rossum (home page: http://www.python.org/~guido/)
From jhylton at gmail.com  Mon Jul 12 14:37:21 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Mon Jul 12 14:37:48 2004
Subject: [Python-Dev] test_pwd is really slow
Message-ID: <e8bf7a5304071205377772a232@mail.gmail.com>

I've been having trouble running the tests lately, because test_pwd is
really slow.  It's a fairly trivial set of tests -- checking the types
of return values and cross-checking getpwall() vs.
getpwnam()/getpwuid().  The problem is if your password database is
provided by LDAP and is really big, the test takes an impractically
long time.

Would anyone object if I changed the test suite to require some
resource to run test_pwd?  Or should I just make some local change to
disable it?

Jeremy
From mwh at python.net  Mon Jul 12 15:08:48 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Jul 12 15:08:49 2004
Subject: [Python-Dev] PyInstance_Check() and new-style classes
In-Reply-To: <200407112011.45276.ewilhelm@sbcglobal.net> (Eric Wilhelm's
	message of "Sun, 11 Jul 2004 20:11:45 -0500")
References: <200407112011.45276.ewilhelm@sbcglobal.net>
Message-ID: <2mwu19s78f.fsf@starship.python.net>

Eric Wilhelm <ewilhelm@sbcglobal.net> writes:

> Hi,
>
> I've succeeded in getting the perl module Inline::Python to work with 
> new-style classes, but it seems kind of clunky and I'm wonding if someone can 
> tell me if I'm doing this "the right way" (TM).

Why does it matter?  I did actually read the rest of your post, but
this failed to leap out at me (apologies if I'm being dumb).

There's no real answer -- there's just not that much difference
between user-defined new-style classes and builtin types (part of
their appeal!) but checking the TP_HEAPTYPE flag in tp_flags may go
some way towards one.

Cheers,
mwh

-- 
  [3] Modem speeds being what they are, large .avi files were
      generally downloaded to the shell server instead[4].
  [4] Where they were usually found by the technical staff, and
      burned to CD.                                   -- Carlfish, asr
From aahz at pythoncraft.com  Mon Jul 12 15:44:44 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Jul 12 15:44:50 2004
Subject: [Python-Dev] file() or open()?
In-Reply-To: <ccntfs$k9g$1@sea.gmane.org>
References: <40EC61DD.50704@ezabel.com> <ccntfs$k9g$1@sea.gmane.org>
Message-ID: <20040712134443.GA21431@panix.com>

On Sat, Jul 10, 2004, Terry Reedy wrote:
>
> In a previous post, Guido justified 'open' on the basis that
> applications 'open' 'files'.  However, in every such application I
> can think of at the moment, 'open' means 'read the data and convert
> it to one of the application's internal native formats, even if it
> was written in a different format by another application.  In other
> words, app.open(path) == file(path).read().convert(filetype(path)).
> Conversely, one never opens a file before saveing a fresh new
> document; one just saves-as.  So Python's open() is quite different
> from the app usage of 'open'.

On the gripping hand, just about every programming language in the Algol
family uses ``open()``.  And yes, I'm specifically including C/C++ and
Perl.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Typing is cheap.  Thinking is expensive."  --Roy Smith, c.l.py
From pje at telecommunity.com  Mon Jul 12 15:48:19 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Jul 12 15:45:02 2004
Subject: [Python-Dev] PyInstance_Check() and new-style classes
In-Reply-To: <200407112011.45276.ewilhelm@sbcglobal.net>
Message-ID: <5.1.1.6.0.20040712093928.01ed4100@mail.telecommunity.com>

At 08:11 PM 7/11/04 -0500, Eric Wilhelm wrote:

>So, is PyInstance_Check() supposed to work, or is there some PyFoo_Check()
>function that will work?

No, to both questions.


>It seems really wrong to have to add-up all of the other types and use this
>process of elimination to identify a new-style class instance.

As of 2.2, *all* Python types *except* InstanceType may be considered to be 
"new-style class instances".

If you want to find out whether a type was defined in Python or C, you can 
use the "heap type" flag in the type structure.  Note, however, that this 
will not let you distinguish between core C types and C extension types, so 
using it might be a bad idea if the reason you're using Python is to use a 
Python module written in C!


>  Also, it
>seems that it would fail if a class inherited from a built-in type.  I'm
>almost sure that I'm way off-course, so if anyone could put me on track, I'd
>really appreciate it.

Why do you want to distinguish between different kinds of objects?  Is it 
because Perl needs/wants Python strings and numbers to become 
scalars?  Perhaps you should simply convert those types that you want an 
exact conversion for, and consider everything else to be 
"instances".  Better yet, provide a way for the user to define Perl 
mappings for specific Python types, which would give more flexibility in 
bridging the two, or require explicit conversion operations.  Or perhaps 
you could use Perl contexts in some way to decide what type of conversion 
should take place?

(Also, isn't there a Python::Object package out there?  What does it 
do?  Has it been updated for Python 2.2?)

From aahz at pythoncraft.com  Mon Jul 12 15:49:52 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Jul 12 15:49:55 2004
Subject: [Python-Dev] Comparison and (user) confusion
In-Reply-To: <20040701173715.GA7389@titan.progiciels-bpi.ca>
References: <20040701173715.GA7389@titan.progiciels-bpi.ca>
Message-ID: <20040712134952.GB21431@panix.com>

On Thu, Jul 01, 2004, Fran?ois Pinard wrote:
>
> The problem appeared when I sub-classed `list', and used a `__cmp__'
> method in my sub-type to effect a specialised sort.  The sort did not
> behave as expected, as the `list' type provides rich comparisons, and
> that `sort', finding `__lt__', merely ignored my own `__cmp__'.
> 
> I wonder if Python could not have helped better here, by diagnosing
> a possible comparison confusion while creating the sub-class.  I do
> not know much know Python internals, but if C slots happen to be used
> for comparison methods, it might not be so unreasonable to expect such
> confusion to be detected whenever those slots are filled.
> 
> I presume here, maybe wrongly, that it does not make good sense mixing
> rich and non-rich comparisons in the same class.  Just a thought. :-)

There is one good reason for implementing both __cmp__() and rich
comparisons: when comparing is expensive, it makes sense to call cmp()
directly if you need the full three-way compare to branch.  (Decimal is
a good example of this -- at least it was when I originally wrote it,
and it probably still has the same need now.)

While your question also makes sense, I'd say it's the province of
PyChecker rather than Python itself.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Typing is cheap.  Thinking is expensive."  --Roy Smith, c.l.py
From ewilhelm at sbcglobal.net  Mon Jul 12 16:01:08 2004
From: ewilhelm at sbcglobal.net (Eric Wilhelm)
Date: Mon Jul 12 15:59:13 2004
Subject: [Python-Dev] PyInstance_Check() and new-style classes
In-Reply-To: <2mwu19s78f.fsf@starship.python.net>
References: <200407112011.45276.ewilhelm@sbcglobal.net>
	<2mwu19s78f.fsf@starship.python.net>
Message-ID: <200407120901.08433.ewilhelm@sbcglobal.net>

# The following was supposedly scribed by
# Michael Hudson
# on Monday 12 July 2004 08:08 am:

>> tell me if I'm doing this "the right way" (TM).
>
>Why does it matter? ?I did actually read the rest of your post, but
>this failed to leap out at me (apologies if I'm being dumb).

For starters, the process-of-elimination way of determining the type is pretty 
clunky, and only works on classes which have subclassed the 'object' type 
(not lists, tuples, strings, etc.)

Also, it seems that the type/class unification has broken the API function 
PyInstance_Check(), which I think should tell me if I'm dealing with an 
instance of a builtin type.  If not, it seems that there should be some 
function which allows me to perform this check.

>There's no real answer -- there's just not that much difference
>between user-defined new-style classes and builtin types (part of
>their appeal!) but checking the TP_HEAPTYPE flag in tp_flags may go
>some way towards one.

I'll have to look into this.  I'm not sure what that would tell me.

Basically, Perl's "objects" are just bless()ed references.  That seems 
analogous to the direction that Python has gone with the type/class 
unification, but Perl is a little less formal about objects.

This brings up another problem with my current code in that a subclass of an 
actual builtin type (as opposed to just 'object') needs to be handled 
differently when constructing the perl values.  e.g. I'd like to return a 
blessed reference to a Perl array if the class is subclassing the python 
array.  This won't work with the current scheme, since the blessed reference 
for a class instance is a reference to a scalar containing its memory 
address. 

However, I'm willing to punt that and just use the pointer to the python 
object (be it a list, tuple, or object) provided that I can identify it as a 
class instance.

But, this won't work with the way things are now, since PySequence_Check() 
would presumably return true for an instance of a class which had subclassed 
the builtin list type.

Granted, we could just pass pointers to python objects for everything, but 
that wouldn't make a very good perl binding, since we'd then have to use 
$string->append("foo") instead of Perl's builtin $string .= "foo" and I'm not 
sure yet how we would get to the actual value of $string under that schema.

--Eric
-- 
"Because understanding simplicity is complicated."
                                        --Eric Raymond
From ewilhelm at sbcglobal.net  Mon Jul 12 16:15:07 2004
From: ewilhelm at sbcglobal.net (Eric Wilhelm)
Date: Mon Jul 12 16:13:13 2004
Subject: [Python-Dev] PyInstance_Check() and new-style classes
In-Reply-To: <5.1.1.6.0.20040712093928.01ed4100@mail.telecommunity.com>
References: <5.1.1.6.0.20040712093928.01ed4100@mail.telecommunity.com>
Message-ID: <200407120915.07913.ewilhelm@sbcglobal.net>

# The following was supposedly scribed by
# Phillip J. Eby
# on Monday 12 July 2004 08:48 am:

>If you want to find out whether a type was defined in Python or C, you can
>use the "heap type" flag in the type structure. ?Note, however, that this 
>will not let you distinguish between core C types and C extension types, so
>using it might be a bad idea if the reason you're using Python is to use a
>Python module written in C!

I'm not sure if I understand what this flag is telling me.  According to the 
api/type-structs.html, Py_TPFLAGS_HEAPTYPE is set when the object is 
allocated.  To me, that reads that it would be set for builtin tuples, etc 
even if they were not subclassed.

What about Py_TPFLAGS_HAVE_CLASS ?


>> ?Also, it
>>seems that it would fail if a class inherited from a built-in type. ?I'm
>>almost sure that I'm way off-course, so if anyone could put me on track,
>> I'd really appreciate it.
>
>Why do you want to distinguish between different kinds of objects? ?Is it
>because Perl needs/wants Python strings and numbers to become
>scalars? ?

To have a useful binding, yes.

We want to create constructors for constructors, methods for methods, and 
translate everything else into native scalars, hashes, and lists.

>Perhaps you should simply convert those types that you want an 
>exact conversion for, and consider everything else to be
>"instances". 

That has been the current approach.  The trouble is, we need to know that they 
are instances so we can bless them, and we need to know that they are 
instances of a user-defined class, even if they are subtypes of builtin 
types.

Thanks,
Eric
From mwh at python.net  Mon Jul 12 16:14:00 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Jul 12 16:14:02 2004
Subject: [Python-Dev] PyInstance_Check() and new-style classes
In-Reply-To: <200407120901.08433.ewilhelm@sbcglobal.net> (Eric Wilhelm's
	message of "Mon, 12 Jul 2004 09:01:08 -0500")
References: <200407112011.45276.ewilhelm@sbcglobal.net>
	<2mwu19s78f.fsf@starship.python.net>
	<200407120901.08433.ewilhelm@sbcglobal.net>
Message-ID: <2miscts47r.fsf@starship.python.net>

Eric Wilhelm <ewilhelm@sbcglobal.net> writes:

> # The following was supposedly scribed by
> # Michael Hudson
> # on Monday 12 July 2004 08:08 am:
>
>>> tell me if I'm doing this "the right way" (TM).
>>
>>Why does it matter? ?I did actually read the rest of your post, but
>>this failed to leap out at me (apologies if I'm being dumb).
>
> For starters, the process-of-elimination way of determining the type is pretty 
> clunky, and only works on classes which have subclassed the 'object' type 
> (not lists, tuples, strings, etc.)

That just restates the problem :-) *Why* do you care if a given object
is an instance of a builtin type or not?

> Also, it seems that the type/class unification has broken the API function 
> PyInstance_Check(), which I think should tell me if I'm dealing with an 
> instance of a builtin type.  If not, it seems that there should be some 
> function which allows me to perform this check.

Well, in Python/C API terms "Instance" usually means "instance of
old-style class".  Apologies, but what you think these functions
should do isn't the most reliable of guides :-)

>>There's no real answer -- there's just not that much difference
>>between user-defined new-style classes and builtin types (part of
>>their appeal!) but checking the TP_HEAPTYPE flag in tp_flags may go
>>some way towards one.
>
> I'll have to look into this.  I'm not sure what that would tell me.

Well, it tells you whether the type objects memory lives on the heap
or is a static object from the C point of view.  This is potentially
interesting because it being true is /almost/ synonmous with
'user-defined type'.

> Basically, Perl's "objects" are just bless()ed references.  That seems 
> analogous to the direction that Python has gone with the type/class 
> unification, but Perl is a little less formal about objects.

Um.  From a knowing-Python-but-not-knowing-much-perl point of view, I
wouldn't agree with that statement...

[snippety]

> Granted, we could just pass pointers to python objects for everything, but 
> that wouldn't make a very good perl binding, since we'd then have to use 
> $string->append("foo") instead of Perl's builtin $string .= "foo" and I'm not 
> sure yet how we would get to the actual value of $string under that schema.

Seriously, I'd advise to you check for the types you *do* know how to
bridge to perl types -- integers, lists, strings, that sort of thing
-- and generically wrap everything else.

What you do with subclasses of builtin types like strings is a
difficult question.  I wouldn't presume to know the best thing to do
here.

Cheers,
mwh

-- 
  There are two kinds of large software systems: those that evolved
  from small systems and those that don't work.
                           -- Seen on slashdot.org, then quoted by amk
From amk at amk.ca  Mon Jul 12 16:37:39 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Jul 12 16:38:24 2004
Subject: [Python-Dev] Bug day outcome
Message-ID: <20040712143739.GA7295@rogue.amk.ca>

The bug day ended up closing 15 bugs and 18 patches.  Thanks to
everyone who participated.  A few python-dev people were around for
much of the day; in particular Brett Cannon was really helpful in
processing bugs and keeping the Wiki page updated.

Anthony has said he expects an alpha2 in three weeks or so.  That puts
the next release around July 29th.  I suggest Saturday, August 7th as
the next bug day.  (Or would Sunday be better for people?)

--amk
From barry at python.org  Mon Jul 12 16:41:29 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Jul 12 16:41:17 2004
Subject: [Python-Dev] test_pwd is really slow
In-Reply-To: <e8bf7a5304071205377772a232@mail.gmail.com>
References: <e8bf7a5304071205377772a232@mail.gmail.com>
Message-ID: <1089643289.11697.31.camel@localhost>

On Mon, 2004-07-12 at 08:37, Jeremy Hylton wrote:

> Would anyone object if I changed the test suite to require some
> resource to run test_pwd?  Or should I just make some local change to
> disable it?

Is there some generic/dynamic way to test whether you're pwd is using
LDAP or not?  If there is, a more generic resource switch that isn't
specific to pwd would be better (e.g. what about grp or other
databases?).

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040712/bb774890/attachment-0001.pgp
From jim.jewett at eds.com  Mon Jul 12 16:43:00 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Mon Jul 12 16:44:02 2004
Subject: [Python-Dev] decimal.py signals & traps
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C22955D90@usahm010.exmi01.exch.eds.com>

Raymond writes:
> I would like change the Context constructor API to [...]
> (list only the flags and traps that you want set).

I assume there will still be a reasonable way to say both

"Keep the context just as it is, but turn *off* flag/trap X"
"Keep the context just as it is, but turn *on* flag/trap X"

The easiest might be to just ensure that there is an easy 
way to list current settings (despite the privacy of the dict).

-jJ
From jim.jewett at eds.com  Mon Jul 12 16:49:40 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Mon Jul 12 16:50:41 2004
Subject: [Python-Dev] Re: decorators and 2.4
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C22955D91@usahm010.exmi01.exch.eds.com>

Martin Zarate:

> So, a solution, mixing them all so that it becomes more than
> a simple list declaration:

>	as [descriptors]
>	def blah blah blah:

If you start a production with "as", then suddenly it matters more
what the line before was/whether that line had a continuation character.
For instance, the following becomes syntactically valid, though I doubt
it means what the user expects.

	import time			\
	as logged_func
	def ...

This objection does not apply to either

	[descriptors] as

or

	@[descriptors]

though the @[] format might still be in use by some user code.

-jJ

From python at rcn.com  Mon Jul 12 05:11:57 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Jul 12 17:13:32 2004
Subject: [Python-Dev] decimal.py signals & traps
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C22955D90@usahm010.exmi01.exch.eds.com>
Message-ID: <000201c467bd$fdc45980$e841fea9@oemcomputer>

> Raymond writes:
> > I would like change the Context constructor API to [...]
> > (list only the flags and traps that you want set).

[Jim]
> I assume there will still be a reasonable way to say both

Yes.


> "Keep the context just as it is, but turn *off* flag/trap X"
> "Keep the context just as it is, but turn *on* flag/trap X"


After simplifying the constructor and altering the representation to
match, there was no longer much of a need to try to hide the dictionary
with a bunch of methods.  Since everyone already knows how to use a
dictionary, the API is easier to learn without method access to traps
and flags:


>>> c = Context(prec=7, traps=[DivisionByZero, Overflow])
>>> c
Context(prec=7, rounding=ROUND_HALF_EVEN, Emin=-999999999,
Emax=999999999, capitals=1, flags=[], traps=[DivisionByZero, Overflow])
>>> c.traps[Overflow] = 0
>>> c
Context(prec=7, rounding=ROUND_HALF_EVEN, Emin=-999999999,
Emax=999999999, capitals=1, flags=[], traps=[DivisionByZero])


As a nice side benefit, eval(repr(c)) now works too.

Privatizing the dictionary can still be done but I don't think it is
worth adding more methods and slowing everything down with an additional
layer of indirection.



Raymond

From mwh at python.net  Mon Jul 12 17:20:46 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Jul 12 17:20:48 2004
Subject: [Python-Dev] decimal.py signals & traps
In-Reply-To: <000201c467bd$fdc45980$e841fea9@oemcomputer> (Raymond
	Hettinger's message of "Sun, 11 Jul 2004 23:11:57 -0400")
References: <000201c467bd$fdc45980$e841fea9@oemcomputer>
Message-ID: <2m8ydps14h.fsf@starship.python.net>

"Raymond Hettinger" <python@rcn.com> writes:

> Privatizing the dictionary can still be done but I don't think it is
> worth adding more methods and slowing everything down with an additional
> layer of indirection.

Well, there's an argument that this may not be the most natural
interface for a C implementation or for an implementation of such
things for floats -- but this is an argument involving predicting the
future, so probably should be ignored.

Cheers,
mwh

-- 
  C is not clean -- the language has _many_ gotchas and traps, and
  although its semantics are _simple_ in some sense, it is not any
  cleaner than the assembly-language design it is based on.
                                        -- Erik Naggum, comp.lang.lisp
From pje at telecommunity.com  Mon Jul 12 18:12:49 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Jul 12 18:09:39 2004
Subject: [Python-Dev] PyInstance_Check() and new-style classes
In-Reply-To: <200407120915.07913.ewilhelm@sbcglobal.net>
References: <5.1.1.6.0.20040712093928.01ed4100@mail.telecommunity.com>
	<5.1.1.6.0.20040712093928.01ed4100@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040712115527.030c97a0@mail.telecommunity.com>

At 09:15 AM 7/12/04 -0500, Eric Wilhelm wrote:
># The following was supposedly scribed by
># Phillip J. Eby
># on Monday 12 July 2004 08:48 am:
>
> >If you want to find out whether a type was defined in Python or C, you can
> >use the "heap type" flag in the type structure.  Note, however, that this
> >will not let you distinguish between core C types and C extension types, so
> >using it might be a bad idea if the reason you're using Python is to use a
> >Python module written in C!
>
>I'm not sure if I understand what this flag is telling me.  According to the
>api/type-structs.html, Py_TPFLAGS_HEAPTYPE is set when the object is
>allocated.  To me, that reads that it would be set for builtin tuples, etc
>even if they were not subclassed.

A "heap type" is a type whose type struct is in heap memory.  A "non-heap" 
type is one whose type struct is declared statically, i.e. in C code.

So, a non-heap type is either a built-in type, or an extension type 
provided by a C extension module.  A heap type is either one coded in pure 
Python, or potentially a C extension coded in an unusual fashion (e.g. via 
Pyrex).


> >Why do you want to distinguish between different kinds of objects?  Is it
> >because Perl needs/wants Python strings and numbers to become
> >scalars?
>
>To have a useful binding, yes.
>
>We want to create constructors for constructors, methods for methods, and
>translate everything else into native scalars, hashes, and lists.

For mappings and sequences (other than Python dicts and lists) you might 
consider using Perl's "tie" facilities, to avoid undesirable changes in 
semantics.


> >Perhaps you should simply convert those types that you want an
> >exact conversion for, and consider everything else to be
> >"instances".
>
>That has been the current approach.  The trouble is, we need to know that 
>they
>are instances so we can bless them, and we need to know that they are
>instances of a user-defined class, even if they are subtypes of builtin
>types.


Here's what I would suggest:

1. Use 'PyWhatever_CheckExact()' tests for known builtin types, where you 
can convert them with full fidelity.  (Or use a switch on the object's 
ob_type field.)

2. For all other types, convert to a blessed reference, *unless* you either 
call an explicit conversion function or method, or use Perl's context 
facilities to decide how to interpret the object.  If the context wants an 
array, test for sequenceness or iterability.  If the context wants a hash, 
test for mapping-ness.  If the context wants a scalar, test for number or 
stringness.

I'm making the assumption here that your code has some way of knowing what 
kind of context the surrounding code provides.  If not, you'll have to have 
conversions like:

     perl_string($python_ref)
     perl_number($python_ref)
     perl_hash($python_ref)
     perl_array($python_ref)
     perl_function($python_ref)

etc.  (Of course, your function names will probably vary.)

Essentially, if you want to be able to map Python objects to Perl values 
with full fidelity, you really need to be able to say what kind of Perl 
value you want, and that was really true even *before* Python 2.2.  DWIM 
isn't really an option except for builtin types, and *only* for the builtin 
types, not their subclasses.



From guido at python.org  Mon Jul 12 18:26:15 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Jul 12 18:26:23 2004
Subject: [Python-Dev] test_pwd is really slow
In-Reply-To: Your message of "Mon, 12 Jul 2004 08:37:21 EDT."
	<e8bf7a5304071205377772a232@mail.gmail.com> 
References: <e8bf7a5304071205377772a232@mail.gmail.com> 
Message-ID: <200407121626.i6CGQFl21951@guido.python.org>

> I've been having trouble running the tests lately, because test_pwd is
> really slow.  It's a fairly trivial set of tests -- checking the types
> of return values and cross-checking getpwall() vs.
> getpwnam()/getpwuid().  The problem is if your password database is
> provided by LDAP and is really big, the test takes an impractically
> long time.
> 
> Would anyone object if I changed the test suite to require some
> resource to run test_pwd?  Or should I just make some local change to
> disable it?

How about instead limiting the number of entries checked to the first
100?  Try changing

        entries = pwd.getpwall()

into

        entries = pwd.getpwall()[:100]

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Jul 12 18:34:04 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Jul 12 18:34:12 2004
Subject: [Python-Dev] Comparison and (user) confusion
In-Reply-To: Your message of "Mon, 12 Jul 2004 09:49:52 EDT."
	<20040712134952.GB21431@panix.com> 
References: <20040701173715.GA7389@titan.progiciels-bpi.ca>  
	<20040712134952.GB21431@panix.com> 
Message-ID: <200407121634.i6CGY4121992@guido.python.org>

> There is one good reason for implementing both __cmp__() and rich
> comparisons: when comparing is expensive, it makes sense to call cmp()
> directly if you need the full three-way compare to branch.  (Decimal is
> a good example of this -- at least it was when I originally wrote it,
> and it probably still has the same need now.)

Another good example comes from the <, <=, >, >= operators on lists.
These require you to step through the lists skipping equal item pairs,
and then the comparison outcome is determined by comparing the first
pair of unequal items.  If the item comparison is expensive, it makes
sense to want to compare each pair of items at most once, using cmp().

--Guido van Rossum (home page: http://www.python.org/~guido/)
From ewilhelm at sbcglobal.net  Mon Jul 12 18:37:40 2004
From: ewilhelm at sbcglobal.net (Eric Wilhelm)
Date: Mon Jul 12 18:35:46 2004
Subject: [Python-Dev] PyInstance_Check() and new-style classes
In-Reply-To: <5.1.1.6.0.20040712115527.030c97a0@mail.telecommunity.com>
References: <5.1.1.6.0.20040712093928.01ed4100@mail.telecommunity.com>
	<5.1.1.6.0.20040712115527.030c97a0@mail.telecommunity.com>
Message-ID: <200407121137.40272.ewilhelm@sbcglobal.net>

# The following was supposedly scribed by
# Phillip J. Eby
# on Monday 12 July 2004 11:12 am:

>>I'm not sure if I understand what this flag is telling me. ?According to
>> the api/type-structs.html, Py_TPFLAGS_HEAPTYPE is set when the object is
>> allocated. ?To me, that reads that it would be set for builtin tuples, etc
>> even if they were not subclassed.
>
>A "heap type" is a type whose type struct is in heap memory. ?A "non-heap"
>type is one whose type struct is declared statically, i.e. in C code.
>
>So, a non-heap type is either a built-in type, or an extension type
>provided by a C extension module. ?A heap type is either one coded in pure
>Python, or potentially a C extension coded in an unusual fashion (e.g. via
>Pyrex).

Okay, looks like this will work just beautifully.  Heaptype is true for an 
instance of a class which inherits from a builtin type, but not for a simple 
instance of a builtin type.

True, if I were trying to wrap a C extension module via Inline::Python, 
heaptype would not be true, but a really easy work-around should exist by 
simply defining a python class which does nothing but inherit the C 
extension.  That would give the Inline::Python code something to grab onto.

Anyway, this is much cleaner:

http://ericwilhelm.homeip.net/svn/Inline-Python/trunk/xs-based/py2pl.c

-       int type_sum =
-                       PyString_Check(obj)+
-                       PyNumber_Check(obj)+
<snip lots of PyFoo_Check(obj) + lines>

        /* wrap an instance of a Python class */
        /* elw: here we need to make these look like instances: */
-       if ((! type_sum && PyType_Check(PyObject_Type(obj))) || 
PyInstance_Check(obj)) {
+       if ((obj->ob_type->tp_flags & Py_TPFLAGS_HEAPTYPE) || 
PyInstance_Check(obj)) {

So,  I think, with everybody's help, I may have found "the right way" (TM.)

Should this be better documented, or is this an obscure usage?  Will it break 
in future versions of Python?

Thanks,
Eric
-- 
I arise in the morning torn between a desire to improve the world and a
desire to enjoy the world. This makes it hard to plan the day. 
                                        --E.B. White
From ewilhelm at sbcglobal.net  Mon Jul 12 18:48:25 2004
From: ewilhelm at sbcglobal.net (Eric Wilhelm)
Date: Mon Jul 12 18:46:30 2004
Subject: [Python-Dev] PyInstance_Check() and new-style classes
In-Reply-To: <2miscts47r.fsf@starship.python.net>
References: <200407112011.45276.ewilhelm@sbcglobal.net>
	<200407120901.08433.ewilhelm@sbcglobal.net>
	<2miscts47r.fsf@starship.python.net>
Message-ID: <200407121148.25400.ewilhelm@sbcglobal.net>

# The following was supposedly scribed by
# Michael Hudson
# on Monday 12 July 2004 09:14 am:

>Eric Wilhelm <ewilhelm@sbcglobal.net> writes:
>> For starters, the process-of-elimination way of determining the type is
>> pretty clunky, and only works on classes which have subclassed the
>> 'object' type (not lists, tuples, strings, etc.)
>
>That just restates the problem :-) *Why* do you care if a given object
>is an instance of a builtin type or not?

I only care that it is an instance of a user-defined class, so that the 
Inline::Python module can bless a reference to its memory address, which then 
allows Perl to locate the bound methods in the symbol table.

>>>but checking the TP_HEAPTYPE flag in tp_flags may go
>>>some way towards one.
>>
>> I'll have to look into this.  I'm not sure what that would tell me.
>
>Well, it tells you whether the type objects memory lives on the heap
>or is a static object from the C point of view.  This is potentially
>interesting because it being true is /almost/ synonmous with
>'user-defined type'.

Are there some other serpents lurking in that /almost/ besides the 
"C-extensions" one?   If not, I think we can accept the simple work-around of 
defining an empty python subclass (e.g. 'pass') (see my reply to Phillip's 
post.)

>What you do with subclasses of builtin types like strings is a
>difficult question.  I wouldn't presume to know the best thing to do
>here.

Ideally, I'd like them to be a proper Perl reference to a Perl scalar 
containing the string.  Though this is really more important for tuples and 
lists than it is for strings.  As it is, my new code will identify anything 
which comes from a classic or new-style class (with the caveat (supposedly) 
that the new-style class was not created as a C extension) and return a 
reference to a scalar which contains the memory address of the object (be it 
an instance or a type.)

Yes, this means that classes which subclass tuples or lists will appear as 
opaque objects in Perl (accessible only via their methods), but I think 
that's acceptable.

Thanks,
Eric
-- 
"It ain't those parts of the Bible that I can't understand that 
bother me, it's the parts that I do understand."
                                        --Mark Twain
From pje at telecommunity.com  Mon Jul 12 18:56:33 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Jul 12 18:53:16 2004
Subject: [Python-Dev] PyInstance_Check() and new-style classes
In-Reply-To: <200407121137.40272.ewilhelm@sbcglobal.net>
References: <5.1.1.6.0.20040712115527.030c97a0@mail.telecommunity.com>
	<5.1.1.6.0.20040712093928.01ed4100@mail.telecommunity.com>
	<5.1.1.6.0.20040712115527.030c97a0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040712125138.02de71c0@mail.telecommunity.com>

At 11:37 AM 7/12/04 -0500, Eric Wilhelm wrote:
>So,  I think, with everybody's help, I may have found "the right way" (TM.)
>
>Should this be better documented, or is this an obscure usage?  Will it break
>in future versions of Python?

Non-heap typeness should remain an indicator of written-in-C-ness, although 
written-in-C-ness doesn't guarantee non-heapness.

IOW, if a future version of Python heap-allocates any "builtin" types, your 
test will fail.

However, it's pretty unlikely that any builtin types will ever be heap 
types, because you can't refer to their type structures as constants, 
unless they're statically allocated.

Therefore, truly core types should remain non-heap types as long as the 
current interpreter architecture is valid.

From nbastin at opnet.com  Mon Jul 12 18:56:06 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Mon Jul 12 18:56:22 2004
Subject: [Python-Dev] test__locale failing on MacOS X
Message-ID: <5DE41A46-D424-11D8-91CD-000D932927FE@opnet.com>

The current cvs HEAD is failing test__locale on MacOS X.  Is anyone 
else seeing this?

--
Nick

From Scott.Daniels at Acm.Org  Mon Jul 12 20:05:41 2004
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Mon Jul 12 20:06:53 2004
Subject: [Python-Dev] PyInstance_Check() and new-style classes
In-Reply-To: <20040712164651.926E11E4003@bag.python.org>
References: <20040712164651.926E11E4003@bag.python.org>
Message-ID: <40F2D2F5.9050809@Acm.Org>

Eric Wilhelm <ewilhelm@sbcglobal.net> wrote:
 > I think we can accept the simple work-around of
 > defining an empty python subclass (e.g. 'pass')
 > (see my reply to Phillip's post.)

It is impossible to subclass some classes -- those without
Py_TPFLAGS_BASETYPE set.

-- 
-Scott David Daniels
Scott.Daniels@Acm.Org
From bac at OCF.Berkeley.EDU  Mon Jul 12 20:30:08 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Jul 12 20:30:20 2004
Subject: [Python-Dev] More joy with test_strptime
In-Reply-To: <1f7befae04071120245dedbd21@mail.gmail.com>
References: <20040712002612.39A4D1E4002@bag.python.org>	<40F1E5CC.1000001@ocf.berkeley.edu>	<1f7befae0407111852780f4fc2@mail.gmail.com>	<1f7befae04071120025e3f3233@mail.gmail.com>
	<1f7befae04071120245dedbd21@mail.gmail.com>
Message-ID: <40F2D8B0.2020004@ocf.berkeley.edu>

Tim Peters wrote:

> [Tim]
> ...
> 
>> So now I'm suspecting that the tricks in PthFile are managing to
>>create a second instance of the time module not related to the
>>instance of the time module test_strptime is monkey-patching with its
>>
>>           time.tzname = (tz_name, tz_name)
>>           time.daylight = 1
>>
>>tricks.
> 
> 
> Using the attached patch yields output
> 
> test___all__
> test_site
> test_strptime
> in strptime ('Eastern Standard Time', 'Eastern Daylight Time') 1
> in test_bad_timezone ('Eastern Standard Time', 'Eastern Standard Time') 1
> in strptime ('Eastern Standard Time', 'Eastern Daylight Time') 1
> ...
> 
> so test_strptime's attempt to monkey-patch the time module indeed has
> no effect on the time module _strptime.py is using.
> 

Nuts.  Some code somewhere must be importing strptime through time and 
that is holding a reference to the original copy of 'time' that was 
imported and then when test_site deletes sys.modules["time"] 
test_strptime imports 'time' again since it can't find a reference and 
uses that to monkey-patch which _strptime itself never sees since it 
holds the reference to the original imported 'time'.

And I just re-read your previous email and you came to essentially the 
same conclusion as above.

This just figures; only I could break my own test with another of my own 
tests.

> The problem can be fixed by using this as the first line of test_bad_timezone:
> 
>         from _strptime import time
> 
> But, of course, that's insane.
> 
> 
>> But if so, I'm at a loss to explain why this would only
>>happen on Windows.
> 
> 
> Still so.
> 

Stop wondering; it happens on OS X as well.

OK, so the question becomes how can I have test_site (which I will fix 
with your suggestions for the removal of existing files and directories; 
when I rewrote that chunk of code I had EAFP on the brain) test to make 
sure that a module is imported?  Is there any other way than to just try 
out another module that I think no one else is going mess with that has 
no importation side-effects?  I guess I could have test_site just import 
site again in the .pth file.  Could possibly use _strptime since I know 
I don't mess with that in any way.  But is there some other way to check 
that I am not thinking of?

-Brett
From tim.peters at gmail.com  Mon Jul 12 20:50:35 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Jul 12 20:50:50 2004
Subject: [Python-Dev] More joy with test_strptime
In-Reply-To: <40F2D8B0.2020004@ocf.berkeley.edu>
References: <20040712002612.39A4D1E4002@bag.python.org>	<40F1E5CC.1000001@ocf.berkeley.edu>	<1f7befae0407111852780f4fc2@mail.gmail.com>	<1f7befae04071120025e3f3233@mail.gmail.com>
	<1f7befae04071120245dedbd21@mail.gmail.com>
	<40F2D8B0.2020004@ocf.berkeley.edu>
Message-ID: <1f7befae04071211501fbab08f@mail.gmail.com>

[Brett]
> Nuts.  Some code somewhere must be importing strptime through time and
> that is holding a reference to the original copy of 'time' that was
> imported and then when test_site deletes sys.modules["time"]
> test_strptime imports 'time' again since it can't find a reference and
> uses that to monkey-patch which _strptime itself never sees since it
> holds the reference to the original imported 'time'.

I can't make time to respond now, but I wanted to make sure to get
another copy of that explanation into the archives for posterity
<wink>.
From bac at OCF.Berkeley.EDU  Mon Jul 12 20:54:19 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Jul 12 20:54:28 2004
Subject: [Python-Dev] More joy with test_strptime
In-Reply-To: <1f7befae04071211501fbab08f@mail.gmail.com>
References: <20040712002612.39A4D1E4002@bag.python.org>	<40F1E5CC.1000001@ocf.berkeley.edu>	<1f7befae0407111852780f4fc2@mail.gmail.com>	<1f7befae04071120025e3f3233@mail.gmail.com>
	<1f7befae04071120245dedbd21@mail.gmail.com>
	<40F2D8B0.2020004@ocf.berkeley.edu>
	<1f7befae04071211501fbab08f@mail.gmail.com>
Message-ID: <40F2DE5B.7040806@ocf.berkeley.edu>

Tim Peters wrote:

> [Brett]
> 
>>Nuts.  Some code somewhere must be importing strptime through time and
>>that is holding a reference to the original copy of 'time' that was
>>imported and then when test_site deletes sys.modules["time"]
>>test_strptime imports 'time' again since it can't find a reference and
>>uses that to monkey-patch which _strptime itself never sees since it
>>holds the reference to the original imported 'time'.
> 
> 
> I can't make time to respond now, but I wanted to make sure to get
> another copy of that explanation into the archives for posterity
> <wink>.

=)  Didn't realize I managed to fit that all into a single sentence. 
How you guys can stand to allow me to continue to write the Summaries I 
sometimes question when instances like this come about.

-Brett
From FBatista at uniFON.com.ar  Mon Jul 12 21:58:38 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Mon Jul 12 22:01:54 2004
Subject: [Python-Dev] decimal.py signals & traps
Message-ID: <A128D751272CD411BC9200508BC2194D03383B68@escpl.tcp.com.ar>

[Raymond Hettinger]

#- This should be tackled in steps and as part of larger 
#- improvement to the
#- context API.

Previous to 2.4 final?

I'm very sorry about helping you, but I'm very complicated with my times,
and not sure about my free time from here to several weeks (an example is
the last bugday, I was willing to help, and even downloaded the CVS files
(+3 hours on a 28.8 modem), but the Saturday worked from 8am to 10pm).

Anyway, I trust your decisions.


#- As a first step, I would like change the Context constructor API to
#- match the format in Context.__repr__.  That format is 
#- somewhat easier to
#- use and less verbose (list only the flags and traps that you 
#- want set).

+1





. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informacion contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informacion
confidencial o propietaria, cuya divulgacion es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no esta
autorizado a divulgar, copiar, distribuir o retener informacion (o parte de
ella) contenida en este mensaje. Por favor notifiquenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magnetico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telefonica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electronicos pueden ser alterados, motivo por el cual
Telefonica Comunicaciones Personales S.A. no aceptara ninguna obligacion
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040712/cc98124c/attachment.html
From scav at blueyonder.co.uk  Mon Jul 12 23:12:13 2004
From: scav at blueyonder.co.uk (Peter Harris)
Date: Mon Jul 12 22:59:50 2004
Subject: [Python-Dev] PEP 309?
In-Reply-To: <20040712164648.66DB41E4004@bag.python.org>
References: <20040712164648.66DB41E4004@bag.python.org>
Message-ID: <40F2FEAD.5030209@blueyonder.co.uk>

I know it's a busy time, but is there any chance that PEP 309 could get 
squeezed in some time between 2.4a2 and the final release?  It's only a 
tiny "waffer-theen" Python module, and as far as I know the code, docs 
and tests are ready to go.
Or if there's still some work to do, I'm ready to do it.

Peter Harris

From bob at redivi.com  Mon Jul 12 23:15:38 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Jul 12 23:15:44 2004
Subject: [Python-Dev] PEP 309?
In-Reply-To: <40F2FEAD.5030209@blueyonder.co.uk>
References: <20040712164648.66DB41E4004@bag.python.org>
	<40F2FEAD.5030209@blueyonder.co.uk>
Message-ID: <9F546182-D448-11D8-9C18-000A95686CD8@redivi.com>

On Jul 12, 2004, at 5:12 PM, Peter Harris wrote:

> I know it's a busy time, but is there any chance that PEP 309 could 
> get squeezed in some time between 2.4a2 and the final release?  It's 
> only a tiny "waffer-theen" Python module, and as far as I know the 
> code, docs and tests are ready to go.
> Or if there's still some work to do, I'm ready to do it.

+1

-bob

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040712/a652c694/smime-0001.bin
From pf_moore at yahoo.co.uk  Mon Jul 12 23:17:30 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Mon Jul 12 23:17:13 2004
Subject: [Python-Dev] Re: PEP 309?
References: <20040712164648.66DB41E4004@bag.python.org>
	<40F2FEAD.5030209@blueyonder.co.uk>
Message-ID: <d631lyc5.fsf@yahoo.co.uk>

Peter Harris <scav@blueyonder.co.uk> writes:

> I know it's a busy time, but is there any chance that PEP 309 could
> get squeezed in some time between 2.4a2 and the final release?  It's
> only a tiny "waffer-theen" Python module, and as far as I know the
> code, docs and tests are ready to go.
> Or if there's still some work to do, I'm ready to do it.

It's something I'd use a lot - I reimplement it regularly...

Paul
-- 
This signature intentionally left blank

From jim at zope.com  Mon Jul 12 23:21:31 2004
From: jim at zope.com (Jim Fulton)
Date: Mon Jul 12 23:21:35 2004
Subject: [Python-Dev] Proposal: C API Macro to decref and set to NULL
Message-ID: <40F300DB.6050106@zope.com>


Often, in C type implementations, you want to set a PyObject * slot
to NULL.  Often this goes like:

   Py_XDECREF(self->foo);
   self->foo = NULL;

However, if your type participates in GC and self->foo is something that
might get traversed, this can lead to traversing self->foo after it has been
DECREFed, but before it's set to NULL.

The correct way to do this is something like:

   tmp = self->foo;
   self->foo = NULL;
   Py_XDECREF(tmp);

I suggest that there should be a standard macro to automate this.

   Py_CLEAR(self->foo)

This would be defined to be the same result as Py_XDECREF except
that the argument will be set to NULL.

Thoughts?

If there are no objections I'll add the following definition to object.h,
after the definition for Py_DECREF:

#define Py_CLEAR(op)				\
         do {                            	\
                 if (op) {			\
                         PyObject *tmp = (op);	\
                         (op) = NULL;		\
                         Py_DECREF(tmp);		\
                 }				\
         } while (0)

and update the docs and the tutorial on creating types in C.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From s.percivall at chello.se  Tue Jul 13 00:15:14 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Tue Jul 13 00:15:18 2004
Subject: [Python-Dev] the new profiler and profiling c calls
Message-ID: <F2AD7475-D450-11D8-B04A-0003934AD54A@chello.se>

I think there's a slight problem with the profiling of C calls in
the new profiler.

When profiling method calls, the C function name is the repr()
of the method, which means that a call for instance on
str.endswith on different string instances ends up as completely
separate calls. This might be intentional, but it's different
than the way Python code is profiled, and it's irritating.

Testing if a thousand different lines ends in a certain way means
the default output from profile gets another thousand lines. I don't
really think that's how it should be.

From tim.peters at gmail.com  Tue Jul 13 00:50:31 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Jul 13 00:50:37 2004
Subject: [Python-Dev] Proposal: C API Macro to decref and set to NULL
In-Reply-To: <40F300DB.6050106@zope.com>
References: <40F300DB.6050106@zope.com>
Message-ID: <1f7befae04071215506b00d40c@mail.gmail.com>

[Jim Fulton]
...
> If there are no objections I'll add the following definition to object.h,
> after the definition for Py_DECREF:
> 
> #define Py_CLEAR(op)                            \
>         do {                                   \
>                 if (op) {                      \
>                         PyObject *tmp = (op);  \
>                         (op) = NULL;           \
>                         Py_DECREF(tmp);                \
>                 }                              \
>         } while (0)
> 
> and update the docs and the tutorial on creating types in C.

+1.  At least pystate.c can get rid of its functionally identical
private ZAP macro then.

BTW, re-invented at least as often is a VISIT macro for use in
tp_traverse slots, like typeobject.c's (and several other files')

#define VISIT(SLOT) \
	if (SLOT) { \
		err = visit((PyObject *)(SLOT), arg); \
		if (err) \
			return err; \
	}
From tdelaney at avaya.com  Tue Jul 13 01:54:03 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Tue Jul 13 01:54:10 2004
Subject: [Python-Dev] Proposal: C API Macro to decref and set to NULL
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01ACB8A6@au3010avexu1.global.avaya.com>

Tim Peters wrote:

> BTW, re-invented at least as often is a VISIT macro for use in
> tp_traverse slots, like typeobject.c's (and several other files')
> 
> #define VISIT(SLOT) \
> 	if (SLOT) { \
> 		err = visit((PyObject *)(SLOT), arg); \
> 		if (err) \
> 			return err; \
> 	}

Shouldn't this also be wrapped in a do {} while (0)?

Tim Delaney
From bob at redivi.com  Tue Jul 13 02:37:11 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Jul 13 02:37:18 2004
Subject: [Python-Dev] Proposal: C API Macro to decref and set to NULL
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01ACB8A6@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01ACB8A6@au3010avexu1.global.avaya.com>
Message-ID: <C78307C6-D464-11D8-A847-000A95686CD8@redivi.com>


On Jul 12, 2004, at 7:54 PM, Delaney, Timothy C (Timothy) wrote:

> Tim Peters wrote:
>
>> BTW, re-invented at least as often is a VISIT macro for use in
>> tp_traverse slots, like typeobject.c's (and several other files')
>>
>> #define VISIT(SLOT) \
>> 	if (SLOT) { \
>> 		err = visit((PyObject *)(SLOT), arg); \
>> 		if (err) \
>> 			return err; \
>> 	}
>
> Shouldn't this also be wrapped in a do {} while (0)?

I think it's obvious that this macro has not been made generic yet, 
given that err, visit, and arg aren't defined anywhere.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040712/66f726dc/smime.bin
From tdelaney at avaya.com  Tue Jul 13 02:44:23 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Tue Jul 13 02:44:29 2004
Subject: [Python-Dev] Proposal: C API Macro to decref and set to NULL
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01ACB906@au3010avexu1.global.avaya.com>

Bob Ippolito wrote:

> I think it's obvious that this macro has not been made generic yet,
> given that err, visit, and arg aren't defined anywhere.

That wasn't my point. Since this is in typeobject.c, my point was that
it probably should be made "semi-colon safe" even though it's not
generic yet.

Tim Delaney
From bac at OCF.Berkeley.EDU  Tue Jul 13 02:50:39 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Jul 13 02:50:50 2004
Subject: [Python-Dev] urllib.urlretrieve() and handling 550 errors when
	reading from FTP
In-Reply-To: <200407120355.i6C3tdJ19164@guido.python.org>
References: <40F05047.2030808@ocf.berkeley.edu>
	<200407110020.i6B0KDN01623@guido.python.org>
	<40F0A01C.4040905@ocf.berkeley.edu>
	<200407111627.i6BGRCD18505@guido.python.org>
	<40F19043.3080108@ocf.berkeley.edu>
	<200407120355.i6C3tdJ19164@guido.python.org>
Message-ID: <40F331DF.5090401@ocf.berkeley.edu>

Guido van Rossum wrote:

[SNIP]
>>If no one has any objections I will just clarify the docs 
>>stating that if an attempt to get a file fails on a 550 error (and of 
>>course I will say what that means) it will then try a LIST command and 
>>if that succeeds that is what is returned.  And if you need more 
>>fine-grained control then use ftplib.
> 
> 
> Sounds good to me.  Thanks!

OK, in as rev. 1.53 for Doc/lib/liburllib.tex in the "Restrictions" 
section.  The wording seems fine to me, but after my already glorious 
email on why test_strptime was faililng, I would appreciate someone else 
reading and making sure it makes sense.  Once people have cleared it for 
public consumption I will backport it.

-Brett
From guido at python.org  Tue Jul 13 03:08:08 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul 13 03:08:15 2004
Subject: [Python-Dev] the new profiler and profiling c calls
In-Reply-To: Your message of "Tue, 13 Jul 2004 00:15:14 +0200."
	<F2AD7475-D450-11D8-B04A-0003934AD54A@chello.se> 
References: <F2AD7475-D450-11D8-B04A-0003934AD54A@chello.se> 
Message-ID: <200407130108.i6D188822616@guido.python.org>

> I think there's a slight problem with the profiling of C calls in
> the new profiler.
> 
> When profiling method calls, the C function name is the repr()
> of the method, which means that a call for instance on
> str.endswith on different string instances ends up as completely
> separate calls. This might be intentional, but it's different
> than the way Python code is profiled, and it's irritating.
> 
> Testing if a thousand different lines ends in a certain way means
> the default output from profile gets another thousand lines. I don't
> really think that's how it should be.

I saw a checkin that changed this to repr() and I didn't understand
the point either, but I can easily be convinced that that was the
wrong fix for whatever it was trying to fix.

(Sorry, no time for more detail.  CVS log is your friend. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From nbastin at opnet.com  Tue Jul 13 03:14:58 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Tue Jul 13 03:15:20 2004
Subject: [Python-Dev] the new profiler and profiling c calls
In-Reply-To: <200407130108.i6D188822616@guido.python.org>
References: <F2AD7475-D450-11D8-B04A-0003934AD54A@chello.se>
	<200407130108.i6D188822616@guido.python.org>
Message-ID: <0ED1529A-D46A-11D8-91CD-000D932927FE@opnet.com>


On Jul 12, 2004, at 9:08 PM, Guido van Rossum wrote:

>> I think there's a slight problem with the profiling of C calls in
>> the new profiler.
>>
>> When profiling method calls, the C function name is the repr()
>> of the method, which means that a call for instance on
>> str.endswith on different string instances ends up as completely
>> separate calls. This might be intentional, but it's different
>> than the way Python code is profiled, and it's irritating.
>>
>> Testing if a thousand different lines ends in a certain way means
>> the default output from profile gets another thousand lines. I don't
>> really think that's how it should be.
>
> I saw a checkin that changed this to repr() and I didn't understand
> the point either, but I can easily be convinced that that was the
> wrong fix for whatever it was trying to fix.

That was my bad.  It's been fixed.

--
Nick

From tim.peters at gmail.com  Tue Jul 13 03:24:29 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Jul 13 03:24:32 2004
Subject: [Python-Dev] Proposal: C API Macro to decref and set to NULL
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01ACB906@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01ACB906@au3010avexu1.global.avaya.com>
Message-ID: <1f7befae04071218242de3feca@mail.gmail.com>

[Bob Ippolito]
>> I think it's obvious that this macro has not been made generic yet,
>> given that err, visit, and arg aren't defined anywhere.

[Delaney, Timothy C (Timothy)]
> That wasn't my point. Since this is in typeobject.c, my point was that
> it probably should be made "semi-colon safe" even though it's not
> generic yet.

I don't see what's special about typeobject.c.  Since it's not in a
header file, only typeobject.c can see it.  All uses of it within
typeobject.c are correct, and there's nothing wrong with writing a
"good enough" macro for local sane use.

defending-guido-is-an-endless-task<wink>-ly y'rs  - tim
From tdelaney at avaya.com  Tue Jul 13 03:48:28 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Tue Jul 13 03:48:34 2004
Subject: [Python-Dev] Proposal: C API Macro to decref and set to NULL
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01B47851@au3010avexu1.global.avaya.com>

Tim Peters wrote:

> I don't see what's special about typeobject.c.  Since it's not in a
> header file, only typeobject.c can see it.  All uses of it within
> typeobject.c are correct, and there's nothing wrong with writing a
> "good enough" macro for local sane use.

Sorry - you're right. I'm getting myself confused about what's where.
I'm going to blame the long, boring and completely unproductive meeting
I was forced to attend this morning ...because obviously the blame
*couldn't* like with me ...

Tim Delaney
From skip at pobox.com  Tue Jul 13 04:36:32 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Jul 13 04:36:39 2004
Subject: [Python-Dev] Proposal: C API Macro to decref and set to NULL
In-Reply-To: <40F300DB.6050106@zope.com>
References: <40F300DB.6050106@zope.com>
Message-ID: <16627.19120.566581.651835@montanaro.dyndns.org>


    Jim> The correct way to do this is something like:

    Jim>    tmp = self->foo;
    Jim>    self-> foo = NULL;
    Jim>    Py_XDECREF(tmp);

    Jim> I suggest that there should be a standard macro to automate this.

    Jim>    Py_CLEAR(self->foo)

    Jim> This would be defined to be the same result as Py_XDECREF except
    Jim> that the argument will be set to NULL.

    Jim> Thoughts?

Just my two cents, but the name suggests to me that it's doing more than a
simple decref and set to NULL.  I can't think of anything else off the top
of my head that seems obviously better though.

Skip
From greg at cosc.canterbury.ac.nz  Tue Jul 13 05:06:46 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Jul 13 05:06:53 2004
Subject: [Python-Dev] PyInstance_Check() and new-style classes
In-Reply-To: <200407121137.40272.ewilhelm@sbcglobal.net>
Message-ID: <200407130306.i6D36kVW002433@cosc353.cosc.canterbury.ac.nz>

Eric Wilhelm <ewilhelm@sbcglobal.net>:

> +       if ((obj->ob_type->tp_flags & Py_TPFLAGS_HEAPTYPE) || 
>           PyInstance_Check(obj)) {

I don't understand why you need to make this check.
Aren't you going to have to check individually anyway for all
the types you want to treat specially? If so, why not just

  if (PyString_CheckExact(obj)) {
    /* convert it to a Perl string */
  }
  elif (PyInt_CheckExact(obj)) {
    /* convert it to a Perl integer */
  }
  ...
  else {
    /* None of the above, wrap it generically */
  }

Using the CheckExact macros will ensure that you don't pick
up any subclasses of the built-in types.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Tue Jul 13 05:08:45 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Jul 13 05:08:51 2004
Subject: [Python-Dev] decimal.py signals & traps
In-Reply-To: <2m8ydps14h.fsf@starship.python.net>
Message-ID: <200407130308.i6D38jl2002458@cosc353.cosc.canterbury.ac.nz>

Michael Hudson <mwh@python.net>:

> Well, there's an argument that this may not be the most natural
> interface for a C implementation or for an implementation of such
> things for floats -- but this is an argument involving predicting the
> future, so probably should be ignored.

I don't think it should be ignored. If the future is uncertain, surely
it would be better to err on the side of *not* exposing implementation
details?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Tue Jul 13 05:18:35 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Jul 13 05:18:40 2004
Subject: [Python-Dev] Proposal: C API Macro to decref and set to NULL
In-Reply-To: <16627.19120.566581.651835@montanaro.dyndns.org>
Message-ID: <200407130318.i6D3IZLq002552@cosc353.cosc.canterbury.ac.nz>

Skip Montanaro <skip@pobox.com>:

> Just my two cents, but the name suggests to me that it's doing more than a
> simple decref and set to NULL.  I can't think of anything else off the top
> of my head that seems obviously better though.

Py_XDECREF_NULL?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From michael.mangino at comcast.net  Tue Jul 13 05:20:37 2004
From: michael.mangino at comcast.net (Mike Mangino)
Date: Tue Jul 13 05:20:41 2004
Subject: [Python-Dev] Developer looking to help fix bugs
Message-ID: <1089688837.2657.5.camel@localhost.localdomain>

Hello,

My name is Mike Mangino. I am an IT weenie in Chicago Illinois, USA who
has used python in various projects for several years. I have recently
finished school (again) and am looking for a project I can work on to
learn. My huge ambition is to eventually port python to the palm pilot
with a reasonable graphical interface. I have learned my lesson in the
past about jumping into ambitious projects too quickly. As such, I am
interested fixing and triaging bugs in the sourceforge tracker. 

Earlier today I added a comment to http://www.python.org/sf/982679 to
tell the user how to fix the problem they were having (it was a bug in
their code, not python) How do I have somebody close that request? In
the future, if I fix one of the issues, is there somebody I should email
to make changes?

I look forward to helping out in whatever way I can. I have experience
in C, Java, databases and many other languages in addition to an MBA
with focus on Finance. My guess is that the C will be most helpful, but
who knows. If you want to discuss asset pricing, I can probably help
with that as well.

If anyone is in the Chicago area, I would love to buy you a beer and
pick your brain.

Mike Mangino


From tim.peters at gmail.com  Tue Jul 13 05:41:02 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Jul 13 05:41:17 2004
Subject: [Python-Dev] decimal.py signals & traps
In-Reply-To: <200407130308.i6D38jl2002458@cosc353.cosc.canterbury.ac.nz>
References: <200407130308.i6D38jl2002458@cosc353.cosc.canterbury.ac.nz>
Message-ID: <1f7befae04071220411ddbcd15@mail.gmail.com>

[Michael Hudson] 
>> Well, there's an argument that this may not be the most natural
>> interface for a C implementation or for an implementation of such
>> things for floats -- but this is an argument involving predicting the
>> future, so probably should be ignored.

[Greg Ewing]
> I don't think it should be ignored. If the future is uncertain, surely
> it would be better to err on the side of *not* exposing implementation
> details?

Since virtually all modern processors implement sticky flags, trap
enablers, and rounding modes in FPU context & control registers,
virtually all modern compilers support some idiosyncratic way to set
and query them.  Google on ieee_flags for one extreme in C, entirely
string-based.  Google on _control87 and/or _controlfp for the other C
extreme, passing ints, with a pile of #defines to give different bits
different meanings.

Both of those actually hide the implementation quite well -- although
the latter is most often implemented in such a way that the bit
positions remarkably <ahem> correspond exactly to their meanings in
the Pentium instructions that fiddle FPU context.
From martin at v.loewis.de  Tue Jul 13 05:48:39 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Jul 13 05:48:43 2004
Subject: [Python-Dev] Developer looking to help fix bugs
In-Reply-To: <1089688837.2657.5.camel@localhost.localdomain>
References: <1089688837.2657.5.camel@localhost.localdomain>
Message-ID: <40F35B97.2010903@v.loewis.de>

Mike Mangino wrote:
> As such, I am
> interested fixing and triaging bugs in the sourceforge tracker. 

That offer is very welcome!

> Earlier today I added a comment to http://www.python.org/sf/982679 to
> tell the user how to fix the problem they were having (it was a bug in
> their code, not python) How do I have somebody close that request? 

I suggest you maintain a list of bug reports and patches for which
you have a proposed action. Post that list to python-dev say, weekly,
and somebody will pick it up.

In the specific case, I just closed it.

> I look forward to helping out in whatever way I can. I have experience
> in C, Java, databases and many other languages in addition to an MBA
> with focus on Finance. My guess is that the C will be most helpful, but
> who knows. 

Personally, I believe patch reviews might help most at this point:
take some of the old patches, and evaluate them. Find out whether they
do what they say they do, and whether they do it correctly. If they
fix a bug, determine whether what they change really is a bug, and
whether the fix won't break existing code. Also check whether a test
case accompanies the fix. If the patch adds a new feature, determine
whether the feature is desirable, and whether it comes with
documentation and test cases. Put your analysis as a comment in
the patch.

If the submitter is unresponsive, determine whether the patch is
worthwhile fixing yourself. If not, add a message indicating that
you recommend to reject the patch.

If you have a list of patches that you have reviewed and for which
you recommend approval or rejection, post them to python-dev. If
I'm the one to execute your proposals, I will check a few reviews
in detail in order to establish trust in your analysis, and later
bulk-apply changes if your analysis is plausible.

The same would hold for bug reports, except that in cases where
a fix is needed, you will have to add a patch to the patches
tracker, which would only add to the backlog of unreviewed
patches. Of course, if the bug is serious, you might still do
so, and post a list of bugs along with the list of your patches
(always link bug and patch in comment messages also).

> If anyone is in the Chicago area, I would love to buy you a beer and
> pick your brain.

Thanks for the offer - I'm on a different continent, unfortunately.

Regards,
Martin

From guido at python.org  Tue Jul 13 06:27:07 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul 13 06:27:16 2004
Subject: [Python-Dev] Proposal: C API Macro to decref and set to NULL
In-Reply-To: Your message of "Tue, 13 Jul 2004 15:18:35 +1200."
	<200407130318.i6D3IZLq002552@cosc353.cosc.canterbury.ac.nz> 
References: <200407130318.i6D3IZLq002552@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200407130427.i6D4R7H22845@guido.python.org>

> > Just my two cents, but the name suggests to me that it's doing
> > more than a simple decref and set to NULL.  I can't think of
> > anything else off the top of my head that seems obviously better
> > though.
> 
> Py_XDECREF_NULL?

Bleah.

In pystate.c you'll find:

#define ZAP(x) { \
	PyObject *tmp = (PyObject *)(x); \
	(x) = NULL; \
	Py_XDECREF(tmp); \
}

Call it Py_ZAP and you have my vote. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tim.peters at gmail.com  Tue Jul 13 07:28:48 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Jul 13 07:28:51 2004
Subject: [Python-Dev] More joy with test_strptime
In-Reply-To: <1f7befae04071120025e3f3233@mail.gmail.com>
References: <20040712002612.39A4D1E4002@bag.python.org>
	<40F1E5CC.1000001@ocf.berkeley.edu>
	<1f7befae0407111852780f4fc2@mail.gmail.com>
	<1f7befae04071120025e3f3233@mail.gmail.com>
Message-ID: <1f7befae0407122228727680c1@mail.gmail.com>

[Tim]
> FWIW, this is a minimal failing set of tests to run:
>
> test___all__
> test_site
> test_strptime

More, if you comment out the

        self.check_all("_strptime")

line in test__all__, the test passes.  If you comment out every
self.check_all line except for that one, the test fails.  If you add
this line after that one, the test passes:

        del sys.modules["_strptime"]

The regrtest framework itself tries to "unload" modules imported by
tests, but it only does so for modules from the test *package*.  So
_strptime remains loaded after test___all__, with the time module it
had at that time.  I really don't grok what test_site is doing, so am
at a loss to explain how it manages to create a distinct time module.

Ah!  It doesn't!  It's because PthFile.cleanup does

                del sys.modules[self.imported]

That destroys the time module that was in sys.modules before test_site
began running.  That's the problem.  Then every module that previously
imported time via test___all__ has a time module distinct from any
time module imported after test_site.

Get rid of that line.  Or if, you want to be anally correct, nuke
self.imported in cleanup() if and only if

    self.imported not in sys.modules

before PthFile imported self.imported

Note that the

        assert self.imported in sys.modules

in test() doesn't do anything useful right now, because time was in
sys.modules before the test ran, so it's not testing that the .pth
actually imported time.

Note too that unittests should not use assert -- they don't test
anything then in an -O run.  You can use

     self.assert_(self.imported in sys.modules)

instead, and that works fine with -O.
From tim.peters at gmail.com  Tue Jul 13 07:33:18 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Jul 13 07:33:24 2004
Subject: [Python-Dev] More joy with test_strptime
In-Reply-To: <40F2D8B0.2020004@ocf.berkeley.edu>
References: <20040712002612.39A4D1E4002@bag.python.org>	<40F1E5CC.1000001@ocf.berkeley.edu>	<1f7befae0407111852780f4fc2@mail.gmail.com>	<1f7befae04071120025e3f3233@mail.gmail.com>
	<1f7befae04071120245dedbd21@mail.gmail.com>
	<40F2D8B0.2020004@ocf.berkeley.edu>
Message-ID: <1f7befae040712223326b78d19@mail.gmail.com>

[Brett]
> OK, so the question becomes how can I have test_site (which I will fix
> with your suggestions for the removal of existing files and directories;
> when I rewrote that chunk of code I had EAFP on the brain) test to make
> sure that a module is imported?

By making sure the module it imports isn't in sys.modules before it
tries to import it.  Beforehand:

    old_version = sys.modules.get(self.imported)
    if old_version is not None:
        del sys.modules[self.imported]

Then make sure the .pth file puts self.imported back in sys.modules.

At the end,

    del sys.modules[self.imported]
    if old_version is not None:
        sys.modules[self.imported] = old_version
From nbastin at opnet.com  Tue Jul 13 08:26:12 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Tue Jul 13 08:26:26 2004
Subject: [Python-Dev] test__locale weirdness
Message-ID: <89018A21-D495-11D8-91CD-000D932927FE@opnet.com>

Brett and I spent the better part of the evening (and my morning) 
trying to figure out what's making test__locale fail on MacOS X.  The 
test has been failing since the changes made in Modules/_localemodule.c 
revision 2.46.  These changes caused nl_langinfo() to return the 
correct value for RADIXCHAR (among others), which it wasn't doing 
before.  However, now nl_langinfo(RADIXCHAR) and 
localeconv()['decimal_point'] don't agree:

Take 'fr_FR' for example:

_localemodule.c:2.45:
 >>> setlocale(LC_NUMERIC, 'fr_FR')
'fr_FR'
 >>> nl_langinfo(RADIXCHAR)
'.'
 >>> localeconv()['decimal_point']
'.'

_localemodule.c:2.46:
 >>> setlocale(LC_NUMERIC, 'fr_FR')
'fr_FR'
 >>> nl_langinfo(RADIXCHAR)
','
 >>> localeconv()['decimal_point']
'.'

The changes made in 2.46 are closer to correct than 2.45, but we don't 
understand how localeconv() is incorrect.  If you stop in 
PyLocale_localeconv in the debugger, you can see that the locale is 
indeed set to 'fr_FR', but localeconv() has decimal_point as '.', not 
comma.  You could write this off as a bug in the c library, but if you 
write a very basic program, it works fine:

int
main (int argc, char *argv[])
{
struct lconv *result;

if (!setlocale(LC_NUMERIC, "fr_FR")) {
   printf("setlocale() failed\n");
   exit(1);
}
if (!( result = localeconv() )) {
   printf("localeconv() failed\n");
   exit(1);
}

printf("Claimed locale:%s\n", setlocale(LC_NUMERIC, NULL));
printf("decimal point: '%s', thousands_sep: '%s'\n", 
result->decimal_point,
result->thousands_sep);

return 0;
}

displays:

Claimed locale:fr_FR
decimal point: ',', thousands_sep: ''

Which is exactly what you'd expect from a working implementation.  Does 
anybody know what Python is doing beyond what this simple test does?

--
Nick

From bac at OCF.Berkeley.EDU  Tue Jul 13 09:04:03 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Jul 13 09:04:14 2004
Subject: [Python-Dev] More joy with test_strptime
In-Reply-To: <1f7befae0407122228727680c1@mail.gmail.com>
References: <20040712002612.39A4D1E4002@bag.python.org>
	<40F1E5CC.1000001@ocf.berkeley.edu>
	<1f7befae0407111852780f4fc2@mail.gmail.com>
	<1f7befae04071120025e3f3233@mail.gmail.com>
	<1f7befae0407122228727680c1@mail.gmail.com>
Message-ID: <40F38963.6010105@ocf.berkeley.edu>

Tim Peters wrote:
> [Tim]
[SNIP]
> The regrtest framework itself tries to "unload" modules imported by
> tests, but it only does so for modules from the test *package*.  So
> _strptime remains loaded after test___all__, with the time module it
> had at that time.  I really don't grok what test_site is doing, so am
> at a loss to explain how it manages to create a distinct time module.
> 
> Ah!  It doesn't!  It's because PthFile.cleanup does
> 
>                 del sys.modules[self.imported]
> 
> That destroys the time module that was in sys.modules before test_site
> began running.  That's the problem.  Then every module that previously
> imported time via test___all__ has a time module distinct from any
> time module imported after test_site.
> 

Yep.

> Get rid of that line.  Or if, you want to be anally correct, nuke
> self.imported in cleanup() if and only if
> 
>     self.imported not in sys.modules
> 
> before PthFile imported self.imported
> 
> Note that the
> 
>         assert self.imported in sys.modules
> 

[Tim - in another email]
 > Then make sure the .pth file puts self.imported back in sys.modules.

Using that solution solves the problem!  Now I just store a reference 
before it is deleted and have a flag that signifies if it is cleaning up 
to prep for running the test or is playing cleanup from the test having 
been run.  That signifies whether the module being checked for is 
deleted or being restored from the reference stored in the PthFile instance.

> in test() doesn't do anything useful right now, because time was in
> sys.modules before the test ran, so it's not testing that the .pth
> actually imported time.
> 

That's why I called PthFile.cleanup() before even handling the .pth 
file; that deleted the entry from sys.modules before the test ran.

> Note too that unittests should not use assert -- they don't test
> anything then in an -O run.  You can use
> 
>      self.assert_(self.imported in sys.modules)
> 

The reason I did it that way was because I didn't want the hassle of 
subclassing unittest.TestCase for PthFile.  But with this assert issue I 
just moved that code into the test case class that had the functions 
calling PthFile so I could get access to self.failUnless().  Figured I 
really don't need to have the test code with the PthFile class.

Thanks for your help, Tim.  I will try not to break more code for a 
while.  =)

-Brett
From walter at livinglogic.de  Tue Jul 13 11:26:25 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue Jul 13 11:26:28 2004
Subject: [Python-Dev] test_pwd is really slow
In-Reply-To: <200407121626.i6CGQFl21951@guido.python.org>
References: <e8bf7a5304071205377772a232@mail.gmail.com>
	<200407121626.i6CGQFl21951@guido.python.org>
Message-ID: <40F3AAC1.6060900@livinglogic.de>

Guido van Rossum wrote:
>>I've been having trouble running the tests lately, because test_pwd is
>>really slow.  It's a fairly trivial set of tests -- checking the types
>>of return values and cross-checking getpwall() vs.
>>getpwnam()/getpwuid().  The problem is if your password database is
>>provided by LDAP and is really big, the test takes an impractically
>>long time.
>>
>>Would anyone object if I changed the test suite to require some
>>resource to run test_pwd?  Or should I just make some local change to
>>disable it?
> 
> 
> How about instead limiting the number of entries checked to the first
> 100?  Try changing
> 
>         entries = pwd.getpwall()
> 
> into
> 
>         entries = pwd.getpwall()[:100]

If the bottleneck is in the n calls to pwd.getpwnam() and pwd.getpwuid()
limiting n is an option. If the bottleneck is the one call to
pwd.getpwall() adding a resource requirement might be the only option.

Bye,
    Walter D?rwald

From jim at zope.com  Tue Jul 13 12:45:32 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 13 12:45:35 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to decref and set
	to NULL)
In-Reply-To: <1f7befae04071215506b00d40c@mail.gmail.com>
References: <40F300DB.6050106@zope.com>
	<1f7befae04071215506b00d40c@mail.gmail.com>
Message-ID: <40F3BD4C.70701@zope.com>

Tim Peters wrote:
> [Jim Fulton]
> ...
> 

...

> BTW, re-invented at least as often is a VISIT macro for use in
> tp_traverse slots, like typeobject.c's (and several other files')
> 
> #define VISIT(SLOT) \
> 	if (SLOT) { \
> 		err = visit((PyObject *)(SLOT), arg); \
> 		if (err) \
> 			return err; \
> 	}

First, I don't like this macro, based on my own experience writing macros
that hide returns. :)

Second, Aaargh!  I missunderstood the use of the visit function
passed to traverse.  The source of my missunderstanding was

   1) visit is an int function and I normally expect int functions to
      return a negative value to indicate an error and

   2) The documentation for the traversal slot says:
      "If visit returns a non-zero value then an error has occurred
      and that value should be returned immediately."
      That is, the documentation says that the return value is an
      error indicator.  I missed the bit about a non-zero return value
      and needing to return it. My bad.

Last night, Tim explained to me that a non-zero return value is not an
error indicator.  In fact, the GC system doesn't really allow errors.
Rather, a non-zero return value provides a way for a visit proc to
short-circuit the traversal process.  As far as Tim knows, this feature
has never been used.  All visit functions in the core always return 0.

Alas, all my extensions that implement tp_traverse and the documentation
I wrote on writing types is wrong and has to be changed. Sigh.

Should we call WHUI (we haven't used it!) on this feature that has been around
for 4 versions of Python, that complicates tp_traverse implementations and
hasn't been used? It would be simpler if we said that the return value could
be ignored.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From jhylton at gmail.com  Tue Jul 13 14:41:34 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Tue Jul 13 14:41:36 2004
Subject: [Python-Dev] test_pwd is really slow
In-Reply-To: <40F3AAC1.6060900@livinglogic.de>
References: <e8bf7a5304071205377772a232@mail.gmail.com>
	<200407121626.i6CGQFl21951@guido.python.org>
	<40F3AAC1.6060900@livinglogic.de>
Message-ID: <e8bf7a5304071305412cf56d0a@mail.gmail.com>

On Tue, 13 Jul 2004 11:26:25 +0200, Walter D?rwald
<walter@livinglogic.de> wrote:
> If the bottleneck is in the n calls to pwd.getpwnam() and pwd.getpwuid()
> limiting n is an option. If the bottleneck is the one call to
> pwd.getpwall() adding a resource requirement might be the only option.

I'll investigate and see which it is.

Jeremy
From pinard at iro.umontreal.ca  Tue Jul 13 15:04:39 2004
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Tue Jul 13 15:10:26 2004
Subject: [Python-Dev] PEP 292 for Python 2.4
In-Reply-To: <20040624142741.GC27922@panix.com>
References: <1087414523.7150.45.camel@localhost>
	<001f01c4538d$f1fcfec0$bfaf2c81@oemcomputer>
	<20040624142741.GC27922@panix.com>
Message-ID: <20040713130439.GA2174@titan.progiciels-bpi.ca>

[Aahz]
> [Raymond Hettinger]

> > Please do give consideration to putting all of this in a single
> > module.  IMO, this is too small of an addition to warrant splitting
> > everything in to packages (which make it more difficult to
> > understand and maintain as a collective unit).

> That's true.  However, there has been a regular low-level discussion
> about creating a ``text`` package; why not simply name it ``string``?

Please do not use, as package names, identifiers that users would likely
want to keep for themselves. `text' and `string' are bad ideas for
package names. `stringlib' seems much more likely do not hurt people.

I know that `string' and `socket' exist, despite `string' is evanescent,
but they surely forced users at choosing other identifiers where `string'
and `socket' would have been perfect.  It is very good news that, now
in Python 2.3, `string' is unneeded most of times.  Let us not repeat
previous mistakes, or even nail them further by trying to be compatible
with them.

-- 
Fran?ois Pinard   http://www.iro.umontreal.ca/~pinard
From pinard at iro.umontreal.ca  Tue Jul 13 15:29:50 2004
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Tue Jul 13 15:50:46 2004
Subject: [Python-Dev] PEP 292 - simpler string substitutions
In-Reply-To: <1080100790.32228.26.camel@geddy.wooz.org>
References: <1080100790.32228.26.camel@geddy.wooz.org>
Message-ID: <20040713132950.GB2174@titan.progiciels-bpi.ca>

[Barry Warsaw]

> http://www.python.org/peps/pep-0292.html

> The reference implementation in the patch adds a couple of extra
> things that aren't part of the PEP, but were discussed at Monday's
> sprint.  Specifically:

> - a top level package called 'stringlib' is added to the standard
> library.

> - the dstring class implemented PEP 292 is available through
> stringlib.dstring

> - some handy dict subclasses that work with the dstring class are also
> provided.

> If this stuff is approved for Python 2.4, I'll add test cases and
> documentation.

This is more generic remarks about the whole PEP process than anything
specific about the above work.

Would such stuff be "approved" without going through the PEP first?
Should not PEPs represent the full idea, the summary of discussions, and
adequately represent the extent of what is being approved, or not?

I sometimes have the feeling that PEPs are used, or avoided, depending
on the barometric pressure. :-) Maybe they are used to punch holes
in the decision process.  Once a hole exists, trucks may go through.
Should not PEPs be updated and completed, in particular, when what they
describe gets implemented differently, at least saving for the posterity
the reasons of the differences?  Should not this be mandatorily done
_before_ inclusion of a feature into a release?  If not that way, many
PEPs are going to stay forever incomplete.

-- 
Fran?ois Pinard   http://www.iro.umontreal.ca/~pinard
From jim at zope.com  Tue Jul 13 16:02:21 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 13 16:02:25 2004
Subject: [Python-Dev] _bsddb too unstable?
Message-ID: <40F3EB6D.8020107@zope.com>


I just wasted some time chasing down being unable to run the
Python test suite because the test_anydbm tests were segfaulting.
Removing the _bsddb module allowed the tests to run without
segfaulting.

(FWIW, I'm running on Fedora Core 1, with db4-4.1.25-14)

I get the impression that this is a common problem.
Berkeley DB APIs often change in non-backward compatable
ways that cause this sort of problem.  Perhaps these APIs
change too rapidly to allow the _bsddb extension to be included
in the distribution.  I assume that this is the fault of
the Berkeley DB libraries, and not the extension, but I don't
really know. :/

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From pinard at iro.umontreal.ca  Tue Jul 13 16:08:07 2004
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Tue Jul 13 16:25:24 2004
Subject: [Python-Dev] Can we please have a better dict interpolation
	syntax?
In-Reply-To: <1066920795.11634.96.camel@anthem>
References: <200310230332.h9N3WIm20194@oma.cosc.canterbury.ac.nz>
	<200310230416.h9N4GJv01528@12-236-54-216.client.attbi.com>
	<20031023123153.GA20072@alcyon.progiciels-bpi.ca>
	<1066920795.11634.96.camel@anthem>
Message-ID: <20040713140807.GC2174@titan.progiciels-bpi.ca>

Hi again.  Getting back to much older messages about `$'-interpolation. :-)

[Barry Warsaw]
> [Fran?ois Pinardi]:
> > [Guido van Rossum]
> > > > > Wouldn't this be even better?
> > > > >     "create index ${table}_lid1_idx on $table($lid1)" % params

> > "Better" because it uses `$' instead of `%'? It is really a matter
> > of taste and aesthetics, more than being "better" on technical
> > grounds.  Technically, the multiplication of aspects and paradigms
> > goes against some unencumberance and simplicity, which made Python
> > attractive to start with.  We would loose something probably not
> > worth the gain.

> Better because the trailing type specifier on %-strings is extremely
> error prone (#1 cause of bugs for Mailman translators is/was leaving
> off the trailing 's').

Such errors are usually caught much more solidly, for example, right
in `msgfmt'.  At least for C, and most likely for Python as well,
it is kind of catastrophic for an application that its translation
file has errors regarding formats.  If a bad PO file crashes an
application, maintainers and users will get mad at internationalisation.
Forgetting a trailing `s' is one error, misspelling a variable name is
another: both should absolutely be caught before the release of an
internationalised application.  Using $-strings instead of %-strings is
far from adequately addressing the real problem.  Translators, despite
all their good will, cannot be trusted to never make errors, and they
usually use `msgfmt' to validate their own PO files before transmission
(or indirectly rely on robots for doing that validation for them).
Whatever they use $-strings or %-strings, validation is necessary.

> Better because the rules for $-strings are simple and easy to explain.

I just read PEP 292 again, and a mapping is necessarily provided, this
evacuates a lot of questions about how variables would be accessed out
of local and global scopes -- this simplifies things a lot.  The PEP
does not say what the behaviour of the substitution when an identifier
is not part of the mapping, and this is an important issue.

At one place, the PEP says that `dstring' could be sub-classed to get
different behaviour, like allowing dots for attribute access, but
elsewhere, it also implies that within "${identifier}", "identifier"
has to be an identifier, that is, no dots.  And if "identifier" could
contain special characters like dots or brackets, it does not say if
brackets may be nested nor if they have to balance (like they apparently
and nicely do with `%' interpolation).

It does not seem all that simple and easy for me.  Granted it could have
been much more difficult.

> And yes, better because it uses $ instead of %; it just seems that
> more people grok that $foo is a placeholder.

Yet, users of `$' in other languages or scripts do not have to
explicitly provide a mapping, so the similarity stays a bit superficial.
But if this makes a few more users happy, and being in a library, stays
away from the Python language, `$-strings' may indeed serve a purpose.

-- 
Fran?ois Pinard   http://www.iro.umontreal.ca/~pinard
From jim at zope.com  Tue Jul 13 16:57:31 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 13 16:57:35 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to decref and
	set to NULL)
In-Reply-To: <40F3BD4C.70701@zope.com>
References: <40F300DB.6050106@zope.com>
	<1f7befae04071215506b00d40c@mail.gmail.com>
	<40F3BD4C.70701@zope.com>
Message-ID: <40F3F85B.70409@zope.com>

Never mind.

Tim pointed out to me that return values other than 0 for the
visit proc are used.

The documentation is still off I think, as it says that a
non-zero value indicates an error, but I don't think that this
is true.

Jim



Jim Fulton wrote:
> Tim Peters wrote:
> 
>> [Jim Fulton]
>> ...
>>
> 
> ...
> 
>> BTW, re-invented at least as often is a VISIT macro for use in
>> tp_traverse slots, like typeobject.c's (and several other files')
>>
>> #define VISIT(SLOT) \
>>     if (SLOT) { \
>>         err = visit((PyObject *)(SLOT), arg); \
>>         if (err) \
>>             return err; \
>>     }
> 
> 
> First, I don't like this macro, based on my own experience writing macros
> that hide returns. :)
> 
> Second, Aaargh!  I missunderstood the use of the visit function
> passed to traverse.  The source of my missunderstanding was
> 
>   1) visit is an int function and I normally expect int functions to
>      return a negative value to indicate an error and
> 
>   2) The documentation for the traversal slot says:
>      "If visit returns a non-zero value then an error has occurred
>      and that value should be returned immediately."
>      That is, the documentation says that the return value is an
>      error indicator.  I missed the bit about a non-zero return value
>      and needing to return it. My bad.
> 
> Last night, Tim explained to me that a non-zero return value is not an
> error indicator.  In fact, the GC system doesn't really allow errors.
> Rather, a non-zero return value provides a way for a visit proc to
> short-circuit the traversal process.  As far as Tim knows, this feature
> has never been used.  All visit functions in the core always return 0.
> 
> Alas, all my extensions that implement tp_traverse and the documentation
> I wrote on writing types is wrong and has to be changed. Sigh.
> 
> Should we call WHUI (we haven't used it!) on this feature that has been 
> around
> for 4 versions of Python, that complicates tp_traverse implementations and
> hasn't been used? It would be simpler if we said that the return value 
> could
> be ignored.
> 
> Jim
> 


-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From tim.peters at gmail.com  Tue Jul 13 17:10:54 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Jul 13 17:11:11 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to decref and
	set to NULL)
In-Reply-To: <40F3BD4C.70701@zope.com>
References: <40F300DB.6050106@zope.com>
	<1f7befae04071215506b00d40c@mail.gmail.com>
	<40F3BD4C.70701@zope.com>
Message-ID: <1f7befae040713081023e68654@mail.gmail.com>

[Jim Fulton, on non-zero return values from a visitproc]
> ...
> Should we call WHUI (we haven't used it!) on this feature that has been around
> for 4 versions of Python, that complicates tp_traverse implementations and
> hasn't been used? It would be simpler if we said that the return value could
> be ignored.

As you and I (but not python-dev) discussed in later email, the
non-zero return gimmick is used by two visitprocs in gcmodule.c after
all.  To match actual uses, instead of

    If visit returns a non-zero value then an error has occurred and
that value should
    be returned immediately

the docs should say

    If visit returns a non-zero value that value should be returned immediately

I believe the latter matches the original design intent too.

I doubt that anyone outside the core has written a visitproc.  If so,
all existing uses could just as well be met by

    If visit returns a non-zero value then 1 should be returned immediately

but that wouldn't really save any cycles.

Note that it's not expected that users will write visitproc.  The type
of visitproc has to be defined so that users can write tp_traverse
functions, not really so they can write their own visitprocs.  Any
visitproc spec "that works" for gcmodule's internal purposes (gcmodule
is the only known supplier of visitprocs) is good enough. 
User-supplied tp_traverse methods just have to play along.

Moving the first last:

> First, I don't like this macro, based on my own experience writing macros
> that hide returns. :)

Except VISIT is useful only inside a tp_traverse implementation, and
all such implementations look exactly the same (VISIT, VISIT, VISIT,
..., sometimes with a surrounding loop).  That's not an accident,
since the only purpose of a tp_traverse implementation is to VISIT
containees.  So it's a special-purpose macro for a highly specific and
uniform task, not a general-purpose macro that has to play well in
arbitrary contexts.
From dan.gass at gmail.com  Tue Jul 13 17:16:09 2004
From: dan.gass at gmail.com (Dan Gass)
Date: Tue Jul 13 17:16:29 2004
Subject: [Python-Dev] Developer looking to help fix bugs
In-Reply-To: <40F35B97.2010903@v.loewis.de>
References: <1089688837.2657.5.camel@localhost.localdomain>
	<40F35B97.2010903@v.loewis.de>
Message-ID: <bc15ba280407130816709c5a67@mail.gmail.com>

> Personally, I believe patch reviews might help most at this point:
> take some of the old patches, and evaluate them. Find out whether they
> do what they say they do, and whether they do it correctly. If they
> fix a bug, determine whether what they change really is a bug, and
> whether the fix won't break existing code. Also check whether a test
> case accompanies the fix. If the patch adds a new feature, determine
> whether the feature is desirable, and whether it comes with
> documentation and test cases. Put your analysis as a comment in
> the patch.

Help reviewing patch #914575 (difflib patch to add HTML side by side
difference) (https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914575&group_id=5470)
would be appreciated.  I am in the process of writing the docs and
tests but the user interface to the new functionality could use
another person's perspective.

> > If anyone is in the Chicago area, I would love to buy you a beer and
> > pick your brain.

I'm just north of you by 90 miles in Milwaukee but don't like beer :-(

Regards,
Dan Gass
From jim at zope.com  Tue Jul 13 17:19:59 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 13 17:20:02 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to decref and
	set to NULL)
In-Reply-To: <1f7befae040713081023e68654@mail.gmail.com>
References: <40F300DB.6050106@zope.com>
	<1f7befae04071215506b00d40c@mail.gmail.com>
	<40F3BD4C.70701@zope.com>
	<1f7befae040713081023e68654@mail.gmail.com>
Message-ID: <40F3FD9F.2040808@zope.com>

Tim Peters wrote:
> [Jim Fulton, on non-zero return values from a visitproc]
> 
...

>>First, I don't like this macro, based on my own experience writing macros
>>that hide returns. :)
> 
> 
> Except VISIT is useful only inside a tp_traverse implementation, and
> all such implementations look exactly the same (VISIT, VISIT, VISIT,
> ..., sometimes with a surrounding loop).  That's not an accident,
> since the only purpose of a tp_traverse implementation is to VISIT
> containees.  So it's a special-purpose macro for a highly specific and
> uniform task, not a general-purpose macro that has to play well in
> arbitrary contexts.

I still don't like it, because it hides the return.

I'd rather do:

	if (self->foo != NULL && (vret = visit(self->foo, arg)))
		return vret;

than

         VISIT(self->foo)

So I'm not interested in the macro myself. :)

I'll say again that I don't like calling the return from the visit
proc an error indicator, because it's not, and because it goes against
the common idiom of using a negative return to indicate an error.
I think that these common idioms are important, because they help
us understand the code we're reading more easily.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From jeremy at alum.mit.edu  Tue Jul 13 17:24:55 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue Jul 13 17:24:56 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to decref
	andset to NULL)
In-Reply-To: <40F3FD9F.2040808@zope.com>
Message-ID: <20040713152454.C48071E4003@bag.python.org>

> I still don't like it, because it hides the return.

It should be used in the core, but you don't have to use it in your own
code.

> I'd rather do:
>
>	if (self->foo != NULL && (vret = visit(self->foo, arg)))
>		return vret;

This violates the Python C style by putting an assignment in a conditional.
You'd need to put the assignment to vret inside the body of an if statement.
In other words, it would look just like the expansion of the VISIT() macro.

Jeremy


From jim at zope.com  Tue Jul 13 17:29:35 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 13 17:29:39 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to decref andset
	to NULL)
In-Reply-To: <20040713153128.A9A972C1C7@ns2.zope.com>
References: <20040713153128.A9A972C1C7@ns2.zope.com>
Message-ID: <40F3FFDF.6000806@zope.com>

Jeremy Hylton wrote:
>>I still don't like it, because it hides the return.
> 
> 
> It should be used in the core, but you don't have to use it in your own
> code.
> 
> 
>>I'd rather do:
>>
>>	if (self->foo != NULL && (vret = visit(self->foo, arg)))
>>		return vret;
> 
> 
> This violates the Python C style by putting an assignment in a conditional.
> You'd need to put the assignment to vret inside the body of an if statement.
> In other words, it would look just like the expansion of the VISIT() macro.

Except that you would see the return statement.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From tim.peters at gmail.com  Tue Jul 13 17:33:42 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Jul 13 17:33:45 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to decref
	and set to NULL)
In-Reply-To: <40F3FD9F.2040808@zope.com>
References: <40F300DB.6050106@zope.com>
	<1f7befae04071215506b00d40c@mail.gmail.com>
	<40F3BD4C.70701@zope.com>
	<1f7befae040713081023e68654@mail.gmail.com>
	<40F3FD9F.2040808@zope.com>
Message-ID: <1f7befae040713083363a57a7b@mail.gmail.com>

[Jim Fulton]
> I still don't like it, because it hides the return.
> 
> I'd rather do:
> 
>        if (self->foo != NULL && (vret = visit(self->foo, arg)))
>                return vret;
> 
> than
> 
>         VISIT(self->foo)

I don't believe you <0.5 wink>.  Here, rewrite this:

	VISIT(self->readline);
	VISIT(self->read);
	VISIT(self->file);
	VISIT(self->memo);
	VISIT(self->stack);
	VISIT(self->pers_func);
	VISIT(self->arg);
	VISIT(self->last_string);
	VISIT(self->find_class);
                return 0;

That's the current Unpickler_traverse, and it's utterly typical.  In
context, hiding the return doesn't matter:  no computation is done
between VISIT invocations.

> So I'm not interested in the macro myself. :)

That's fair, but the lack of one will make correct docs for
tp_traverse functions clumsier to write and read (you'll have to write
out the tedious & error-prone (*because* tedious) expansion instead --
and that will encourage readers to code their tp_traverses in this
brittle way too).

> I'll say again that I don't like calling the return from the visit
> proc an error indicator, because it's not,

I agree (again <wink>).

> and because it goes against the common idiom of using a negative return to
> indicate an error.

Well, it's not an error indicator (although it *may* be), so error
idioms don't apply.  I'll change that bit of the docs.
From jim at zope.com  Tue Jul 13 17:36:26 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 13 17:36:32 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to decref	andset
	to NULL)
In-Reply-To: <20040713152454.C48071E4003@bag.python.org>
References: <20040713152454.C48071E4003@bag.python.org>
Message-ID: <40F4017A.1040400@zope.com>

Jeremy Hylton wrote:
>>I still don't like it, because it hides the return.
> 
> 
> It should be used in the core, but you don't have to use it in your own
> code.
> 
> 
>>I'd rather do:
>>
>>	if (self->foo != NULL && (vret = visit(self->foo, arg)))
>>		return vret;
> 
> 
> This violates the Python C style by putting an assignment in a conditional.

Where is this documented?  Assignments in conditionals are used widely
in the C sources, including such places as gcmodule.c, object.c, and
typeobject.c.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From tim.peters at gmail.com  Tue Jul 13 17:47:51 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Jul 13 17:48:05 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to decref
	andset to NULL)
In-Reply-To: <40F4017A.1040400@zope.com>
References: <20040713152454.C48071E4003@bag.python.org>
	<40F4017A.1040400@zope.com>
Message-ID: <1f7befae0407130847bbbae91@mail.gmail.com>

[Jeremy]
> > This violates the Python C style by putting an assignment in a conditional.

[Jim]
> Where is this documented?  Assignments in conditionals are used widely
> in the C sources, including such places as gcmodule.c, object.c, and
> typeobject.c.

It's not actually part of PEP 7 (the Python C style guidelines).

It was an informal rule in PythonLabs (at least the non-Guido part of
it <wink>), triggered by noting how many subtle bugs in real life
ended up involving an embedded-in-a-conditional assignment.  I don't
run around getting rid of those for their own sake, but if I'm
rewriting a piece of code for other reasons it's a transformation I
routinely make now.  And you can count the number of Python bugs
traced to my C code over the last decade on one finger <wink -- but
avoiding doing multiple things on a single line really helps, and so
does avoiding side effects in the interior of expressions>.
From tim.peters at gmail.com  Tue Jul 13 19:33:53 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Jul 13 19:33:59 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to decref
	and set to NULL)
In-Reply-To: <40F3F85B.70409@zope.com>
References: <40F300DB.6050106@zope.com>
	<1f7befae04071215506b00d40c@mail.gmail.com>
	<40F3BD4C.70701@zope.com> <40F3F85B.70409@zope.com>
Message-ID: <1f7befae040713103319ce624b@mail.gmail.com>

[Jim]
> The documentation is still off I think, as it says that a
> non-zero value indicates an error, but I don't think that this
> is true.

I checked in doc chages, to clarify this, and to clarify that a NULL
cannot be passed as the first arg to the visit() callback, and to
clarify that users aren't normally expected to write visit() callbacks
themselves (they're supplied by the core).
From FBatista at uniFON.com.ar  Tue Jul 13 21:53:00 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Jul 13 21:56:15 2004
Subject: [Python-Dev] Regression tests
Message-ID: <A128D751272CD411BC9200508BC2194D03383B8A@escpl.tcp.com.ar>

This is my first time doing the regression tests after an installation, and
I have some questions:

- It gave me a lot of skipped:

    42 tests skipped:
        test__locale test_aepack test_al test_applesingle test_bsddb185
        test_cd test_cl test_codecmaps_cn test_codecmaps_jp
        test_codecmaps_kr test_codecmaps_tw test_commands test_crypt
        test_curses test_dbm test_dl test_fcntl test_fork1 test_gdbm
        test_gl test_grp test_imgfile test_ioctl test_linuxaudiodev
        test_macfs test_macostools test_mhlib test_mpz test_nis
        test_normalization test_openpty test_ossaudiodev test_plistlib
        test_poll test_posix test_pty test_pwd test_resource
        test_scriptpackages test_signal test_sunaudiodev test_timing
    Those skips are all expected on win32.

As long these are all expected to skip, shouldn't they not be tested in
-uall mode?

- It gave me some failed:

    6 tests failed:
        test_cfgparser test_decimal test_socket_ssl test_timeout
        test_urllib2 test_urllibnet

What should I do with these? I'll certainly take a look at test_decimal, but
with the others? Inform them here? Open bugs?

Thank you!

Facundo Batista
Desarrollo de Red
fbatista@unifon.com.ar
(54 11) 5130-4643
Cel: 15 5097 5024


From tim.peters at gmail.com  Tue Jul 13 22:10:14 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Jul 13 22:10:17 2004
Subject: [Python-Dev] Regression tests
In-Reply-To: <A128D751272CD411BC9200508BC2194D03383B8A@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D03383B8A@escpl.tcp.com.ar>
Message-ID: <1f7befae040713131071166a11@mail.gmail.com>

[Batista, Facundo]
> This is my first time doing the regression tests after an installation, and
> I have some questions:
>
> - It gave me a lot of skipped:
>
>    42 tests skipped:
>        test__locale test_aepack test_al test_applesingle test_bsddb185
>        test_cd test_cl test_codecmaps_cn test_codecmaps_jp
>        test_codecmaps_kr test_codecmaps_tw test_commands test_crypt
>        test_curses test_dbm test_dl test_fcntl test_fork1 test_gdbm
>        test_gl test_grp test_imgfile test_ioctl test_linuxaudiodev
>        test_macfs test_macostools test_mhlib test_mpz test_nis
>        test_normalization test_openpty test_ossaudiodev test_plistlib
>        test_poll test_posix test_pty test_pwd test_resource
>        test_scriptpackages test_signal test_sunaudiodev test_timing
>    Those skips are all expected on win32.
> 
> As long these are all expected to skip, shouldn't they not be tested in
> -uall mode?

It's very clear which tests "should" and "shouldn't" get skipped on
Windows, because the Python Windows installation includes all the
3rd-party code too (Berkely DB, Tcl/Tk, zlib, bz2, etc).  It's much
fuzzier on other systems.  In order to avoid special-casing Windows,
regrtest.py just tries everything on all platforms.

> - It gave me some failed:
> 
>    6 tests failed:
>        test_cfgparser test_decimal test_socket_ssl test_timeout
>        test_urllib2 test_urllibnet
> 
> What should I do with these? I'll certainly take a look at test_decimal, but
> with the others? Inform them here? Open bugs?

The test_cfgparser and test_decimal failures are due to missing test
files in the 2.4a1 installer.  I already opened a bug report about
those.

I'm not sure about the others.  I've never tried -uall on Windows. 
test_cfgparser and test_decimal were the only ones that failed on my
WinXP installation when doing

    regrtest.py -u network,largefile,bsddb,decimal

I don't *think* the other 4 should have failed.  So please open bug
reports, and set the version to Python 2.4.
From greg at electricrain.com  Tue Jul 13 22:54:02 2004
From: greg at electricrain.com (Gregory P. Smith)
Date: Tue Jul 13 22:54:06 2004
Subject: [Python-Dev] _bsddb too unstable?
In-Reply-To: <40F3EB6D.8020107@zope.com>
References: <40F3EB6D.8020107@zope.com>
Message-ID: <20040713205402.GA5003@zot.electricrain.com>

On Tue, Jul 13, 2004 at 10:02:21AM -0400, Jim Fulton wrote:
> 
> I just wasted some time chasing down being unable to run the
> Python test suite because the test_anydbm tests were segfaulting.
> Removing the _bsddb module allowed the tests to run without
> segfaulting.
> 
> (FWIW, I'm running on Fedora Core 1, with db4-4.1.25-14)

Where is the sf bugreport?  what python cvs version/date was this,
where did the segfault occur, what stack trace does the coredump show?
test_anydbm.py with _bsddb using 4.1 or 4.2 do not coredump for me,
if theres a bug i'd like to fix it.

> I get the impression that this is a common problem.
> Berkeley DB APIs often change in non-backward compatable
> ways that cause this sort of problem.  Perhaps these APIs
> change too rapidly to allow the _bsddb extension to be included
> in the distribution.  I assume that this is the fault of
> the Berkeley DB libraries, and not the extension, but I don't
> really know. :/

-1

The Sleepycat BerkeleyDB API rarely changes.  The extension is much
more mutable as support for more of the sleepycat apis are added and
enhancements are made to the compatibility interface, i'd expect this
type of bug to be there first.

The recent need to update the compatibility interface to do its own
unlink() rather than calling BerkeleyDB's remove() function is the
-only- time i can recall that a sleepycat change caused any problems.
(and i'm not sure if that was a due to a sleepycat midstream change
because i based my work on adding 4.2 support on a prerelease version
of BerkeleyDB that didn't yet require that change)

greg

From nbastin at opnet.com  Tue Jul 13 23:37:51 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Tue Jul 13 23:38:02 2004
Subject: [Python-Dev] What is test__locale supposed to test, anyhow?
Message-ID: <E4310D2D-D514-11D8-91CD-000D932927FE@opnet.com>

test__locale, as written, tests that the data we get from nl_langinfo() 
matches the data acquired from localeconv(), but it doesn't actually 
make sure that any part of this data is correct.  Is this intentional?  
Is anybody going to mind if I rewrite this test to check that the 
values are actually *correct*?  This will mean more failures on 
platforms like solaris, where the system locale data is incorrect.

--
Nick

From martin at v.loewis.de  Wed Jul 14 00:11:36 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Jul 14 00:11:38 2004
Subject: [Python-Dev] What is test__locale supposed to test, anyhow?
In-Reply-To: <E4310D2D-D514-11D8-91CD-000D932927FE@opnet.com>
References: <E4310D2D-D514-11D8-91CD-000D932927FE@opnet.com>
Message-ID: <40F45E18.3040209@v.loewis.de>

Nick Bastin wrote:
> test__locale, as written, tests that the data we get from nl_langinfo() 
> matches the data acquired from localeconv(), but it doesn't actually 
> make sure that any part of this data is correct.  Is this intentional? 

Yes, see python.org/sf/798145. The test checks that the emulated
nl_langinfo matches the emulated localeconv for the LC_NUMERIC category.

These days, we are not supposed to emulate LC_NUMERIC anymore, so the
test is somewhat pointless now.

Checking for the actual data will indeed trigger alot of 'failures'.
Some of these might be debatable, because it might not be clear that
the reference data you compare against is universally agreed upon
among the people who live in that locale.

Regards,
Martin

From andrewm at object-craft.com.au  Wed Jul 14 02:27:58 2004
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Wed Jul 14 02:28:04 2004
Subject: [Python-Dev] _bsddb too unstable? 
In-Reply-To: <20040713205402.GA5003@zot.electricrain.com> 
References: <40F3EB6D.8020107@zope.com>
	<20040713205402.GA5003@zot.electricrain.com>
Message-ID: <20040714002758.C390F3C043@coffee.object-craft.com.au>

>> Berkeley DB APIs often change in non-backward compatable
>> ways that cause this sort of problem.  Perhaps these APIs
>> change too rapidly to allow the _bsddb extension to be included
>> in the distribution.  I assume that this is the fault of
>> the Berkeley DB libraries, and not the extension, but I don't
>> really know. :/
>
>-1
>
>The Sleepycat BerkeleyDB API rarely changes.  The extension is much
>more mutable as support for more of the sleepycat apis are added and
>enhancements are made to the compatibility interface, i'd expect this
>type of bug to be there first.

In contrast, I've found the Sleepycat DB libraries to be source of
never-ending grief, almost always because of subtle changes to the API or
the underlying data file format (but not bugs in the library itself). In
the early days (1.85), libdb fast, easy to use, lightweight, and reliable
(provided you avoided the well known bugs), and we used it extensively, but
I'm yet to have a wholely satisfactory experience with any version >= 2.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/
From bac at OCF.Berkeley.EDU  Wed Jul 14 03:18:03 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Jul 14 03:18:11 2004
Subject: [Python-Dev] What is test__locale supposed to test, anyhow?
In-Reply-To: <40F45E18.3040209@v.loewis.de>
References: <E4310D2D-D514-11D8-91CD-000D932927FE@opnet.com>
	<40F45E18.3040209@v.loewis.de>
Message-ID: <40F489CB.5000904@ocf.berkeley.edu>

Martin v. L?wis wrote:
> Nick Bastin wrote:
> 
>> test__locale, as written, tests that the data we get from 
>> nl_langinfo() matches the data acquired from localeconv(), but it 
>> doesn't actually make sure that any part of this data is correct.  Is 
>> this intentional? 
> 
> 
> Yes, see python.org/sf/798145. The test checks that the emulated
> nl_langinfo matches the emulated localeconv for the LC_NUMERIC category.
> 
> These days, we are not supposed to emulate LC_NUMERIC anymore, so the
> test is somewhat pointless now.
> 

So are you suggesting the test go away?

-Brett
From bac at OCF.Berkeley.EDU  Wed Jul 14 03:25:19 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Jul 14 03:25:26 2004
Subject: [Python-Dev] test__locale weirdness
In-Reply-To: <89018A21-D495-11D8-91CD-000D932927FE@opnet.com>
References: <89018A21-D495-11D8-91CD-000D932927FE@opnet.com>
Message-ID: <40F48B7F.7000501@ocf.berkeley.edu>

Nick Bastin wrote:
[SNIP - showing how locale.localeconv() is broken on OS X]
> int
> main (int argc, char *argv[])
> {
> struct lconv *result;
> 
> if (!setlocale(LC_NUMERIC, "fr_FR")) {
>   printf("setlocale() failed\n");
>   exit(1);
> }
> if (!( result = localeconv() )) {
>   printf("localeconv() failed\n");
>   exit(1);
> }
> 
> printf("Claimed locale:%s\n", setlocale(LC_NUMERIC, NULL));
> printf("decimal point: '%s', thousands_sep: '%s'\n", result->decimal_point,
> result->thousands_sep);
> 
> return 0;
> }
> 
> displays:
> 
> Claimed locale:fr_FR
> decimal point: ',', thousands_sep: ''
> 
> Which is exactly what you'd expect from a working implementation.  Does 
> anybody know what Python is doing beyond what this simple test does?
> 

In case anyone else is following/working on this, I stuck the C code 
shown above into the beginning of the module initialization function and 
it is wrong at that point already.

-Brett
From kbk at shore.net  Wed Jul 14 04:34:10 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed Jul 14 04:34:15 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200407140234.i6E2YAgf027536@hydra.localdomain>


Patch / Bug Summary
___________________

Patches :  278 open (-16) /  2458 closed (+24) /  2736 total ( +8)
Bugs    :  761 open (-17) /  4230 closed (+33) /  4991 total (+16)
RFE     :  145 open ( +1) /   129 closed ( +0) /   274 total ( +1)

New / Reopened Patches
______________________

tarfile.py and multiple extended headers  (2004-07-10)
CLOSED http://python.org/sf/988444  opened by  Lars Gustäbel 

urllib2.AbstractHTTPHandler broken  (2004-07-10)
CLOSED http://python.org/sf/988504  opened by  John J Lee 

Fix for bug 980327  (2004-07-10)
CLOSED http://python.org/sf/988607  opened by  Paul Moore 

Update HTTPRespnse.length  (2004-07-10)
       http://python.org/sf/988642  opened by  John Ehresman 

Untested fix for 988120  (2004-07-10)
CLOSED http://python.org/sf/988643  opened by  John J Lee 

compiler.transformer fix for (a, b) = 1, 2  (2004-07-10)
CLOSED http://python.org/sf/988698  opened by  John Ehresman 

re.split emptyok flag (fix for #852532)  (2004-07-10)
       http://python.org/sf/988761  opened by  Mike Coleman 

Support using Tk without a mainloop  (2004-07-12)
       http://python.org/sf/989712  opened by  Noam Raphael 

Patches Closed
______________

make test_urllib2 work on Windows  (2004-01-04)
       http://python.org/sf/870606  closed by  jjlee

tarfile.py and multiple extended headers  (2004-07-10)
       http://python.org/sf/988444  closed by  akuchling

cgitb: escape title of page  (2004-07-08)
       http://python.org/sf/987052  closed by  akuchling

add SO_EXCLUSIVEADDRUSE constant in socketmodule.c  (2004-06-30)
       http://python.org/sf/982665  closed by  akuchling

urllib2.AbstractHTTPHandler broken  (2004-07-10)
       http://python.org/sf/988504  closed by  akuchling

asyncore.dispatcher.set_reuse_addr() is broken on windows  (2004-06-30)
       http://python.org/sf/982681  closed by  akuchling

urllib2 silently returns None when auth_uri is mismatched  (2003-11-30)
       http://python.org/sf/851752  closed by  kbk

asyncore loop_once and remove poll2  (2002-12-04)
       http://python.org/sf/648322  closed by  akuchling

urllib2 handler algorithm docs  (2004-06-13)
       http://python.org/sf/972310  closed by  akuchling

Don't print to stdout in cookielib.py exception handler   (2004-06-09)
       http://python.org/sf/969907  closed by  akuchling

cookielib doc minor corrections and clarifications  (2004-06-09)
       http://python.org/sf/969900  closed by  akuchling

nonblocking i/o with ssl socket not working at all  (2004-04-30)
       http://python.org/sf/945642  closed by  akuchling

Fix for bug 980327  (2004-07-10)
       http://python.org/sf/988607  closed by  bcannon

inspect.getargspec: None instead of ()  (2002-11-12)
       http://python.org/sf/637217  closed by  bcannon

Untested fix for 988120  (2004-07-10)
       http://python.org/sf/988643  closed by  jjlee

_ssl.c compatibility  (2004-03-03)
       http://python.org/sf/909007  closed by  akuchling

improve markupbase.py error reporting  (2004-06-02)
       http://python.org/sf/965175  closed by  akuchling

webbrower support for firebird / firefox  (2004-06-29)
       http://python.org/sf/981794  closed by  akuchling

PyArg_VaParseTupleAndKeywords  (2002-04-30)
       http://python.org/sf/550732  closed by  bcannon

compiler.transformer fix for (a, b) = 1, 2  (2004-07-11)
       http://python.org/sf/988698  closed by  mwh

difflib empty list IndexError fix (Bug #980117)  (2004-06-25)
       http://python.org/sf/980120  closed by  bcannon

urllib2 FTPHandler bugs  (2004-06-13)
       http://python.org/sf/972332  closed by  kbk

add missing doc for datetime C API  (2004-07-06)
       http://python.org/sf/986010  closed by  tim_one

Patch for Vinay Sajip's Python-logging package  (2004-03-22)
       http://python.org/sf/921318  closed by  vsajip

New / Reopened Bugs
___________________

non local link in local docs  (2004-07-10)
CLOSED http://python.org/sf/988387  opened by  brian vdb 

inspect.getmodule symlink-related failur  (2002-06-18)
       http://python.org/sf/570300  reopened by  amitar

Move urllib2 functional tests into test suite  (2004-07-10)
CLOSED http://python.org/sf/988602  opened by  John J Lee 

compiler.transformer and tuple unpacking  (2004-07-10)
CLOSED http://python.org/sf/988613  opened by  Felix Wiemann 

Windows installer missing test files  (2004-07-11)
       http://python.org/sf/988784  opened by  Tim Peters 

profile.run raises TypeError encountering a list  (2004-07-11)
CLOSED http://python.org/sf/989066  opened by  Simon Percivall 

unicode.width broken for combining characters  (2004-07-11)
       http://python.org/sf/989185  opened by  Matthew Mueller 

logging.config: Does not check logging.handlers for handlers  (2004-07-12)
CLOSED http://python.org/sf/989199  opened by  Tim Leslie 

Python-2.4.0a1 test_strptime.py fails on HP-UX11i  (2004-07-12)
       http://python.org/sf/989273  opened by  Richard Townsend 

Empty curses module is loaded in win32  (2004-07-12)
       http://python.org/sf/989333  opened by  Miki Tebeka 

test_descr fails on win2k  (2004-07-12)
       http://python.org/sf/989337  opened by  Miki Tebeka 

test_unicode_file fails on win2k  (2004-07-12)
       http://python.org/sf/989338  opened by  Miki Tebeka 

Python Logging filename & file number   (2004-06-07)
       http://python.org/sf/968245  reopened by  superleo303

misinforming help messages in pdb.py  (2004-07-12)
       http://python.org/sf/989672  opened by  Simon Percivall 

cgi.parse_qsl broken  (2004-07-13)
       http://python.org/sf/990307  opened by  Neil Schemenauer 

testtar.tar is partially insane  (2004-07-13)
       http://python.org/sf/990325  opened by  Tim Peters 

mistyped example  (2004-07-13)
       http://python.org/sf/990497  opened by  Daniel Pezely 

online documentation says "Nac" in the link, "Nav" for mod  (2004-07-13)
       http://python.org/sf/990524  opened by  Matthew Shomphe 

printing date causes crash  (2004-07-13)
       http://python.org/sf/990569  opened by  Justin 

Bugs Closed
___________

non local link in local docs  (2004-07-10)
       http://python.org/sf/988387  closed by  rhettinger

Windows installer for 2.4a1 shows non-standard version  (2004-07-09)
       http://python.org/sf/987870  closed by  pmoore

Typo in cgi doc  (2004-07-08)
       http://python.org/sf/987486  closed by  rhettinger

pep 311 functions missing documentation  (2004-07-09)
       http://python.org/sf/987835  closed by  akuchling

[2.3.2] zipfile test failure on AIX 5.1  (2003-11-03)
       http://python.org/sf/835145  closed by  akuchling

Race condition in asyncore poll  (2002-10-26)
       http://python.org/sf/629097  closed by  akuchling

Problem w/6.27.2.2 GNUTranslations ungettext() example code  (2004-05-07)
       http://python.org/sf/949832  closed by  akuchling

inspect.getmodule symlink-related failur  (2002-06-17)
       http://python.org/sf/570300  closed by  bcannon

Relative Path causing crash in RotatingFileHandler  (2004-06-29)
       http://python.org/sf/982049  closed by  bcannon

smtpd.py needs documentation  (2001-08-14)
       http://python.org/sf/450803  closed by  fdrake

zipfile.py - pack filesize as unsigned allows files &gt; 2 gig  (2003-02-03)
       http://python.org/sf/679953  closed by  bcannon

Move urllib2 functional tests into test suite  (2004-07-10)
       http://python.org/sf/988602  closed by  akuchling

Demo/dbm.py - Rewrite using anydbm?  (2003-01-08)
       http://python.org/sf/664715  closed by  bcannon

ntpath normpath  (2004-06-26)
       http://python.org/sf/980327  closed by  bcannon

compiler.transformer and tuple unpacking  (2004-07-10)
       http://python.org/sf/988613  closed by  mwh

urllib.urlretrieve() with ftp error   (2002-11-08)
       http://python.org/sf/635453  closed by  bcannon

TimedRotatingFileHandler problems  (2004-07-08)
       http://python.org/sf/987166  closed by  vsajip

IMAP4_SSL() class incompatible with socket.timeout  (2004-06-22)
       http://python.org/sf/977680  closed by  bcannon

os.path.realpath can't handle symlink loops  (2004-04-05)
       http://python.org/sf/930024  closed by  bcannon

Tiny suggestion to improve optparse doc  (2004-07-03)
       http://python.org/sf/984691  closed by  bcannon

smtplib.SMTP prints debug stuff to stdout  (2004-06-27)
       http://python.org/sf/980938  closed by  bcannon

Simplifiy coding in cmd.py  (2004-05-18)
       http://python.org/sf/956408  closed by  rhettinger

diffutil errors when comparing 2 0 byte entries  (2004-06-25)
       http://python.org/sf/979794  closed by  bcannon

Index error for empty lists in Difflib  (2004-06-25)
       http://python.org/sf/980117  closed by  bcannon

weakref should have a WeakSet  (2004-07-06)
       http://python.org/sf/985753  closed by  rhettinger

Parser wart  (2003-08-27)
       http://python.org/sf/796116  closed by  rhettinger

urllib2 silently returns None when auth_uri is mismatched  (2003-10-09)
       http://python.org/sf/820583  closed by  kbk

Queue class has logic error when non-blocking  (2003-08-14)
       http://python.org/sf/788520  closed by  tim_one

profile.run raises TypeError encountering a list  (2004-07-11)
       http://python.org/sf/989066  closed by  mondragon

logging.config: Does not check logging.handlers for handlers  (2004-07-12)
       http://python.org/sf/989199  closed by  vsajip

Trap OSError when calling RotatingFileHandler.doRollover  (2004-06-24)
       http://python.org/sf/979252  closed by  vsajip

logging functionality non-intuitive, levels confusing  (2004-05-21)
       http://python.org/sf/958180  closed by  vsajip

Bad state of multi btree database file after large inserts  (2004-06-30)
       http://python.org/sf/982679  closed by  loewis

db4 4.2 == :-(  (test_anydbm and test_bsddb3)  (2004-02-15)
       http://python.org/sf/897820  closed by  greg

BSDDB set_location bug  (2004-02-05)
       http://python.org/sf/890872  closed by  greg

From nbastin at opnet.com  Wed Jul 14 06:35:19 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Wed Jul 14 06:35:42 2004
Subject: [Python-Dev] What is test__locale supposed to test, anyhow?
In-Reply-To: <40F45E18.3040209@v.loewis.de>
References: <E4310D2D-D514-11D8-91CD-000D932927FE@opnet.com>
	<40F45E18.3040209@v.loewis.de>
Message-ID: <35EBDEEC-D54F-11D8-91CD-000D932927FE@opnet.com>


On Jul 13, 2004, at 6:11 PM, Martin v. L?wis wrote:

> Nick Bastin wrote:
>> test__locale, as written, tests that the data we get from 
>> nl_langinfo() matches the data acquired from localeconv(), but it 
>> doesn't actually make sure that any part of this data is correct.  Is 
>> this intentional?
>
> Yes, see python.org/sf/798145. The test checks that the emulated
> nl_langinfo matches the emulated localeconv for the LC_NUMERIC 
> category.
>
> These days, we are not supposed to emulate LC_NUMERIC anymore, so the
> test is somewhat pointless now.

Well, it's not completely pointless - it's pointing out that localeconv 
information generated by Python, at least on MacOS X, is incorrect.

> Checking for the actual data will indeed trigger alot of 'failures'.
> Some of these might be debatable, because it might not be clear that
> the reference data you compare against is universally agreed upon
> among the people who live in that locale.

Well, it seems we ought to have *some* kind of test that _locale is 
working properly.  The current test doesn't seem to fit the bill.

--
Nick

From martin at v.loewis.de  Wed Jul 14 07:07:17 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Jul 14 07:07:19 2004
Subject: [Python-Dev] What is test__locale supposed to test, anyhow?
In-Reply-To: <35EBDEEC-D54F-11D8-91CD-000D932927FE@opnet.com>
References: <E4310D2D-D514-11D8-91CD-000D932927FE@opnet.com>	<40F45E18.3040209@v.loewis.de>
	<35EBDEEC-D54F-11D8-91CD-000D932927FE@opnet.com>
Message-ID: <40F4BF85.2030905@v.loewis.de>

Nick Bastin wrote:
> Well, it seems we ought to have *some* kind of test that _locale is 
> working properly.  The current test doesn't seem to fit the bill.

It qualifies as *some* test, though. Remember that there is also
test_locale.py.

Regards,
Martin

From eric at enthought.com  Wed Jul 14 07:07:42 2004
From: eric at enthought.com (eric jones)
Date: Wed Jul 14 07:08:20 2004
Subject: [Python-Dev] ANN: Reminder -- SciPy 04 is coming up
Message-ID: <40F4BF9E.8060103@enthought.com>

Hey folks,

Just a reminder that SciPy 04 is coming up.  More information is here:

http://www.scipy.org/wikis/scipy04

About the Conference and Keynote Speaker
---------------------------------------------
The 1st annual *SciPy Conference* will be held this year at Caltech, 
September 2-3, 2004.  As some of you may know, we've experienced great 
participation in two SciPy "Workshops" (with ~70 attendees in both 2002 
and 2003) and this year we're graduating to a "conference."  With the 
prestige of a conference comes the responsibility of a keynote address.  
This year, Jim Hugunin has answered the call and will be speaking to 
kickoff the meeting on Thursday September 2nd.  Jim is the creator of 
Numeric Python, Jython, and co-designer of AspectJ. Jim is currently 
working on IronPython--a fast implementation of Python for .NET and Mono.

Presenters
-----------
We still have room for a few more standard talks, and there is plenty of 
room for lightning talks. Because of this, we are extending the abstract 
deadline until July 23rd.  Please send your abstract to 
abstracts@scipy.org.  Travis Oliphant is organizing the presentations 
this year. (Thanks!)  Once accepted, papers and/or presentation slides 
are acceptable and are due by August 20, 2004. 

Registration
-------------
Early registration ($100.00) has been extended to July 23rd.  Follow the 
links off of the main conference site:

http://www.scipy.org/wikis/scipy04

After July 23rd, registration will be $150.00.  Registration includes 
breakfast and lunch Thursday & Friday and a very nice dinner Thursday 
night.  Please register as soon as possible as it will help us in 
planning for food, room sizes, etc.

Sprints
--------
As of now, we really haven't had much of a call for coding sprints for 
the 3 days prior to SciPy 04.  Below is the original announcement about 
sprints.  If you would like to suggest a topic and see if others are 
interested, please send a message to the list.  Otherwise, we'll forgo 
the sprints session this year.

    We're also planning three days of informal "Coding Sprints" prior to
    the conference -- August 30 to September 1, 2004.  Conference
    registration is not required to participate in the sprints.  Please
    email the list, however, if you plan to attend.  Topics for these
    sprints will be determined via the mailing lists as well, so please
    submit any suggestions for topics to the scipy-user list:

    list signup: http://www.scipy.org/mailinglists/
    list address: scipy-user@scipy.org


thanks,
eric

From martin at v.loewis.de  Wed Jul 14 07:09:18 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Jul 14 07:09:22 2004
Subject: [Python-Dev] test__locale weirdness
In-Reply-To: <40F48B7F.7000501@ocf.berkeley.edu>
References: <89018A21-D495-11D8-91CD-000D932927FE@opnet.com>
	<40F48B7F.7000501@ocf.berkeley.edu>
Message-ID: <40F4BFFE.3010808@v.loewis.de>

Brett C. wrote:
> In case anyone else is following/working on this, I stuck the C code 
> shown above into the beginning of the module initialization function and 
> it is wrong at that point already.

You could try to trace the setlocale calls that have been made up
that that point.

There is also a possibility that Python links with independent locale
implementations: one providing nl_langinfo, and the other providing
localeconv.

Regards,
Martin

From foom at fuhm.net  Wed Jul 14 08:41:19 2004
From: foom at fuhm.net (James Y Knight)
Date: Wed Jul 14 08:41:30 2004
Subject: [Python-Dev] Tagged integers
Message-ID: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>

So I was saying to someone the other day "Gee, I wonder why Python 
doesn't use tagged integers, it seems like it would be a lot faster 
than allocating new objects all the time.", and they said "Cause you'd 
have to change everything, too much work!" and I said "Nah, you only 
need to change a few things to use macros, it'd only take a few hours, 
mostly query-replace".

So, of course, I had to do it then, and it only took a couple hours, 
and appears to be at least somewhat faster.

On the test that probably puts my change in the most positive light 
possible: "x = 0;  while x < 50000000:  x = x + 1", it achieves about a 
50% increase in speed. More normal integer-heavy things seem to be at 
most 20% faster.

My implementation works as follows:
- Assumes at least 32-bit words
- Lower 2 bits of machine word are the tag. tag 0 == normal object 
pointer (all object pointers are already word aligned, so their lower 2 
bits will be 0). tag 1 == integer. 2&3 unassigned.
- Integers that fit in 30 bits are stored into the word directly, not 
in allocated memory.
   - For example, integer "5" would be stored as (5 << 2) | 1 == 0x15.
- Thus, an arbitrary "PyObject *" can no longer be relied upon to 
actually be a pointer. Sometimes it will be tagged data instead.
- Thus, no code can directly access object fields ->ob_refcnt, or 
->ob_type. Introduced Py_GETTYPE/Py_GETREF macros and search&replaced 
code in python to use them. These macros check the tag bits, and do the 
right thing if it is tagged data.
- I kept tagged integers as the same class as allocated int objects, as 
it simplified implementation (nothing outside intobject knows the 
difference). It would probably be nicer to do away with heap allocated 
int objects and overflow directly to long objects after 30 bits, but, 
IIRC, there are various bits of python that require integers up to the 
platform's full integer width to fit in an instance of PyInt_Type. 
Also, subclasses of integer must be heap allocated too, of course.
- intobject.c cannot directly access ->ob_ival. Instead, I made it use 
PyInt_AS_LONG, which is modified to know about tagged integers.
- That's about it!

So, why doesn't python already use tagged integers? Surely someone's 
thought to "just do it" before? I only see discussion of it with 
relation to pypy.

A couple things:
- It's almost certainly not fully portable -- that's fine! Keep using 
heap allocated int objects on architectures that it doesn't work with.
- It does cause incompatibility with external C modules. Not only 
binary incompatibility -- the source also needs to be modified to not 
use ->ob_type/->ob_refcnt directly. ob_refcnt is hardly used outside of 
INCREF/DECREF macros already, so that's good. ob_type is used a lot in 
extension modules' PyXXX_Check macros.

Here's the patch I have against Python-2.3.3. Please note this is just 
a couple hour hack, it may have errors. Most of the diff is quite 
boring and repetitious.
<http://fuhm.net/~jknight/python233-tagint.diff.gz>.

So, any thoughts? Worth continuing on with this? If this is something 
that people are interested in actually doing, I could update the patch 
against latest CVS and put the changes in #ifdefs so it's compile-time 
selectable.

Thoughts for future development:
   There is space available for 2 more tagged data types. Could they be 
productively used? Perhaps one for single element tuples. Perhaps one 
for single character unicode strings. Dunno if those are easily doable 
and would actually increase performance.

James

PS:
Here's the interesting portions of the changes. Yes, I realize typeof() 
and ({ are GCC extensions, but this was the most straightforward way to 
implement inline expression macros that may use their arguments more 
than once. Maybe they should be inline functions instead of macros?

===== object.h

#define Py_OBJ_TAG(op) (((Py_uintptr_t)(op)) & 0x03)

#define Py_GETTYPE(op) ({typeof((op)) __op = (op);  \
    (!Py_OBJ_TAG(__op))?__op->ob_type:Py_tagged_types[Py_OBJ_TAG(__op)]; 
})

#define Py_GETREF(op) ({typeof((op)) __op = (op); \
      Py_OBJ_TAG(__op)?1:__op->ob_refcnt; })
#define Py_SETREF(op, val) ({typeof((op)) __op = (op); \
    if(!Py_OBJ_TAG(__op)) \
      __op->ob_refcnt = (val); \
})

#define Py_ADDREF(op, amt) ({typeof((op)) ___op = (op); \
    if(!Py_OBJ_TAG(___op)) \
      ___op->ob_refcnt += (amt); \
    Py_GETREF(___op); \
})

#define Py_INCREF(op) (                         \
         _Py_INC_REFTOTAL  _Py_REF_DEBUG_COMMA   \
         Py_ADDREF(op, 1), (void)0)

#define Py_DECREF(op)                                   \
         if (_Py_DEC_REFTOTAL  _Py_REF_DEBUG_COMMA       \
             Py_ADDREF(op, -1) != 0)                     \
                 _Py_CHECK_REFCNT(op)                    \
         else                                            \
                 _Py_Dealloc((PyObject *)(op))

===== intobject.h

#define PyInt_AS_LONG(op) ({typeof((op)) __op = (op);  \
    Py_OBJ_TAG(__op)?(((long)__op) >> 2):((PyIntObject *)__op)->ob_ival; 
})

From mwh at python.net  Wed Jul 14 10:58:20 2004
From: mwh at python.net (Michael Hudson)
Date: Wed Jul 14 10:58:22 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Doc/lib
 libshutil.tex,1.14,1.15
In-Reply-To: <E1BkXxE-0000Bp-PX@sc8-pr-cvs1.sourceforge.net>
	(gvanrossum@users.sourceforge.net's
	message of "Tue, 13 Jul 2004 17:49:00 -0700")
References: <E1BkXxE-0000Bp-PX@sc8-pr-cvs1.sourceforge.net>
Message-ID: <2mpt6zx8wj.fsf@starship.python.net>

gvanrossum@users.sourceforge.net writes:

> Update of /cvsroot/python/python/dist/src/Doc/lib
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv725
>
> Modified Files:
> 	libshutil.tex 
> Log Message:
> - Bug #981530: Fix UnboundLocalError in shutil.rmtree().  This affects
>   the documented behavior: the function passed to the onerror()
>   handler can now also be os.listdir.
>
> [I could've sworn I checked this in, but apparently I didn't, or it
> got lost???]

Pretty sure I haven't read that checkin message before, if that's any
reassurance :-)

Cheers,
mwh

-- 
  You can lead an idiot to knowledge but you cannot make him 
  think.  You can, however, rectally insert the information, 
  printed on stone tablets, using a sharpened poker.        -- Nicolai
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html
From mal at egenix.com  Wed Jul 14 11:16:29 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Jul 14 11:16:31 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
References: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
Message-ID: <40F4F9ED.2020505@egenix.com>

James Y Knight wrote:
> So I was saying to someone the other day "Gee, I wonder why Python 
> doesn't use tagged integers, it seems like it would be a lot faster than 
> allocating new objects all the time.", and they said "Cause you'd have 
> to change everything, too much work!" and I said "Nah, you only need to 
> change a few things to use macros, it'd only take a few hours, mostly 
> query-replace".
> 
> So, of course, I had to do it then, and it only took a couple hours, and 
> appears to be at least somewhat faster.
> 
> On the test that probably puts my change in the most positive light 
> possible: "x = 0;  while x < 50000000:  x = x + 1", it achieves about a 
> 50% increase in speed. More normal integer-heavy things seem to be at 
> most 20% faster.

Interesting. I would have thought that hackery like this
would result in more significant speedups. Looks like the
Python implementation is darn fast already :-)

Note that Python shares small integers and uses
a free list for the rest, so most of the times, the
implementation can create objects using already allocated
memory.

I'd rather not like to see hackery like misused pointers
in the core, so you can count me as -1 on this.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 14 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From brian at sweetapp.com  Wed Jul 14 11:40:40 2004
From: brian at sweetapp.com (Brian Quinlan)
Date: Wed Jul 14 11:36:48 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
References: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
Message-ID: <40F4FF98.2000204@sweetapp.com>

James Y Knight wrote:
> So I was saying to someone the other day "Gee, I wonder why Python 
> doesn't use tagged integers, it seems like it would be a lot faster than 
> allocating new objects all the time.", and they said "Cause you'd have 
> to change everything, too much work!" and I said "Nah, you only need to 
> change a few things to use macros, it'd only take a few hours, mostly 
> query-replace".
> 
> So, of course, I had to do it then, and it only took a couple hours, and 
> appears to be at least somewhat faster.

It made integer-heavy things 20% faster. How much slower did it make 
everything else? Did you try running a complete benchmark (e.g. 
pystone) before and after?

And the lack of portability is a real problem - it means that every 
code path must be tested twice AND there might be architectures and 
problems were tagged integers work but it actually makes things slower.

I'm skeptical until I see a complete system benchmark but it's great 
that you tried this!

Cheers,
Brian
From ncoghlan at iinet.net.au  Wed Jul 14 11:50:35 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Jul 14 11:50:38 2004
Subject: [Python-Dev] PEP 292 for Python 2.4
In-Reply-To: <20040713130439.GA2174@titan.progiciels-bpi.ca>
References: <1087414523.7150.45.camel@localhost>	<001f01c4538d$f1fcfec0$bfaf2c81@oemcomputer>	<20040624142741.GC27922@panix.com>
	<20040713130439.GA2174@titan.progiciels-bpi.ca>
Message-ID: <40F501EB.4080700@iinet.net.au>

Fran?ois Pinard wrote:
> I know that `string' and `socket' exist, despite `string' is evanescent,
> but they surely forced users at choosing other identifiers where `string'
> and `socket' would have been perfect.  It is very good news that, now
> in Python 2.3, `string' is unneeded most of times.  Let us not repeat
> previous mistakes, or even nail them further by trying to be compatible
> with them.

I would suggest that bare type names are rarely appropriate for use a 
variable names, except in toy examples.

If I'm reading someone else's code, and they create a string or a 
socket, I want to know what it is _for_, rather than the mere fact this 
it is a string or a socket.

If the type is all that is important, then prepending some simple word 
such as 'a_string' or 'the_string' or 'my_string' makes it clear to the 
maintainer that the object doesn't really have any significant semantic 
meaning beyond its type.

Regards,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From jim at zope.com  Wed Jul 14 13:07:06 2004
From: jim at zope.com (Jim Fulton)
Date: Wed Jul 14 13:07:12 2004
Subject: [Python-Dev] _bsddb too unstable?
In-Reply-To: <20040713205402.GA5003@zot.electricrain.com>
References: <40F3EB6D.8020107@zope.com>
	<20040713205402.GA5003@zot.electricrain.com>
Message-ID: <40F513DA.8010404@zope.com>

Gregory P. Smith wrote:
> On Tue, Jul 13, 2004 at 10:02:21AM -0400, Jim Fulton wrote:
> 
>>I just wasted some time chasing down being unable to run the
>>Python test suite because the test_anydbm tests were segfaulting.
>>Removing the _bsddb module allowed the tests to run without
>>segfaulting.
>>
>>(FWIW, I'm running on Fedora Core 1, with db4-4.1.25-14)
> 
> 
> Where is the sf bugreport?  what python cvs version/date was this,
> where did the segfault occur, what stack trace does the coredump show?
> test_anydbm.py with _bsddb using 4.1 or 4.2 do not coredump for me,
> if theres a bug i'd like to fix it.

Fair enough, I'll submit one.

> 
>>I get the impression that this is a common problem.
>>Berkeley DB APIs often change in non-backward compatable
>>ways that cause this sort of problem.  Perhaps these APIs
>>change too rapidly to allow the _bsddb extension to be included
>>in the distribution.  I assume that this is the fault of
>>the Berkeley DB libraries, and not the extension, but I don't
>>really know. :/
> 
> 
> -1
> 
> The Sleepycat BerkeleyDB API rarely changes. 

This has not been my experience, but ...

 > The extension is much
> more mutable as support for more of the sleepycat apis are added and
> enhancements are made to the compatibility interface, i'd expect this
> type of bug to be there first.

OK.

It sounds like you are very committed to making tis work. That
means a lot to me and is greatly appreciated.

Jim


-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From pinard at iro.umontreal.ca  Wed Jul 14 14:22:27 2004
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Wed Jul 14 15:18:12 2004
Subject: [Python-Dev] PEP 292 for Python 2.4
In-Reply-To: <40F501EB.4080700@iinet.net.au>
References: <1087414523.7150.45.camel@localhost>
	<001f01c4538d$f1fcfec0$bfaf2c81@oemcomputer>
	<20040624142741.GC27922@panix.com>
	<20040713130439.GA2174@titan.progiciels-bpi.ca>
	<40F501EB.4080700@iinet.net.au>
Message-ID: <20040714122227.GA12473@titan.progiciels-bpi.ca>

[Nick Coghlan]

> Fran?ois Pinard wrote:

> >I know that `string' and `socket' [modules] exist, despite `string'
> >is evanescent, but they surely forced users at choosing other
> >identifiers where `string' and `socket' would have been perfect.

> I would suggest that bare type names are rarely appropriate for use a
> variable names, except in toy examples.

Or small enough functions.  Small functions are not necessarily toys.

> If I'm reading someone else's code, and they create a string or a
> socket, I want to know what it is _for_, rather than the mere fact
> this it is a string or a socket.

If I write a function receiving a string as an argument, and the effect
of the function being already documented, I see no point writing
`parameter_string' or `the_argument_of_the_function' instead of
`string', which is clear, clean and simple.  Some people would write
`s' instead, but for one, I stopped overly liking algebraic notation in
programs after I left FORTRAN :-). When you speak to someone else about
the argument of a simple function, don't you say "then the function
takes the string, it massages the string this way, etc.".  I like naming
my variables the way I would speak about them! :-)

> If the type is all that is important, then prepending some simple word
> such as 'a_string' or 'the_string' or 'my_string' makes it clear to
> the maintainer that the object doesn't really have any significant
> semantic meaning beyond its type.

Come on, be serious! :-)

-- 
Fran?ois Pinard   http://www.iro.umontreal.ca/~pinard
From aahz at pythoncraft.com  Wed Jul 14 15:55:57 2004
From: aahz at pythoncraft.com (Aahz)
Date: Wed Jul 14 15:56:01 2004
Subject: [Python-Dev] Developer looking to help fix bugs
In-Reply-To: <1089688837.2657.5.camel@localhost.localdomain>
References: <1089688837.2657.5.camel@localhost.localdomain>
Message-ID: <20040714135557.GA24990@panix.com>

On Mon, Jul 12, 2004, Mike Mangino wrote:
>
> Earlier today I added a comment to http://www.python.org/sf/982679 to
> tell the user how to fix the problem they were having (it was a bug in
> their code, not python) How do I have somebody close that request? In
> the future, if I fix one of the issues, is there somebody I should email
> to make changes?

BTW, have you read http://www.python.org/dev/dev_intro.html

Was it helpful?  If not, what was missing?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Typing is cheap.  Thinking is expensive."  --Roy Smith, c.l.py
From guido at python.org  Wed Jul 14 17:40:57 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Jul 14 17:41:02 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: Your message of "Wed, 14 Jul 2004 11:16:29 +0200."
	<40F4F9ED.2020505@egenix.com> 
References: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>  
	<40F4F9ED.2020505@egenix.com> 
Message-ID: <200407141540.i6EFeve27174@guido.python.org>

> I'd rather not like to see hackery like misused pointers
> in the core, so you can count me as -1 on this.

-1000 here.  In a past life (the ABC implementation) we used this and
it was a horrible experience.  Basically, there were too many places
where you had to make sure you didn't have an integer before
dereferencing a pointer, and finding the omissions one core dump at a
time was a nightmare.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Wed Jul 14 18:22:03 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Jul 14 18:18:57 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <200407141540.i6EFeve27174@guido.python.org>
References: <Your message of "Wed,
	14 Jul 2004 11:16:29 +0200." <40F4F9ED.2020505@egenix.com>
	<D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
	<40F4F9ED.2020505@egenix.com>
Message-ID: <5.1.1.6.0.20040714115351.01e9d330@mail.telecommunity.com>

At 08:40 AM 7/14/04 -0700, Guido van Rossum wrote:
> > I'd rather not like to see hackery like misused pointers
> > in the core, so you can count me as -1 on this.
>
>-1000 here.  In a past life (the ABC implementation) we used this and
>it was a horrible experience.  Basically, there were too many places
>where you had to make sure you didn't have an integer before
>dereferencing a pointer, and finding the omissions one core dump at a
>time was a nightmare.

That certainly seems like it would be so, except...  doesn't every bit of 
Python code that's doing anything *other* than accessing ob_type or 
ob_refcnt have to first check if ob_type is the type that code is looking 
for anyway?

And, don't the vast majority of accesses to ob_type and ob_refcnt occur 
inside Python core API macros and functions anyway?

If in addition to Mr. Knight's patch, ob_type and ob_refcnt were renamed so 
as to break any existing direct access in the core or extension modules, 
upgrading them to use Py_GETTYPE and Py_GETREF would be straightforward 
because one would "find the omissions" one *compilation error* at a time.

Further, renaming those fields to something like 'XXXusePy_GETTYPE_instead' 
would provide a built-in hint that you weren't supposed to use the field 
directly.  :)

An additional trick: make it so that the macros for defining new object 
types still create an ob_type field, but let PyObject * point to the 
structure with the XXX fields.  Thus, you can only access ob_type directly 
if you've already cast to a non PyObject * type.  (i.e., you've already 
used an appropriate PyXXX_Check on that variable and would like to use it 
as a regular object now).

Naturally, the issues of portability and what impact Mr. Knight's approach 
has on non-integer code would need to be taken into consideration as well, 
but it seems to me that this approach *could* be made type-safe, at the 
cost of some initial pain to update the approximately 640 uses of 'ob_type' 
in the current source base.

From jepler at unpythonic.net  Wed Jul 14 18:28:27 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Wed Jul 14 18:28:37 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
References: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
Message-ID: <20040714162827.GF26214@unpythonic.net>

I tried this back in 2002.

http://mail.python.org/pipermail/python-dev/2002-August/027685.html

Excerpt:
[...]
Subject: [Python-Dev] Performance (non)optimization: 31-bit ints in pointers
|
| Performance results are mixed.  A small program designed to test the
| speed of all-integer arithmetic comes out faster by 14% (3.38 vs 2.90
| "user" time on my machine) but pystone comes out 5% slower (14124 vs 13358
| "pystones/second").
| 
| I don't know if anybody's barked up this tree before, but I think
| these results show that it's almost certainly not worth the effort to
| incorporate this "performance" hack in Python.  I'll keep my tree around
| for awhile, in case anybody else wants to see it, but beware that it
| still has serious issues even in the core:
|     >>> 0+0j
|     Traceback (most recent call last):
|       File "<stdin>", line 1, in ?
|     TypeError: unsupported operand types for +: 'int' and 'complex'
|     >>> (0).__class__
|     Segmentation fault

Guido replied:
| We used *exactly* this approach in ABC.  I decided not to go with it
| in Python, for two reasons that are essentially what you write up
| here: (1) the changes are very pervasive (in ABC, we kept finding
| places where we had pointer-manipulating code that had to be fixed to
| deal with the small ints), and (2) it wasn't at all clear if it was a
| performance win in the end (all the extra tests and special cases
| may cost as much as you gain).
http://mail.python.org/pipermail/python-dev/2002-August/027686.html

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040714/47519ac4/attachment.pgp
From squirrel at WPI.EDU  Wed Jul 14 20:36:51 2004
From: squirrel at WPI.EDU (Christopher T King)
Date: Wed Jul 14 20:36:55 2004
Subject: [Python-Dev] Proper tail recursion
Message-ID: <Pine.LNX.4.44.0407141423310.20797-100000@ccc1.wpi.edu>

JanC recommended I post this on python-dev to get feedback.  To sum up the 
previous posts in my thread on comp.lang.python, I've created a patch that 
optimizes tail calls in the CPython interpreter, so that the stack is not 
used for functions called in a tail context.

The current patch is only a partial implementation, and might have some
memory leaks with regards to exceptions (I haven't compared this to
vanilla 2.4a1's behaviour yet), but works for simple test cases.  If such
behaviour is deemed desirable, I'll finish the patch, clean up the code,
and work out any bugs.

The relevant portion of the original message follows.

---

I have a preliminary implementation ready, the patch is against 2.4a1: 
http://users.wpi.edu/~squirrel/temp/tailcall.diff.gz

This patch only works for simple functions (i.e. those that can be 
handled by fast_function), but extension to other function types should 
be trivial. I haven't fully tested this with regards to exception 
handling and whatnot, so I want to stick with a simple case for now.

The patch works roughly as follows:
1. When a CALL_FUNCTION opcode is encountered, check if the following 
opcode is RETURN_VALUE. If so, execute the tail call code.
2. The tail call code calls a modified version of call_function that 
sets up and returns a frame object for the function to be called (if 
possible), rather than recursively calling PyEval_EvalFrame. This frame 
is stored in a temporary variable, and a RETURN_VALUE is simulated to 
exit the loop.
3. After cleaning up the current frame, PyEval_EvalFrame loops back up 
to the top, now using the temporarily stored frame as the current frame.

Of course, instead of a loop, gcc's tail-call optimization feature could 
be used, but this would be non-portable to other compilers.

An example of the patch in action:

# Note default arguments aren't supported in the current patch
def fact2(n,v):
     if n:
         return fact2(n-1,v*n)
     else:
         return v

def fact(n):
     return fact2(n,1)

Without the patch:
 >>> fact(10000)
RuntimeError: maximum recursion depth exceeded

With the patch:
 >>> fact(10000)
<really really huge number>

Any feedback would be greatly appreciated!


From mwh at python.net  Wed Jul 14 20:45:01 2004
From: mwh at python.net (Michael Hudson)
Date: Wed Jul 14 20:45:03 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <Pine.LNX.4.44.0407141423310.20797-100000@ccc1.wpi.edu>
	(Christopher
	T. King's message of "Wed, 14 Jul 2004 14:36:51 -0400 (EDT)")
References: <Pine.LNX.4.44.0407141423310.20797-100000@ccc1.wpi.edu>
Message-ID: <2m4qoawhqq.fsf@starship.python.net>

Christopher T King <squirrel@WPI.EDU> writes:

> JanC recommended I post this on python-dev to get feedback.  To sum
> up the previous posts in my thread on comp.lang.python, I've created
> a patch that optimizes tail calls in the CPython interpreter, so
> that the stack is not used for functions called in a tail context.

I guess I could have said this in c.l.py, but: what's the point?  I
don't think I've ever had a situation where hitting the recursion
limit wasn't a bug, and given a) the ease with which a non-contrivedly
tail recursive algorithm can usually be rewritten iteratively and b)
the (lack of) speed of function calls in today's Python I can't see
any huge real advantage to this patch.

It's a cute hack though :-)

Cheers,
mwh

-- 
  if-you-need-your-own-xxx.py-you-know-where-to-shove-it<wink>-ly 
  y'rs - tim
              -- Tim Peters dishes out versioning advice on python-dev
From michael.walter at gmail.com  Wed Jul 14 20:46:29 2004
From: michael.walter at gmail.com (Michael Walter)
Date: Wed Jul 14 20:46:33 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <2m4qoawhqq.fsf@starship.python.net>
References: <Pine.LNX.4.44.0407141423310.20797-100000@ccc1.wpi.edu>
	<2m4qoawhqq.fsf@starship.python.net>
Message-ID: <877e9a1704071411462e7e9f62@mail.gmail.com>

I think it's pretty cool as it gives the programmer more freetime to
use "functional style".

Cheers,
Michael

On Wed, 14 Jul 2004 19:45:01 +0100, Michael Hudson <mwh@python.net> wrote:
> Christopher T King <squirrel@WPI.EDU> writes:
> 
> > JanC recommended I post this on python-dev to get feedback.  To sum
> > up the previous posts in my thread on comp.lang.python, I've created
> > a patch that optimizes tail calls in the CPython interpreter, so
> > that the stack is not used for functions called in a tail context.
> 
> I guess I could have said this in c.l.py, but: what's the point?  I
> don't think I've ever had a situation where hitting the recursion
> limit wasn't a bug, and given a) the ease with which a non-contrivedly
> tail recursive algorithm can usually be rewritten iteratively and b)
> the (lack of) speed of function calls in today's Python I can't see
> any huge real advantage to this patch.
> 
> It's a cute hack though :-)
> 
> Cheers,
> mwh
> 
> --
>   if-you-need-your-own-xxx.py-you-know-where-to-shove-it<wink>-ly
>   y'rs - tim
>               -- Tim Peters dishes out versioning advice on python-dev
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/michael.walter%40gmail.com
>
From squirrel at WPI.EDU  Wed Jul 14 21:13:37 2004
From: squirrel at WPI.EDU (Christopher T King)
Date: Wed Jul 14 21:14:20 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <2m4qoawhqq.fsf@starship.python.net>
Message-ID: <Pine.LNX.4.44.0407141501470.21617-100000@ccc1.wpi.edu>

On Wed, 14 Jul 2004, Michael Hudson wrote:

> I guess I could have said this in c.l.py, but: what's the point?  I
> don't think I've ever had a situation where hitting the recursion
> limit wasn't a bug, and given a) the ease with which a non-contrivedly
> tail recursive algorithm can usually be rewritten iteratively and b)
> the (lack of) speed of function calls in today's Python I can't see
> any huge real advantage to this patch.

To tell the truth, I don't really have any immediate use for this
functionality, either, but since it turned out to be so easy to implement,
I ask, "why not?" ;) Presuming it doesn't break any existing
functionality, of course.* I prefer to think of it in the same light as
the opcode prediction macros, i.e. as a performance enhancement, rather
than as an attempt to introduce a new paradigm to the language (which it
isn't).

* I know it breaks stack traces, but the final implementation will only be 
used when the -O flag is specified, or some other such indicator is set.


From martin at v.loewis.de  Wed Jul 14 21:30:02 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Jul 14 21:30:06 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <Pine.LNX.4.44.0407141501470.21617-100000@ccc1.wpi.edu>
References: <Pine.LNX.4.44.0407141501470.21617-100000@ccc1.wpi.edu>
Message-ID: <40F589BA.8020103@v.loewis.de>

Christopher T King wrote:

> To tell the truth, I don't really have any immediate use for this
> functionality, either, but since it turned out to be so easy to implement,
> I ask, "why not?" ;) 

The traditional answer to that question is "because Jython cannot
support it".

> * I know it breaks stack traces, but the final implementation will only be 
> used when the -O flag is specified, or some other such indicator is set.

That would be bad. It would change the behaviour of programs, which 
should not happen for -O (except for the documented changes, of course,
whose list should be short).

Regards,
Martin

From squirrel at WPI.EDU  Wed Jul 14 21:53:19 2004
From: squirrel at WPI.EDU (Christopher T King)
Date: Wed Jul 14 21:54:17 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <40F589BA.8020103@v.loewis.de>
Message-ID: <Pine.LNX.4.44.0407141540210.21617-100000@ccc1.wpi.edu>

On Wed, 14 Jul 2004, [ISO-8859-1] "Martin v. L=F6wis" wrote:

> The traditional answer to that question is "because Jython cannot
> support it".

Which is why I prefer to view it as a performance enhancement, rather than
a language feature.  Deep recursive algorithms would still run under
Jython; they would just use a heck of a lot more memory than in CPython.


From michael.walter at gmail.com  Wed Jul 14 22:12:22 2004
From: michael.walter at gmail.com (Michael Walter)
Date: Wed Jul 14 22:12:26 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <Pine.LNX.4.44.0407141540210.21617-100000@ccc1.wpi.edu>
References: <Pine.LNX.4.44.0407141540210.21617-100000@ccc1.wpi.edu>
Message-ID: <877e9a170407141312c439e8a@mail.gmail.com>

Is is strictly impossible to implement tail recursion elimination in
Jython with its current design (which I'm not familiar with)?

Cheers,
Michael

On Wed, 14 Jul 2004 15:53:19 -0400 (EDT), Christopher T King
<squirrel@wpi.edu> wrote:
> On Wed, 14 Jul 2004, [ISO-8859-1] "Martin v. L?wis" wrote:
> 
> > The traditional answer to that question is "because Jython cannot
> > support it".
> 
> Which is why I prefer to view it as a performance enhancement, rather than
> a language feature.  Deep recursive algorithms would still run under
> Jython; they would just use a heck of a lot more memory than in CPython.
> 
> 
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/michael.walter%40gmail.com
>
From python at rcn.com  Wed Jul 14 10:23:45 2004
From: python at rcn.com (Raymond Hettinger)
Date: Wed Jul 14 22:25:15 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Include
	objimpl.h, 2.58, 2.59
In-Reply-To: <E1Bkp75-000469-Sv@sc8-pr-cvs1.sourceforge.net>
Message-ID: <001401c4697b$e1ee4800$e841fea9@oemcomputer>

> Documented the new Py_VISIT macro to simplify implementation of
> tp_traverse handlers. (Tim made me do it. ;)

That Tim has been tyrant lately ;-)



> + /* Utility macro to help write tp_traverse functions */
> + #define Py_VISIT(op)					\
> +         do { 						\
> +                 if (op) {				\
> +                         int vret = visit((op), arg);	\
> +                         if (vret)			\
> +                                 return vret;		\
> +                 }					\
> +         } while (0)

I tried out this macro with the itertools module and found that it did a
nice job of compacting boilerplate code.

One thought jumped out though.  The macro assumes that the enclosing
function has consistent with the use of the variable names "visit" and
"arg": 

  static int
  islice_traverse(isliceobject *lz, visitproc visit, void *arg)
  {
      Py_VISIT(lz->it);
      return 0;
  }

If the parameters are named something else ("vfunc" for example),
MSVC++6.0 complains:

C:\py24\Modules\itertoolsmodule.c(1118) : warning C4013: 'visit'
undefined; assuming extern returning int

Don't know if it is worth it, but you could safeguard the macro by
including visit and arg in the parameter list.


Raymond Hettinger

From martin at v.loewis.de  Wed Jul 14 22:31:14 2004
From: martin at v.loewis.de (Martin v. Löwis)
Date: Wed Jul 14 22:31:18 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <Pine.LNX.4.44.0407141540210.21617-100000@ccc1.wpi.edu>
References: <Pine.LNX.4.44.0407141540210.21617-100000@ccc1.wpi.edu>
Message-ID: <40F59812.6070809@v.loewis.de>

Christopher T King wrote:
> Which is why I prefer to view it as a performance enhancement, rather than
> a language feature.  Deep recursive algorithms would still run under
> Jython; they would just use a heck of a lot more memory than in CPython.

In that case, any implementation of tail recursion should still consider
the recursion limit.

Regards,
Martin

From barry at python.org  Wed Jul 14 23:29:28 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Jul 14 23:29:28 2004
Subject: [Python-Dev] 
	Re: [Python-checkins] python/dist/src/Include object.h, 2.126, 2.127
In-Reply-To: <E1Bkp6t-00042J-Sw@sc8-pr-cvs1.sourceforge.net>
References: <E1Bkp6t-00042J-Sw@sc8-pr-cvs1.sourceforge.net>
Message-ID: <1089840568.8804.135.camel@localhost>

On Wed, 2004-07-14 at 15:08, dcjim@users.sourceforge.net wrote:

> Log Message:
> Implemented a new Py_CLEAR macro. This macro should be used when 
> decrementing the refcount of variables that might be accessed as a
> result of calling Python

Didn't Guido pronounce that it should be Py_ZAP instead?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040714/27516bd7/attachment.pgp
From joseph_klepach at msn.com  Sun Jul 11 04:15:37 2004
From: joseph_klepach at msn.com (Joseph Klepach)
Date: Wed Jul 14 23:35:12 2004
Subject: [Python-Dev] Python 2.4a1 use of c:\winnt directory
Message-ID: <BAY4-F11Wm2VenfrgeT00071ddc@hotmail.com>

[Klepach, Joseph]
>>The python 2.4a1 release using the msi installer insists on copying
>>python24.dll to the C:\WINNT\system32 directory instead the of requested 
>>user
>>directory.
>>
>>This did not occur in the past and is causing a problem because my company
>>locks the c:\winnt directory on all user pc's.
...
>>Will this functionality be part of the official release?

[Martin v. Löwis]
>Yes. Chose a per-user installation if you don't want to install Python
>into the system directory.

Ah, OK, you're saying this is *already* available ... and it is.
Joseph, when you run the .msi thingie, click the Advanced button in
the second screen.  Then there's a radio button controlling whether to
install for all users, or just the current user.  I selected the
latter, and python24.dll was installed under my Python24 directory
instead of under a system directory.

[Joseph Klepach]
That's what I tried to do. Since I do not have admin privileges on my C:\ 
drive, I expected the python24.dll to be installed under my Python24 
directory.
The Advanced button informed me that I did not have admin privileges and 
would install under my local user directory: it did not.
The only other thing different was that I did not use the default 
c:\python24 directory, but chose another one.

Regards

_________________________________________________________________
FREE pop-up blocking with the new MSN Toolbar – get it now! 
http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/

From squirrel at wpi.edu  Sat Jul 10 06:55:50 2004
From: squirrel at wpi.edu (Chris King)
Date: Wed Jul 14 23:35:55 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
Message-ID: <40EF76D6.7030900@wpi.edu>

While recently goofing around with the bytecode, I thought of doing 
something like this:

  case LOAD_CONST:
      x = GETITEM(consts, oparg);
      Py_INCREF(x);
+     if (*next_instr == RETURN_VAL) {
+         retval = x;
+         why = WHY_RETURN;
+         goto fast_block_end;
+     }
      PUSH(x);
      goto fast_next_opcode;

This would skip the stack and a trip through the loop without changing 
the parser or the bytecode,  and with a minimal amount of added code or 
overhead. This could (of course) be applied to other opcodes, too.

Perhaps instead of littering the function with that block, a macro 
"PUSH_MAYBE_RET" could be added that would replace the final PUSH in the 
opcode's case block:

#define PUSH_MAYBE_RET { if (*next_instr == RETURN_VAL) { \
                           retval = x; \
                           why = WHY_RETURN; \
                           goto fast_block_end; } \
                       PUSH(x); }

Not sure how much this would help speed, if any.
From skip at pobox.com  Wed Jul 14 23:45:23 2004
From: skip at pobox.com (Skip Montanaro)
Date: Wed Jul 14 23:45:28 2004
Subject: [Python-Dev] Pie-thon at OSCON 2004?
Message-ID: <16629.43379.604258.804345@montanaro.dyndns.org>


Have Guido and Dan been practicing their aim with a pie?  The topic of the
Python/Parrot shootout hasn't come up in awhile.  I thought I'd ask since
OSCON 2K4 is only a couple weeks away.

Skip
From nbastin at opnet.com  Thu Jul 15 00:01:14 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Thu Jul 15 00:01:30 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <40EF76D6.7030900@wpi.edu>
References: <40EF76D6.7030900@wpi.edu>
Message-ID: <5302CD9C-D5E1-11D8-9B88-000D932927FE@opnet.com>


On Jul 10, 2004, at 12:55 AM, Chris King wrote:

> While recently goofing around with the bytecode, I thought of doing 
> something like this:
>
>  case LOAD_CONST:
>      x = GETITEM(consts, oparg);
>      Py_INCREF(x);
> +     if (*next_instr == RETURN_VAL) {
> +         retval = x;
> +         why = WHY_RETURN;
> +         goto fast_block_end;
> +     }
>      PUSH(x);
>      goto fast_next_opcode;
>
> This would skip the stack and a trip through the loop without changing 
> the parser or the bytecode,  and with a minimal amount of added code 
> or overhead. This could (of course) be applied to other opcodes, too.

This seems unlikely to have any effect whatsoever.  I would venture 
that the vast majority of LOAD_CONST found in compiled bytecode would 
be for the implicit 'return none', which oftentimes never gets executed 
anyhow, because it follows an explicit RETURN_VALUE.  A cursory run 
through some python I have here indicates that of the LOAD_CONST 
instructions that are actually executed, very few are followed by a 
RETURN_VALUE (usually a BINARY_ADD, or STORE_FAST, or the like).

Applying it to other opcodes (like what, LOAD_FAST?) seems like a bad 
idea, since my past experience with profiling the main interpreter loop 
tells me that a majority of the trailing opcodes would have to be 
predicted correctly to offset the expense of the check, and I don't 
know if that's possible.  Obviously, if someone can find a strong 
correlation, great, but I don't think this is one.

--
Nick

From amk at amk.ca  Thu Jul 15 00:30:19 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Thu Jul 15 00:31:09 2004
Subject: [Python-Dev] Pie-thon at OSCON 2004?
In-Reply-To: <16629.43379.604258.804345@montanaro.dyndns.org>
References: <16629.43379.604258.804345@montanaro.dyndns.org>
Message-ID: <20040714223019.GA17521@rogue.amk.ca>

On Wed, Jul 14, 2004 at 04:45:23PM -0500, Skip Montanaro wrote:
> Have Guido and Dan been practicing their aim with a pie?  The topic of the
> Python/Parrot shootout hasn't come up in awhile.  I thought I'd ask since
> OSCON 2K4 is only a couple weeks away.

An outdated weblog entry I wrote:
http://www.amk.ca/diary/archives/cat_python.html#003176.  I'll try to
write another update once I actually get it to run Python code.

Serious development activity in Parrot CVS started a few days before
the end of June.  At this point there are a few basic data types
(string, int, float, lists, complex), a subset of the built-in
functions (divmod, hex, chr, etc. -- map/filter/reduce are the most
complicated ones implemented), and exceptions work to some degree.  It
can run a version of the pi() function in b0.py that's been rewritten
to not use yield.  Single-parent classes are being implemented at this
point, but it's not clear whether they work at all; ask again in a few
days.

Can they finish within the time available?  I'm not sure; Python is a
relatively small language but it's not *that* small, and there's still
a lot left to do.  The approach being taken is translating the
bytecode; there's a languages/python/piethon.pl script that takes a
.pyc and produces a Parrot assembly file.  There's some subtlety to
invoking it, though, because the PASM produced doesn't actually
assemble for me.  I haven't yet figured out how to run a snippet of
Python code using Parrot, so this assessment comes from looking at the
code.  (This means I obviously don't have any performance numbers
either.)

I forget the terms of the bet; is it running on Python 2.3.4, or the
2.4alpha1 release, or on Python CVS?  Maybe we should check that
2.4CVS hasn't regressed in performance when compared to 2.3.

--amk

From foom at fuhm.net  Thu Jul 15 00:46:43 2004
From: foom at fuhm.net (James Y Knight)
Date: Thu Jul 15 00:46:49 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <20040714162827.GF26214@unpythonic.net>
References: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
	<20040714162827.GF26214@unpythonic.net>
Message-ID: <AD872C9A-D5E7-11D8-B5AB-000A95A50FB2@fuhm.net>

On Jul 14, 2004, at 12:28 PM, Jeff Epler wrote:
> I tried this back in 2002.
> http://mail.python.org/pipermail/python-dev/2002-August/027685.html

Interesting that I somehow missed that in my search. I had thought 
surely someone must've tried it already. Ah well. :) But, I'm glad I 
did it, because I do have somewhat different results.

1) No segfaults or other weird behavior (all tests pass)
2) With tagged integers does actually benchmark faster than without 
(although, not by all that much).

The version of the patch I sent earlier was indeed slightly slower at 
pystone than standard python. However, I have made a few changes that 
have since made it faster, instead:
1) made Py_tagged_types a const array.
2) removed inlining of creation of small ints for PyInt_FromLong. Turns 
out that makes it go slower not faster.

Anyhow, current result is a 3% overall speedup of pystone. 35791 
pystones/second (standard python 2.3.3), vs. 36900 pystones/second 
(tagged integers).

I also tried running pybench -w 5. Here's the results, compared to 
standard python 2.3.3. Overall speedup of 7.19%. I must admit I find 
these results quite puzzling, as I certainly didn't expect it to affect 
list slicing that much. Also, that Compare* are so much slower seems 
strange as well. Perhaps there's something that can be done about that.


Tests:                              per run    per oper.    diff *)
------------------------------------------------------------------------
           BuiltinFunctionCalls:     227.75 ms    0.45 us    -3.72%
            BuiltinMethodLookup:     386.30 ms    0.18 us    +3.33%
                  CompareFloats:     283.50 ms    0.16 us   +57.46%
          CompareFloatsIntegers:     343.35 ms    0.19 us    -1.07%
                CompareIntegers:     261.90 ms    0.07 us    -0.44%
         CompareInternedStrings:     210.10 ms    0.11 us   +19.68%
                   CompareLongs:     244.45 ms    0.14 us   +30.65%
                 CompareStrings:     338.05 ms    0.17 us   +11.73%
                 CompareUnicode:     265.35 ms    0.18 us   +17.70%
                  ConcatStrings:     456.20 ms    0.76 us    -3.31%
                  ConcatUnicode:     564.00 ms    0.94 us    -2.92%
                CreateInstances:     300.30 ms    1.79 us    -2.52%
        CreateStringsWithConcat:     178.35 ms    0.22 us    +4.54%
        CreateUnicodeWithConcat:     559.35 ms    0.70 us    +3.84%
                   DictCreation:     230.50 ms    0.38 us   -12.02%
              DictWithFloatKeys:     504.20 ms    0.21 us    -5.86%
            DictWithIntegerKeys:     225.10 ms    0.09 us   -22.82%
             DictWithStringKeys:     247.20 ms    0.10 us    -7.98%
                       ForLoops:     183.85 ms    4.60 us   -18.97%
                     IfThenElse:     211.60 ms    0.08 us    -2.78%
                    ListSlicing:      81.70 ms    5.84 us   -60.64%
                 NestedForLoops:     141.95 ms    0.09 us   -12.38%
           NormalClassAttribute:     296.05 ms    0.12 us    -4.25%
        NormalInstanceAttribute:     266.15 ms    0.11 us   -10.51%
            PythonFunctionCalls:     336.45 ms    0.51 us   -10.05%
              PythonMethodCalls:     277.80 ms    0.93 us    -1.35%
                      Recursion:     233.50 ms    4.67 us   -18.99%
                   SecondImport:     191.85 ms    1.92 us    +4.44%
            SecondPackageImport:     205.75 ms    2.06 us    +3.03%
          SecondSubmoduleImport:     255.75 ms    2.56 us    +3.67%
        SimpleComplexArithmetic:     206.70 ms    0.23 us    +4.21%
         SimpleDictManipulation:     172.50 ms    0.14 us   -14.22%
          SimpleFloatArithmetic:     296.15 ms    0.13 us    +8.32%
       SimpleIntFloatArithmetic:     183.85 ms    0.07 us   -33.65%
        SimpleIntegerArithmetic:     182.00 ms    0.07 us   -32.62%
         SimpleListManipulation:     217.50 ms    0.20 us   -10.12%
           SimpleLongArithmetic:     161.70 ms    0.25 us    +6.73%
                     SmallLists:     562.85 ms    0.55 us   +13.40%
                    SmallTuples:     468.30 ms    0.49 us   +15.70%
          SpecialClassAttribute:     289.00 ms    0.12 us    -5.97%
       SpecialInstanceAttribute:     460.00 ms    0.19 us    -1.33%
                 StringMappings:     319.10 ms    0.63 us    -0.17%
               StringPredicates:     297.50 ms    0.27 us    -0.77%
                  StringSlicing:     270.00 ms    0.39 us    -1.93%
                      TryExcept:     203.20 ms    0.03 us    +0.15%
                 TryRaiseExcept:     217.40 ms    3.62 us   -28.71%
                   TupleSlicing:     223.55 ms    0.53 us   -42.15%
                UnicodeMappings:     250.10 ms    3.47 us    +0.44%
              UnicodePredicates:     257.60 ms    0.29 us    +2.79%
              UnicodeProperties:     398.55 ms    0.50 us    +6.69%
                 UnicodeSlicing:     395.95 ms    0.57 us    +1.71%
------------------------------------------------------------------------
             Average round time:   15663.00 ms               -7.19%


For comparison, here's the results from standard python:
Tests:                              per run    per oper.   overhead
------------------------------------------------------------------------
           BuiltinFunctionCalls:     236.55 ms    0.46 us    0.50 ms
            BuiltinMethodLookup:     373.85 ms    0.18 us    1.00 ms
                  CompareFloats:     180.05 ms    0.10 us    1.00 ms
          CompareFloatsIntegers:     347.05 ms    0.19 us    1.00 ms
                CompareIntegers:     263.05 ms    0.07 us    2.00 ms
         CompareInternedStrings:     175.55 ms    0.09 us    3.50 ms
                   CompareLongs:     187.10 ms    0.10 us    1.00 ms
                 CompareStrings:     302.55 ms    0.15 us    3.50 ms
                 CompareUnicode:     225.45 ms    0.15 us    2.50 ms
                  ConcatStrings:     471.80 ms    0.79 us    1.00 ms
                  ConcatUnicode:     580.95 ms    0.97 us    1.00 ms
                CreateInstances:     308.05 ms    1.83 us    1.00 ms
        CreateStringsWithConcat:     170.60 ms    0.21 us    1.00 ms
        CreateUnicodeWithConcat:     538.65 ms    0.67 us    1.00 ms
                   DictCreation:     262.00 ms    0.44 us    1.00 ms
              DictWithFloatKeys:     535.60 ms    0.22 us    3.50 ms
            DictWithIntegerKeys:     291.65 ms    0.12 us    3.50 ms
             DictWithStringKeys:     268.65 ms    0.11 us    3.00 ms
                       ForLoops:     226.90 ms    5.67 us    0.50 ms
                     IfThenElse:     217.65 ms    0.08 us    2.50 ms
                    ListSlicing:     207.55 ms   14.83 us    0.50 ms
                 NestedForLoops:     162.00 ms    0.11 us    0.00 ms
           NormalClassAttribute:     309.20 ms    0.13 us    1.50 ms
        NormalInstanceAttribute:     297.40 ms    0.12 us    1.50 ms
            PythonFunctionCalls:     374.05 ms    0.57 us    1.00 ms
              PythonMethodCalls:     281.60 ms    0.94 us    0.50 ms
                      Recursion:     288.25 ms    5.77 us    0.50 ms
                   SecondImport:     183.70 ms    1.84 us    0.50 ms
            SecondPackageImport:     199.70 ms    2.00 us    0.50 ms
          SecondSubmoduleImport:     246.70 ms    2.47 us    0.50 ms
        SimpleComplexArithmetic:     198.35 ms    0.23 us    1.00 ms
         SimpleDictManipulation:     201.10 ms    0.17 us    0.50 ms
          SimpleFloatArithmetic:     273.40 ms    0.12 us    2.00 ms
       SimpleIntFloatArithmetic:     277.10 ms    0.10 us    2.00 ms
        SimpleIntegerArithmetic:     270.10 ms    0.10 us    2.00 ms
         SimpleListManipulation:     242.00 ms    0.22 us    1.00 ms
           SimpleLongArithmetic:     151.50 ms    0.23 us    0.50 ms
                     SmallLists:     496.35 ms    0.49 us    2.50 ms
                    SmallTuples:     404.75 ms    0.42 us    1.50 ms
          SpecialClassAttribute:     307.35 ms    0.13 us    2.00 ms
       SpecialInstanceAttribute:     466.20 ms    0.19 us    2.00 ms
                 StringMappings:     319.65 ms    0.63 us    1.50 ms
               StringPredicates:     299.80 ms    0.27 us    5.00 ms
                  StringSlicing:     275.30 ms    0.39 us    1.50 ms
                      TryExcept:     202.90 ms    0.03 us    3.00 ms
                 TryRaiseExcept:     304.95 ms    5.08 us    1.00 ms
                   TupleSlicing:     386.40 ms    0.92 us    0.50 ms
                UnicodeMappings:     249.00 ms    3.46 us    1.00 ms
              UnicodePredicates:     250.60 ms    0.28 us    6.50 ms
              UnicodeProperties:     373.55 ms    0.47 us    6.50 ms
                 UnicodeSlicing:     389.30 ms    0.56 us    2.00 ms
------------------------------------------------------------------------
             Average round time:   16876.00 ms

James


From skip at pobox.com  Thu Jul 15 01:31:38 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Jul 15 01:31:44 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <40EF76D6.7030900@wpi.edu>
References: <40EF76D6.7030900@wpi.edu>
Message-ID: <16629.49754.921324.158376@montanaro.dyndns.org>


    Chris> While recently goofing around with the bytecode, I thought of
    Chris> doing something like this:

    Chris>   case LOAD_CONST:
    Chris>       x = GETITEM(consts, oparg);
    Chris>       Py_INCREF(x);
    Chris> +     if (*next_instr == RETURN_VAL) {
    Chris> +         retval = x;
    Chris> +         why = WHY_RETURN;
    Chris> +         goto fast_block_end;
    Chris> +     }
    Chris>       PUSH(x);
    Chris>       goto fast_next_opcode;

    Chris> This would skip the stack and a trip through the loop without
    Chris> changing the parser or the bytecode, and with a minimal amount of
    Chris> added code or overhead. 

You'd have to test it.  You'd speed up getting to the return point at the
expense of slowing down every other LOAD_CONST instruction.

Skip
From skip at pobox.com  Thu Jul 15 01:36:22 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Jul 15 01:36:42 2004
Subject: [Python-Dev] Pie-thon at OSCON 2004?
In-Reply-To: <20040714223019.GA17521@rogue.amk.ca>
References: <16629.43379.604258.804345@montanaro.dyndns.org>
	<20040714223019.GA17521@rogue.amk.ca>
Message-ID: <16629.50038.912854.6179@montanaro.dyndns.org>


    amk> I forget the terms of the bet; is it running on Python 2.3.4, or the
    amk> 2.4alpha1 release, or on Python CVS?  Maybe we should check that
    amk> 2.4CVS hasn't regressed in performance when compared to 2.3.

I  saw somewhere today when  searching (maybe Dan S's  website?) that it was
2.3 as of about December 2003.

Skip
From python at rcn.com  Wed Jul 14 13:56:50 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Jul 15 01:58:19 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <40EF76D6.7030900@wpi.edu>
Message-ID: <002f01c46999$a694d120$e841fea9@oemcomputer>

>   case LOAD_CONST:
>       x = GETITEM(consts, oparg);
>       Py_INCREF(x);
> +     if (*next_instr == RETURN_VAL) {
> +         retval = x;
> +         why = WHY_RETURN;
> +         goto fast_block_end;
> +     }
>       PUSH(x);
>       goto fast_next_opcode;
> 
> This would skip the stack and a trip through the loop without changing
> the parser or the bytecode,  and with a minimal amount of added code
or
> overhead. This could (of course) be applied to other opcodes, too.

This approach is a total disaster.  It adds overhead (an unpredictable)
branch to one of the most common opcodes.  It burdens the common case to
save only a tiny bit in a special case.  LOAD_CONST and RETURN_VAL are
already among the fastest bytecodes.

The returning of values is not where Python code is spending all its
time.  Better to focus on attribute lookup and function calls.


Raymond

From t-meyer at ihug.co.nz  Thu Jul 15 02:17:11 2004
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Thu Jul 15 02:17:32 2004
Subject: [Python-Dev] Pie-thon at OSCON 2004?
In-Reply-To: <1ED4ECF91CDED24C8D012BCF2B034F13071A3726@its-xchg4.massey.ac.nz>
Message-ID: <1ED4ECF91CDED24C8D012BCF2B034F1304678142@its-xchg4.massey.ac.nz>

[amk]
> I forget the terms of the bet; is it running on Python 2.3.4, or the
> 2.4alpha1 release, or on Python CVS?  Maybe we should check that
> 2.4CVS hasn't regressed in performance when compared to 2.3.

[Skip]
> I  saw somewhere today when  searching (maybe Dan S's  
> website?) that it was 2.3 as of about December 2003.

That's correct:

<http://mail.python.org/pipermail/python-dev/2003-December/040977.html>
<http://mail.python.org/pipermail/python-dev/2003-December/041527.html>

It's also in the README.txt file in the parrotbench directory.

=Tony Meyer

From greg at cosc.canterbury.ac.nz  Thu Jul 15 03:15:57 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Jul 15 03:16:08 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
Message-ID: <200407150115.i6F1Fv4b010419@cosc353.cosc.canterbury.ac.nz>

James Y Knight <foom@fuhm.net>:

> On the test that probably puts my change in the most positive light 
> possible: "x = 0;  while x < 50000000:  x = x + 1", it achieves about a 
> 50% increase in speed. More normal integer-heavy things seem to be at 
> most 20% faster.

And you have to ask yourself -- how integer-heavy does
typical Python code get?

Most Python code I write deals with higher-level things
most of the time -- strings, lists, dicts, class instances.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From guido at python.org  Thu Jul 15 03:36:16 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul 15 03:36:23 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: Your message of "Wed, 14 Jul 2004 14:36:51 EDT."
	<Pine.LNX.4.44.0407141423310.20797-100000@ccc1.wpi.edu> 
References: <Pine.LNX.4.44.0407141423310.20797-100000@ccc1.wpi.edu> 
Message-ID: <200407150136.i6F1aG928897@guido.python.org>

> JanC recommended I post this on python-dev to get feedback.  To sum up the 
> previous posts in my thread on comp.lang.python, I've created a patch that 
> optimizes tail calls in the CPython interpreter, so that the stack is not 
> used for functions called in a tail context.

I'm not interested in adding this to the official Python release.

One reason is that if an exception happens in such a tail-recursive
call, the stack trace will be confusing.

Another reason is that I don't think it's a good idea to try to
encourage a Scheme-ish "solve everything with recursion" programming
style in Python.

But feel free to maintain this as an independent modification, a la
Stackless -- I'm sure there are people who would like to try this
out.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Jul 15 03:42:54 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul 15 03:43:01 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: Your message of "Wed, 14 Jul 2004 12:22:03 EDT."
	<5.1.1.6.0.20040714115351.01e9d330@mail.telecommunity.com> 
References: <Your message of "Wed,
	14 Jul 2004 11:16:29 +0200." <40F4F9ED.2020505@egenix.com>
	<D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
	<40F4F9ED.2020505@egenix.com> 
	<5.1.1.6.0.20040714115351.01e9d330@mail.telecommunity.com> 
Message-ID: <200407150142.i6F1gs128928@guido.python.org>

> >-1000 here.  In a past life (the ABC implementation) we used this and
> >it was a horrible experience.  Basically, there were too many places
> >where you had to make sure you didn't have an integer before
> >dereferencing a pointer, and finding the omissions one core dump at a
> >time was a nightmare.
> 
> That certainly seems like it would be so, except...  doesn't every bit of 
> Python code that's doing anything *other* than accessing ob_type or 
> ob_refcnt have to first check if ob_type is the type that code is looking 
> for anyway?
> 
> And, don't the vast majority of accesses to ob_type and ob_refcnt occur 
> inside Python core API macros and functions anyway?
> 
> If in addition to Mr. Knight's patch, ob_type and ob_refcnt were renamed so 
> as to break any existing direct access in the core or extension modules, 
> upgrading them to use Py_GETTYPE and Py_GETREF would be straightforward 
> because one would "find the omissions" one *compilation error* at a time.
> 
> Further, renaming those fields to something like 'XXXusePy_GETTYPE_instead' 
> would provide a built-in hint that you weren't supposed to use the field 
> directly.  :)
> 
> An additional trick: make it so that the macros for defining new object 
> types still create an ob_type field, but let PyObject * point to the 
> structure with the XXX fields.  Thus, you can only access ob_type directly 
> if you've already cast to a non PyObject * type.  (i.e., you've already 
> used an appropriate PyXXX_Check on that variable and would like to use it 
> as a regular object now).
> 
> Naturally, the issues of portability and what impact Mr. Knight's approach 
> has on non-integer code would need to be taken into consideration as well, 
> but it seems to me that this approach *could* be made type-safe, at the 
> cost of some initial pain to update the approximately 640 uses of 'ob_type' 
> in the current source base.

Sorry, I'm still not convinced that it's worth to break all the 3rd
party extensions that undoubtedly are doing all sorts of things that
strictly speaking they shouldn't do.

And what about all the extra code generated for Py_DECREF and
Py_INCREF calls?  These now all contain an extra jump.  Horrors!

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Jul 15 03:45:30 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul 15 03:45:37 2004
Subject: [Python-Dev] 
	Re: [Python-checkins] python/dist/src/Include object.h, 2.126, 2.127
In-Reply-To: Your message of "Wed, 14 Jul 2004 17:29:28 EDT."
	<1089840568.8804.135.camel@localhost> 
References: <E1Bkp6t-00042J-Sw@sc8-pr-cvs1.sourceforge.net>  
	<1089840568.8804.135.camel@localhost> 
Message-ID: <200407150145.i6F1jUF28961@guido.python.org>

> > Implemented a new Py_CLEAR macro. This macro should be used when=20
> > decrementing the refcount of variables that might be accessed as a
> > result of calling Python
> 
> Didn't Guido pronounce that it should be Py_ZAP instead?

Clear is fine too.  I proposed zap because someone didn't like clear.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Jul 15 03:47:29 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul 15 03:47:35 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: Your message of "Sat, 10 Jul 2004 00:55:50 EDT."
	<40EF76D6.7030900@wpi.edu> 
References: <40EF76D6.7030900@wpi.edu> 
Message-ID: <200407150147.i6F1lTt28974@guido.python.org>

> While recently goofing around with the bytecode, I thought of doing 
> something like this:
> 
>   case LOAD_CONST:
>       x = GETITEM(consts, oparg);
>       Py_INCREF(x);
> +     if (*next_instr == RETURN_VAL) {
> +         retval = x;
> +         why = WHY_RETURN;
> +         goto fast_block_end;
> +     }
>       PUSH(x);
>       goto fast_next_opcode;
> 
> This would skip the stack and a trip through the loop without changing 
> the parser or the bytecode,  and with a minimal amount of added code or 
> overhead. This could (of course) be applied to other opcodes, too.

At the cost of an extra pointer dereference and jump, and usually for
naught (the uses of LOAD_CONST not in a return statement must
certainly vastly outnumber those in return statements).

Have you timed this?  I doubt you can measure a consistent performance
increase across platforms.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Jul 15 03:49:14 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul 15 03:49:21 2004
Subject: [Python-Dev] Pie-thon at OSCON 2004?
In-Reply-To: Your message of "Wed, 14 Jul 2004 16:45:23 CDT."
	<16629.43379.604258.804345@montanaro.dyndns.org> 
References: <16629.43379.604258.804345@montanaro.dyndns.org> 
Message-ID: <200407150149.i6F1nEh28985@guido.python.org>

> Have Guido and Dan been practicing their aim with a pie?  The topic of the
> Python/Parrot shootout hasn't come up in awhile.  I thought I'd ask since
> OSCON 2K4 is only a couple weeks away.

Last week I've answered a few emails from Dan's minion who is
responsible for the actual Python port, and Kevin Altis found us an
impartial judge.

So I better start practice my throwing, and Dan better start practice
his ducking. ;-)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Jul 15 03:50:38 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul 15 03:50:54 2004
Subject: [Python-Dev] Pie-thon at OSCON 2004?
In-Reply-To: Your message of "Wed, 14 Jul 2004 18:30:19 EDT."
	<20040714223019.GA17521@rogue.amk.ca> 
References: <16629.43379.604258.804345@montanaro.dyndns.org>  
	<20040714223019.GA17521@rogue.amk.ca> 
Message-ID: <200407150150.i6F1ocP28999@guido.python.org>

> I forget the terms of the bet; is it running on Python 2.3.4, or the
> 2.4alpha1 release, or on Python CVS?  Maybe we should check that
> 2.4CVS hasn't regressed in performance when compared to 2.3.

Whatever's in CVS for 2.4.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Jul 15 03:52:33 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul 15 03:52:41 2004
Subject: [Python-Dev] Pie-thon at OSCON 2004?
In-Reply-To: Your message of "Thu, 15 Jul 2004 12:17:11 +1200."
	<1ED4ECF91CDED24C8D012BCF2B034F1304678142@its-xchg4.massey.ac.nz> 
References: <1ED4ECF91CDED24C8D012BCF2B034F1304678142@its-xchg4.massey.ac.nz> 
Message-ID: <200407150152.i6F1qXZ29016@guido.python.org>

> [Skip]
> > I  saw somewhere today when  searching (maybe Dan S's  
> > website?) that it was 2.3 as of about December 2003.
> 
> That's correct:
> 
> <http://mail.python.org/pipermail/python-dev/2003-December/040977.html>
> <http://mail.python.org/pipermail/python-dev/2003-December/041527.html>
> 
> It's also in the README.txt file in the parrotbench directory.

It's Python 2.3 bytecode, but the 2.4 CVS implementation is allowed.
(Did we change the bytecode at all?)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From bac at OCF.Berkeley.EDU  Thu Jul 15 04:10:50 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Jul 15 04:10:59 2004
Subject: [Python-Dev] What is test__locale supposed to test, anyhow?
In-Reply-To: <40F4BF85.2030905@v.loewis.de>
References: <E4310D2D-D514-11D8-91CD-000D932927FE@opnet.com>	<40F45E18.3040209@v.loewis.de>	<35EBDEEC-D54F-11D8-91CD-000D932927FE@opnet.com>
	<40F4BF85.2030905@v.loewis.de>
Message-ID: <40F5E7AA.7000003@ocf.berkeley.edu>

Martin v. L?wis wrote:

> Nick Bastin wrote:
> 
>> Well, it seems we ought to have *some* kind of test that _locale is 
>> working properly.  The current test doesn't seem to fit the bill.
> 
> 
> It qualifies as *some* test, though. Remember that there is also
> test_locale.py.
> 

But not on OS X.  It raises TestSkipped saying how locale support is so 
minimal that it isn't worth testing.

-Brett
From python at rcn.com  Wed Jul 14 16:27:02 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Jul 15 04:28:15 2004
Subject: [Python-Dev] Pie-thon at OSCON 2004?
In-Reply-To: <200407150152.i6F1qXZ29016@guido.python.org>
Message-ID: <000001c469ae$a2fafc00$e841fea9@oemcomputer>

> It's Python 2.3 bytecode, but the 2.4 CVS implementation is allowed.
> (Did we change the bytecode at all?)

Added two new codes (LIST_APPEND and NOP).
No other changes, so 2.3 bytecode should just run fine.


Raymond

From jepler at unpythonic.net  Thu Jul 15 04:32:18 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Thu Jul 15 04:32:57 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
References: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
Message-ID: <20040715023218.GA32423@unpythonic.net>

On Wed, Jul 14, 2004 at 02:41:19AM -0400, James Y Knight wrote:
> - Thus, no code can directly access object fields ->ob_refcnt, or 
> ->ob_type. Introduced Py_GETTYPE/Py_GETREF macros and search&replaced 
> code in python to use them. These macros check the tag bits, and do the 
> right thing if it is tagged data.

If this looks like a promising direction, Py_GETTYPE and Py_GETREF
macros could be added today, in 2.5 or 2.6 ob_refcnt and ob_type would
be removed/renamed, and 2.6 or 2.7 would see the first actual use of
tagged types.  Sure, you couldn't compile a 2.4 extension on 2.6, but
will it matter?

By the way, I don't know what optimizers actually do, but given
    PyObject *spam(PyObject *o) {
        Py_GETTYPE(o); junk; Py_GETREF(o); junk;
        Py_INCREF(o); junk; Py_DECREF(o); junk;
    }
the compiler is free to notice that "o" can't change (its address is never
taken), so the "is it a tagged object" test can be done once, not 4 times.

I assume that tagged integers could be turned off at compile time, for
the platforms where they don't work for one reason or another.

I'm showing my ignorance here, but what other types are traditionally
tagged, and do any of those translate well to Python?

I suppose you could tag some singleton objects, like None, True, False,
and (), but those are already shared.

Tagging length-1 strings and unicode strings seems like it might have its
advantages, though the length-0 and length-1 strings, length-0 unicode
strings, and first 256 length-1 unicode strings are already shared too.

I can't see much use for a 21-bit-mantissa floating-point type, and
I can't see how you could use tagging for any other types in Python.

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040714/85ce004a/attachment.pgp
From tim.peters at gmail.com  Thu Jul 15 04:48:20 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Jul 15 04:48:29 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Include
	objimpl.h, 2.58, 2.59
In-Reply-To: <001401c4697b$e1ee4800$e841fea9@oemcomputer>
References: <001401c4697b$e1ee4800$e841fea9@oemcomputer>
Message-ID: <1f7befae0407141948403a6903@mail.gmail.com>

[Jim Fulton]
>> Documented the new Py_VISIT macro to simplify implementation of
>> tp_traverse handlers. (Tim made me do it. ;)

[Raymond Hettinger]
> That Tim has been tyrant lately ;-)

It's the only way left to get things done <wink>.

>> + /* Utility macro to help write tp_traverse functions */
>> + #define Py_VISIT(op)                                        \
>> +         do {                                                \
>> +                 if (op) {                           \
>> +                         int vret = visit((op), arg);        \
>> +                         if (vret)                   \
>> +                                 return vret;                \
>> +                 }                                   \
>> +         } while (0)

> I tried out this macro with the itertools module and found that it did a
> nice job of compacting boilerplate code.
>
> One thought jumped out though.  The macro assumes that the enclosing
> function has consistent with the use of the variable names "visit" and
> "arg":
> 
>  static int
>  islice_traverse(isliceobject *lz, visitproc visit, void *arg)
>  {
>      Py_VISIT(lz->it);
>      return 0;
>  }
> 
> If the parameters are named something else ("vfunc" for example),
> MSVC++6.0 complains:
> 
> C:\py24\Modules\itertoolsmodule.c(1118) : warning C4013: 'visit'
> undefined; assuming extern returning int
> 
> Don't know if it is worth it, but you could safeguard the macro by
> including visit and arg in the parameter list.

-1.  Instead it should be documented that the tp_traverse
implementation *must* name its arguments "visit" and "arg" if it wants
to use this macro.  Since tp_traverse implementations *can* look so
relentlessy the same, it's a Positive Good to encourage them to look
relentlessly the same.  I want the process of writing a correct
tp_traverse implementation to be mindlessly simple, and likewise I
want the process of reviewing one to be as simple as asking "did it
apply VISIT to each member and then return 0?".  Rigid uniformity
helps both goals -- and as there's no creative joy to be had in trying
to write a clever tp_traverse either, deviation from a rigid template
would just be gratuitous deviation.

principled-perversion-is-a-different-story-ly y'rs  - tim
From guido at python.org  Thu Jul 15 04:53:53 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul 15 04:54:01 2004
Subject: [Python-Dev] Pie-thon at OSCON 2004?
In-Reply-To: Your message of "Wed, 14 Jul 2004 10:27:02 EDT."
	<000001c469ae$a2fafc00$e841fea9@oemcomputer> 
References: <000001c469ae$a2fafc00$e841fea9@oemcomputer> 
Message-ID: <200407150253.i6F2rrk29176@guido.python.org>

> > It's Python 2.3 bytecode, but the 2.4 CVS implementation is allowed.
> > (Did we change the bytecode at all?)
> 
> Added two new codes (LIST_APPEND and NOP).
> No other changes, so 2.3 bytecode should just run fine.

Except that the magic number changed.  Can we modify import.c so that
it accepts the 2.3 magic number as well?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tim.peters at gmail.com  Thu Jul 15 05:03:51 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Jul 15 05:03:53 2004
Subject: [Python-Dev] Pie-thon at OSCON 2004?
In-Reply-To: <200407150152.i6F1qXZ29016@guido.python.org>
References: <1ED4ECF91CDED24C8D012BCF2B034F1304678142@its-xchg4.massey.ac.nz>
	<200407150152.i6F1qXZ29016@guido.python.org>
Message-ID: <1f7befae04071420036957b665@mail.gmail.com>

[Guido]
> It's Python 2.3 bytecode, but the 2.4 CVS implementation is allowed.
> (Did we change the bytecode at all?)

Not much, but Raymond is hard at work on a new PARROTBENCH opcode. 
The plan is to check that in the night before the contest.  It's a
good plan.  Please advise ASAP about the exact cache architecture of
the contest machine, so PARROTBENCH can be made as impartial as
possible.
From nbastin at opnet.com  Thu Jul 15 07:02:39 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Thu Jul 15 07:02:58 2004
Subject: [Python-Dev] What is test__locale supposed to test, anyhow?
In-Reply-To: <40F5E7AA.7000003@ocf.berkeley.edu>
References: <E4310D2D-D514-11D8-91CD-000D932927FE@opnet.com>	<40F45E18.3040209@v.loewis.de>	<35EBDEEC-D54F-11D8-91CD-000D932927FE@opnet.com>
	<40F4BF85.2030905@v.loewis.de> <40F5E7AA.7000003@ocf.berkeley.edu>
Message-ID: <32387EE6-D61C-11D8-9B88-000D932927FE@opnet.com>


On Jul 14, 2004, at 10:10 PM, Brett C. wrote:

> Martin v. L?wis wrote:
>
>> Nick Bastin wrote:
>>> Well, it seems we ought to have *some* kind of test that _locale is 
>>> working properly.  The current test doesn't seem to fit the bill.
>> It qualifies as *some* test, though. Remember that there is also
>> test_locale.py.
>
> But not on OS X.  It raises TestSkipped saying how locale support is 
> so minimal that it isn't worth testing.

Yeah, I'm going remove that and test it out...that message may be bogus 
at this point - it was put there a year and a half ago by Jack, and I'm 
reasonably sure that no one's tested it since (and I don't believe it 
anyhow - it's more like Python's support is bad - MacOS X's support for 
locale is pretty good).  We've definitely got something strange going 
on with locale, however, since I can't make it work at all in python, 
but it works fine outside of Python.

--
Nick

From martin at v.loewis.de  Thu Jul 15 07:22:28 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul 15 07:22:33 2004
Subject: [Python-Dev] What is test__locale supposed to test, anyhow?
In-Reply-To: <40F5E7AA.7000003@ocf.berkeley.edu>
References: <E4310D2D-D514-11D8-91CD-000D932927FE@opnet.com>	<40F45E18.3040209@v.loewis.de>	<35EBDEEC-D54F-11D8-91CD-000D932927FE@opnet.com>
	<40F4BF85.2030905@v.loewis.de> <40F5E7AA.7000003@ocf.berkeley.edu>
Message-ID: <40F61494.1030903@v.loewis.de>

Brett C. wrote:
> But not on OS X.  It raises TestSkipped saying how locale support is so 
> minimal that it isn't worth testing.

What happens if your remove the darwin check from the test?

Regards,
Martin

From nbastin at opnet.com  Thu Jul 15 07:28:44 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Thu Jul 15 07:28:59 2004
Subject: [Python-Dev] What is test__locale supposed to test, anyhow?
In-Reply-To: <40F61494.1030903@v.loewis.de>
References: <E4310D2D-D514-11D8-91CD-000D932927FE@opnet.com>	<40F45E18.3040209@v.loewis.de>	<35EBDEEC-D54F-11D8-91CD-000D932927FE@opnet.com>
	<40F4BF85.2030905@v.loewis.de> <40F5E7AA.7000003@ocf.berkeley.edu>
	<40F61494.1030903@v.loewis.de>
Message-ID: <D70D78F1-D61F-11D8-9B88-000D932927FE@opnet.com>


On Jul 15, 2004, at 1:22 AM, Martin v. L?wis wrote:

> Brett C. wrote:
>> But not on OS X.  It raises TestSkipped saying how locale support is 
>> so minimal that it isn't worth testing.
>
> What happens if your remove the darwin check from the test?

It fails.  I'm currently investigating.

--
Nick

From martin at v.loewis.de  Thu Jul 15 07:39:21 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul 15 07:39:23 2004
Subject: [Python-Dev] Pie-thon at OSCON 2004?
In-Reply-To: <200407150253.i6F2rrk29176@guido.python.org>
References: <000001c469ae$a2fafc00$e841fea9@oemcomputer>
	<200407150253.i6F2rrk29176@guido.python.org>
Message-ID: <40F61889.7050704@v.loewis.de>

Guido van Rossum wrote:
> Except that the magic number changed.  Can we modify import.c so that
> it accepts the 2.3 magic number as well?

I believe it should correctly load and execute the old files, if that is
what you are asking.

Apart from the two new bytecodes, marshal now also does the string
interning. I don't know of any further changes to the format.

Regards,
Martin

From adamsz at gmail.com  Thu Jul 15 07:52:08 2004
From: adamsz at gmail.com (Adam Souzis)
Date: Thu Jul 15 07:52:11 2004
Subject: [Python-Dev] Planned changes to logging
In-Reply-To: <000f01c45c99$5e644dc0$652b6992@alpha>
References: <000f01c45c99$5e644dc0$652b6992@alpha>
Message-ID: <d7cee4a704071422523dc9173c@mail.gmail.com>

(belatedly...) These are good changes, expecially the new basicConfig
constructor, but I'd like point out two issues with the logging
package that i think can easily be addressed:

One is a usability bug: calling fileConfig() disables any logger
objects already created that are not explicitly specified in the log
config file. Since loggers are often created when a module is
imported, it's hard to keep track of the order in which loggers are
created, leading to unexpected behavior because an unspecifed logger
created after the call to fileConfig behave differently -- it will
inherit its settings from its parent logger.
Here's an example illustrating the problem:

import logging, logging.config
import mymodule #this module contains a line like log =
logging.getLogger("mymodule")

if __name__ == '__main__':
  loggging.config.fileConfig("log.config") 
  #mymodule.log is now disabled! 
  #this seems like a bad design -- difficult to figure out what's going on
  #work-around: re-enable the loggers
  for logger in logging.Logger.manager.loggerDict.itervalues():
      logger.disabled = 0        

Second, i'd recommend a minor change that would greatly increase the
flexibility of the architecture: store the entire keyword dictionary
in the LogRecord, not just exc_info -- that way arbitrary objects
could be passed to particular handlers or formatters that know how to
use them.  Some use cases for this:
* pass a unique error number.
* pass structured information about the message, e.g. for writing to a
database table
* for a localized message formatter (LogRecord already stores the
argument list for string interpolation but this is cumbersome for
localization since they depends on the order they appear in the
message).

thanks,
adam

On Sun, 27 Jun 2004 23:52:03 +0100, Vinay Sajip
<vinay_sajip@red-dove.com> wrote:
> I'm planning to check in some changes to the logging package. I'd appreciate
> comments on the following:
> 
> 1. A backwards-compatible change to basicConfig(), making simple
> configuration of logging easier than before. For example, to change the
> level of the root logger to DEBUG:
> 
> logging.basicConfig(level=logging.DEBUG)
> 
> For example, to change the message format:
> 
> logging.basicConfig(format="%(asctime)s %(levelname)-5s %(message)s")
> 
> To log to a file in append mode (for write mode, add a filemode="w"
> argument):
> 
> logging.basicConfig(filename="/logs/myapp.log")
> 
> To log to an existing stream:
> 
> s = open("/logs/myapp.log", "w")
> ...
> logging.basicConfig(stream=s)
> 
> 2. Refactoring of RotatingFileHandler into BaseRotatingHandler and
> RotatingFileHandler. The reason for this is the addition of a new class,
> TimedRotatingFileHandler, which rotates files based on time-dependent
> criteria. (See SF patch #921318 for more details).
> 
> 3. Logger.log() changed to throw a TypeError if raiseExceptions is set and
> the level passed in is not an integer.
> 
> If the changes seem generally acceptable, then I'll also add a section in
> the docs which describes the above basic use cases under a "Basic use of the
> logging package" section which appears earlier in the logging docs than it
> does currently. Except for the documentation changes, I'm planning to commit
> by 3 July.
> 
> Thanks,
> 
> Vinay
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/adamsz%40gmail.com
>
From foom at fuhm.net  Thu Jul 15 08:37:45 2004
From: foom at fuhm.net (James Y Knight)
Date: Thu Jul 15 08:37:49 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <200407150142.i6F1gs128928@guido.python.org>
References: <Your message of "Wed,
	14 Jul 2004 11:16:29 +0200." <40F4F9ED.2020505@egenix.com>
	<D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
	<40F4F9ED.2020505@egenix.com>
	<5.1.1.6.0.20040714115351.01e9d330@mail.telecommunity.com>
	<200407150142.i6F1gs128928@guido.python.org>
Message-ID: <7B67AEB4-D629-11D8-B5AB-000A95A50FB2@fuhm.net>

On Jul 14, 2004, at 9:42 PM, Guido van Rossum wrote:
> Sorry, I'm still not convinced that it's worth to break all the 3rd
> party extensions that undoubtedly are doing all sorts of things that
> strictly speaking they shouldn't do.

There really is a minimal set of things you can do that will not 
already cause problems. The only thing you can do with an arbitrary 
PyObject * is access its ob_type or ob_refcnt. Anything else will break 
with objects today. So, those accesses all need to be cleaned up to use 
the new macros.

The other thing a 3rd party extension could do that would now break is 
to access a PyIntObject's ->ob_ival field directly.There's already the 
PyInt_AS_LONG macro they ought to be using for that.

If the idea to rename the ob_type/ob_refcnt fields is implemented, I 
really don't see any mysterious runtime failures occurring.

The idea of doing this in stages (as Jeff Epler says) is probably a 
good one. One question though (I am somewhat unknowledgeable in this 
area): are C extensions generally binary compatible between python 
major versions anyways? I had thought that they weren't.

> And what about all the extra code generated for Py_DECREF and
> Py_INCREF calls?  These now all contain an extra jump.  Horrors!

Indeed. That (and the similar branch in Py_GETTYPE) is why some 
operations are slower. The only question here is: do the speedups 
outweigh the slowdowns.

Investigating the pybench some more, it seems that speedup for many of 
the tests is because of the shortcut for tagged types in Py_INCREF and 
Py_DECREF. E.g. with the pybench TupleSlicing test, as written, the 
speed diff is -39.13%. However, changing the tuple to contain strings 
instead of integers causes the change to be +0.03%.

This makes me think that perhaps using an inline tagged representation 
for things like True, False, and single character strings < 256 might 
be a win as well, even though they don't ever cause memory allocation 
-- just because of the refcounting. Well, I tried making booleans a 
tagged types as well: pybench then runs 11% faster than standard python 
(vs the 7% with just tagged integers).

The all-important benchmark Parrotbench runs 4.8% faster than standard 
python.

James

P.S.: I have also previously experimented with removing refcounting 
from Python completely, and using the Boehm GC, because I theorized 
that refcounting could very well be slower than a full conservative GC. 
It mostly worked (I did not finish, so it did not support destructors 
or weakrefs), but was not a win, because it was no longer possible to 
use large block allocation and free list tricks for integers, and thus 
integer allocation caused a huge slowdown. I might revisit that now. I 
do realize using Boehm GC has pretty much a negative probability of 
making it into standard python, though. :)

From foom at fuhm.net  Thu Jul 15 09:11:14 2004
From: foom at fuhm.net (James Y Knight)
Date: Thu Jul 15 09:11:17 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <20040715023218.GA32423@unpythonic.net>
References: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
	<20040715023218.GA32423@unpythonic.net>
Message-ID: <28C1E9D9-D62E-11D8-B5AB-000A95A50FB2@fuhm.net>

On Jul 14, 2004, at 10:32 PM, Jeff Epler wrote:
> By the way, I don't know what optimizers actually do, but given
>     PyObject *spam(PyObject *o) {
>         Py_GETTYPE(o); junk; Py_GETREF(o); junk;
>         Py_INCREF(o); junk; Py_DECREF(o); junk;
>     }
> the compiler is free to notice that "o" can't change (its address is 
> never
> taken), so the "is it a tagged object" test can be done once, not 4 
> times.

I was unable to convince GCC 3.3.3 to be this smart. I don't think it 
even optimized "Py_GETTYPE(o) == X || Py_GETTYPE(o) == Y" to remove the 
redundant Py_GETTYPE. Perhaps someone versed in compiler-trickery could 
explain why it refuses to do common-subexpression-elimination there, or 
else trick it into doing so.

> I'm showing my ignorance here, but what other types are traditionally
> tagged, and do any of those translate well to Python?
>
> I suppose you could tag some singleton objects, like None, True, False,
> and (), but those are already shared.
>
> Tagging length-1 strings and unicode strings seems like it might have 
> its
> advantages, though the length-0 and length-1 strings, length-0 unicode
> strings, and first 256 length-1 unicode strings are already shared too.
>
> I can't see much use for a 21-bit-mantissa floating-point type, and
> I can't see how you could use tagging for any other types in Python.

In general, there's not much point in using tagged objects for 
singleton types like booleans. It's a waste of a tag value. However see 
my other post for counter-results in the case of Python.

You've basically got the standard tagged objects covered there: 
integers, and characters. E.g. see 
<http://www.xemacs.org/Documentation/21.5/html/internals_8.html> I had 
an idea that it might be possible to have a single-element tuple be a 
tagged object as well, but that may well be not worth the overhead.

On a 64-bit computers it could be possible to steal two bits from the 
exponent of (small) floats to do inline 62-bit floats. I wouldn't want 
to steal bits from the mantissa, you'd end up with different (non IEEE 
standard) results.

James

From mwh at python.net  Thu Jul 15 11:10:07 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Jul 15 11:10:08 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <200407150115.i6F1Fv4b010419@cosc353.cosc.canterbury.ac.nz> (Greg
	Ewing's message of "Thu, 15 Jul 2004 13:15:57 +1200")
References: <200407150115.i6F1Fv4b010419@cosc353.cosc.canterbury.ac.nz>
Message-ID: <2mwu15vdow.fsf@starship.python.net>

Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

> James Y Knight <foom@fuhm.net>:
>
>> On the test that probably puts my change in the most positive light 
>> possible: "x = 0;  while x < 50000000:  x = x + 1", it achieves about a 
>> 50% increase in speed. More normal integer-heavy things seem to be at 
>> most 20% faster.
>
> And you have to ask yourself -- how integer-heavy does
> typical Python code get?
>
> Most Python code I write deals with higher-level things
> most of the time -- strings, lists, dicts, class instances.

It's kinda hard to work seriously with lists and strings without
integers :-) This was, at the very least, a worthwhile experiment...

Cheers,
mwh

-- 
  Python enjoys making tradeoffs that drive *someone* crazy <wink>.
                                       -- Tim Peters, comp.lang.python
From mwh at python.net  Thu Jul 15 11:15:23 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Jul 15 11:16:01 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <7B67AEB4-D629-11D8-B5AB-000A95A50FB2@fuhm.net> (James Y.
	Knight's message of "Thu, 15 Jul 2004 02:37:45 -0400")
References: <Your message of "Wed, 14 Jul 2004 11:16:29 +0200."
	<40F4F9ED.2020505@egenix.com>
	<D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
	<40F4F9ED.2020505@egenix.com>
	<5.1.1.6.0.20040714115351.01e9d330@mail.telecommunity.com>
	<200407150142.i6F1gs128928@guido.python.org>
	<7B67AEB4-D629-11D8-B5AB-000A95A50FB2@fuhm.net>
Message-ID: <2msmbtvdg4.fsf@starship.python.net>

James Y Knight <foom@fuhm.net> writes:

> Investigating the pybench some more, it seems that speedup for many of
> the tests is because of the shortcut for tagged types in Py_INCREF and
> Py_DECREF. E.g. with the pybench TupleSlicing test, as written, the
> speed diff is -39.13%. However, changing the tuple to contain strings
> instead of integers causes the change to be +0.03%.
>
> This makes me think that perhaps using an inline tagged representation
> for things like True, False, and single character strings < 256 might
> be a win as well, even though they don't ever cause memory allocation
> -- just because of the refcounting. Well, I tried making booleans a
> tagged types as well: pybench then runs 11% faster than standard
> python (vs the 7% with just tagged integers).

Interesting!

> P.S.: I have also previously experimented with removing refcounting
> from Python completely, and using the Boehm GC, because I theorized
> that refcounting could very well be slower than a full conservative
> GC. It mostly worked (I did not finish, so it did not support
> destructors or weakrefs), but was not a win, because it was no longer
> possible to use large block allocation and free list tricks for
> integers, and thus integer allocation caused a huge slowdown. I might
> revisit that now. I do realize using Boehm GC has pretty much a
> negative probability of making it into standard python, though. :)

The last time I mentioned that the Boehm GC didn't help Python much
(on comp.lang.lisp) Hans Boehm showed up and claimed that it was
because we weren't using it properly :-) Let us know how it goes...

Cheers,
mwh

-- 
  A difference which makes no difference is no difference at all.
                        -- William James (I think.  Reference anyone?)
From mwh at python.net  Thu Jul 15 11:19:04 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Jul 15 11:19:06 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <Pine.LNX.4.44.0407141501470.21617-100000@ccc1.wpi.edu>
	(Christopher
	T. King's message of "Wed, 14 Jul 2004 15:13:37 -0400 (EDT)")
References: <Pine.LNX.4.44.0407141501470.21617-100000@ccc1.wpi.edu>
Message-ID: <2mllhlvd9z.fsf@starship.python.net>

Christopher T King <squirrel@WPI.EDU> writes:

> On Wed, 14 Jul 2004, Michael Hudson wrote:
>
>> I guess I could have said this in c.l.py, but: what's the point?  I
>> don't think I've ever had a situation where hitting the recursion
>> limit wasn't a bug, and given a) the ease with which a non-contrivedly
>> tail recursive algorithm can usually be rewritten iteratively and b)
>> the (lack of) speed of function calls in today's Python I can't see
>> any huge real advantage to this patch.
>
> To tell the truth, I don't really have any immediate use for this
> functionality, either, but since it turned out to be so easy to implement,
> I ask, "why not?" ;) Presuming it doesn't break any existing
> functionality, of course.* I prefer to think of it in the same light as
> the opcode prediction macros, i.e. as a performance enhancement, rather
> than as an attempt to introduce a new paradigm to the language (which it
> isn't).

*Is* it actually a performance enhancement?

> * I know it breaks stack traces, but the final implementation will only be 
> used when the -O flag is specified, or some other such indicator is set.

It doesn't seem to me that it necessarily has to break
tracebacks...  Haven't looked at the patch, though.  I guess there's a
fair chance that any performance enhancements it acheives are from
resuing frame objects (if that's what it does) though.

Cheers,
mwh

-- 
  I've even been known to get Marmite *near* my mouth -- but never
  actually in it yet.  Vegamite is right out.
 UnicodeError: ASCII unpalatable error: vegamite found, ham expected
                                       -- Tim Peters, comp.lang.python
From tismer at stackless.com  Thu Jul 15 12:04:18 2004
From: tismer at stackless.com (Christian Tismer)
Date: Thu Jul 15 12:03:33 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <40F4FF98.2000204@sweetapp.com>
References: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
	<40F4FF98.2000204@sweetapp.com>
Message-ID: <40F656A2.70803@stackless.com>

Brian Quinlan wrote:

...

> And the lack of portability is a real problem - it means that every code 
> path must be tested twice AND there might be architectures and problems 
> were tagged integers work but it actually makes things slower.

I think it was not the proposal to use tagging on
every platform, but just those where it works.
Having this as a compile time feature would make it
possible to optimize your Python for your purposes.

Things get interesting when it comes to compare
refcounting cost vs. pointer decoding cost
vs. garbage collection cost.

I have no idea what the results will be, and I think
it is just *great* to try this out, because the effect
could be dramatic:

If very many tiny, often-used objects are tagged, then
very many refcount actions can be saved, and the overall
object count shrinks, Boehm GC might become feasible.
If we then can write C extensions without having to think
of reference counts, code would be simplified quite
much, and I would save 80 percent of debugging time.

Maybe we have a set of rules which depend on each other:

1) "Tagging is -1000 because of the headaches"

2) "Everything is object and has refcount"

3) "Many tiny objects must be allocated as blocks"

4) "Boehm GC disables bulk allocation of tiny objects"

1) is just a claim which I would not sign, see the post about
systematically renaming fields to find all places with
the compiler.

1) leads to 2), 2) leads to 3), and this disables using 4).
By solving 1), it might be possible to break the whole chain,
and I would even accept a slitghly slower result, if
the whole system gets *simpler* by dropping refcounts.

So whatever is the outcome, I'm happy that somebody
took it on to try this out.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From mwh at python.net  Thu Jul 15 12:27:40 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Jul 15 12:27:42 2004
Subject: [Python-Dev] What is test__locale supposed to test, anyhow?
In-Reply-To: <32387EE6-D61C-11D8-9B88-000D932927FE@opnet.com> (Nick Bastin's
	message of "Thu, 15 Jul 2004 01:02:39 -0400")
References: <E4310D2D-D514-11D8-91CD-000D932927FE@opnet.com>
	<40F45E18.3040209@v.loewis.de>
	<35EBDEEC-D54F-11D8-91CD-000D932927FE@opnet.com>
	<40F4BF85.2030905@v.loewis.de> <40F5E7AA.7000003@ocf.berkeley.edu>
	<32387EE6-D61C-11D8-9B88-000D932927FE@opnet.com>
Message-ID: <2mhds9va3n.fsf@starship.python.net>

Nick Bastin <nbastin@opnet.com> writes:

> On Jul 14, 2004, at 10:10 PM, Brett C. wrote:
>
>> But not on OS X.  It raises TestSkipped saying how locale support is
>> so minimal that it isn't worth testing.
>
> Yeah, I'm going remove that and test it out...that message may be
> bogus at this point - it was put there a year and a half ago by Jack,
> and I'm reasonably sure that no one's tested it since (and I don't
> believe it anyhow - it's more like Python's support is bad - MacOS X's
> support for locale is pretty good).

Uh, sure about that?  It certainly wasn't the impression I had.

Cheers,
mwh

-- 
  I like silliness in a MP skit, but not in my APIs. :-)
                                       -- Guido van Rossum, python-dev
From Jack.Jansen at cwi.nl  Thu Jul 15 12:52:24 2004
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Thu Jul 15 12:52:15 2004
Subject: [Python-Dev] Pie-thon at OSCON 2004?
In-Reply-To: <200407150149.i6F1nEh28985@guido.python.org>
References: <16629.43379.604258.804345@montanaro.dyndns.org>
	<200407150149.i6F1nEh28985@guido.python.org>
Message-ID: <0DD8FFF3-D64D-11D8-8BD2-000D934FF6B4@cwi.nl>


On 15 Jul 2004, at 03:49, Guido van Rossum wrote:
> So I better start practice my throwing, and Dan better start practice
> his ducking. ;-)

Shouldn't you do practice sessions?

I would have loved to volunteer, but I won't be at OSCON. But maybe we 
could take a vote as to which python-dev member we would really like to 
see as your sparring partner:-)
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman

From Jack.Jansen at cwi.nl  Thu Jul 15 13:07:10 2004
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Thu Jul 15 13:07:01 2004
Subject: [Python-Dev] What is test__locale supposed to test, anyhow?
In-Reply-To: <32387EE6-D61C-11D8-9B88-000D932927FE@opnet.com>
References: <E4310D2D-D514-11D8-91CD-000D932927FE@opnet.com>	<40F45E18.3040209@v.loewis.de>	<35EBDEEC-D54F-11D8-91CD-000D932927FE@opnet.com>
	<40F4BF85.2030905@v.loewis.de> <40F5E7AA.7000003@ocf.berkeley.edu>
	<32387EE6-D61C-11D8-9B88-000D932927FE@opnet.com>
Message-ID: <1DFE251E-D64F-11D8-8BD2-000D934FF6B4@cwi.nl>


On 15 Jul 2004, at 07:02, Nick Bastin wrote:
>> But not on OS X.  It raises TestSkipped saying how locale support is 
>> so minimal that it isn't worth testing.
>
> Yeah, I'm going remove that and test it out...that message may be 
> bogus at this point - it was put there a year and a half ago by Jack, 
> and I'm reasonably sure that no one's tested it since (and I don't 
> believe it anyhow - it's more like Python's support is bad - MacOS X's 
> support for locale is pretty good).

MacOSX's support for locale was abysmal up to 10.2, a trait which it 
inherited from FreeBSD. It is much better under 10.3 (but still 
completely separate from the MacOSX native way to specify currency 
signs and what have you).

But it seems the C library doesn't use the locale info, here's the 
output from test_locale (from cvs head, on 10.3.4):

'%f' % 1024 =? '1,024.000000' ... no
'%f' % 1024 == '1024.000000' != '1,024.000000'
'%f' % 102 =? '102.000000' ... yes
'%f' % -42 =? '-42.000000' ... yes
'%+f' % -42 =? '-42.000000' ... yes
'%20.f' % -42 =? '                 -42' ... yes
'%+10.f' % -4200 =? '    -4,200' ... no
'%+10.f' % -4200 == '     -4200' != '    -4,200'
'%-10.f' % 4200 =? '4,200     ' ... no
'%-10.f' % 4200 == '4200      ' != '4,200     '

--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman

From tanzer at swing.co.at  Thu Jul 15 13:08:14 2004
From: tanzer at swing.co.at (tanzer@swing.co.at)
Date: Thu Jul 15 13:08:16 2004
Subject: [Python-Dev] RELEASED Python 2.4, alpha 1
In-Reply-To: Your message of "Fri, 09 Jul 2004 17:11:34 +1000."
	<40EE4526.5060306@python.org>
Message-ID: <E1Bl462-0007dg-00@swing.co.at>

In February, I reported a doctest bug against 2.3.3 (request id
902628) and attached a patch to fix the problem.

2.4 alpha 1 still shows this behavior:

Traceback (most recent call last):
  File "/tmp/confuse_doctest.py", line 14, in ?
    import doctest, confuse_doctest
  File "/tmp/confuse_doctest.py", line 15, in ?
    doctest.testmod(confuse_doctest)
  File "/usr/local/lib/python2.4/doctest.py", line 1155, in testmod
    f, t = tester.rundict(m.__dict__, name, m)
  File "/usr/local/lib/python2.4/doctest.py", line 915, in rundict
    f2, t2 = self.__runone(value, name + "." + thisname)
  File "/usr/local/lib/python2.4/doctest.py", line 1076, in __runone
    return self.rundoc(target, name)
  File "/usr/local/lib/python2.4/doctest.py", line 836, in rundoc
    f2, t2 = self.run__test__(d, name)
  File "/usr/local/lib/python2.4/doctest.py", line 944, in run__test__
    raise TypeError("Tester.run__test__: values in "
TypeError: Tester.run__test__: values in dict must be strings, functions, methods, or classes; <super: <class 'A'>, NULL>

What can I do to help getting this fixed?

-- 
Christian Tanzer                                    http://www.c-tanzer.at/

From python at rcn.com  Thu Jul 15 02:18:10 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Jul 15 14:19:23 2004
Subject: [Python-Dev] Pie-thon at OSCON 2004?
In-Reply-To: <200407150150.i6F1ocP28999@guido.python.org>
Message-ID: <000301c46a01$3653f540$e841fea9@oemcomputer>

>   Maybe we should check that
> > 2.4CVS hasn't regressed in performance when compared to 2.3.

Test results on my machine comparing Py2.4a1 vs Py2.3.4:

Parrotbench runs 10% faster on Py2.4a1
PyBench runs 11% faster on Py2.4a1
PyStone runs 6% faster on Py2.4a1
test_decimal.py runs 7% faster on Py2.4a1

My personal apps run 8 to 17% faster on Py2.4a1 without using new
features.

These tests gain 2-4% if profiling of C functions is disabled for an
apples to apples comparison to Py2.3 which doesn't profile C functions.


Raymond

From mcherm at mcherm.com  Thu Jul 15 14:24:54 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu Jul 15 14:24:50 2004
Subject: [Python-Dev] Proper tail recursion
Message-ID: <1089894294.40f677961fc20@login.werra.lunarpages.com>


Christopher King writes:
> To tell the truth, I don't really have any immediate use for this
> functionality, either, but since it turned out to be so
> easy to implement,
> I ask, "why not?" ;)

Martin replies:
> The traditional answer to that question is "because Jython cannot
> support it".

No... that answer applies to *language features*, but not *implementation
details*. The recursion limit (the value of it anyhow) is an
implementation detail.

Think of it this way: if Iron Python, Jython, or PyPy were to implement
a clever trick that made loops run 10x faster, that would be swell,
even if the same trick couldn't be done in CPython. If CPython can find
a way to run recursion far more efficiently, then that's swell too. Of
course, the actual results from running the program must not differ,
but that doesn't seem too difficult to achieve.

Guido writes:
> I'm not interested in adding this to the official Python release.
>
> One reason is that if an exception happens in such a tail-recursive
> call, the stack trace will be confusing.

This doesn't bother me as much as it apparently bothers you. But for
that matter, we hardly care about performance if we're going to be
generating a stack trace, so we could probably construct the stack
trace after-the-fact if needed.

Guido continues:


From mcherm at mcherm.com  Thu Jul 15 14:32:05 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu Jul 15 14:32:15 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <1089894294.40f677961fc20@login.werra.lunarpages.com>
References: <1089894294.40f677961fc20@login.werra.lunarpages.com>
Message-ID: <1089894725.40f67945bc06d@login.werra.lunarpages.com>

Drat!

I mis-clicked the "Send" button before finishing my previous email.

Here's the entire thing:

Christopher King writes:
> To tell the truth, I don't really have any immediate use for this
> functionality, either, but since it turned out to be so
> easy to implement,
> I ask, "why not?" ;)

Martin replies:
> The traditional answer to that question is "because Jython cannot
> support it".

No... that answer applies to *language features*, but not *implementation
details*. The recursion limit (the value of it anyhow) is an
implementation detail.

Think of it this way: if Iron Python, Jython, or PyPy were to implement
a clever trick that made loops run 10x faster, that would be swell,
even if the same trick couldn't be done in CPython. If CPython can find
a way to run recursion far more efficiently, then that's swell too. Of
course, the actual results from running the program must not differ,
but that doesn't seem too difficult to achieve.

Guido writes:
> I'm not interested in adding this to the official Python release.
>
> One reason is that if an exception happens in such a tail-recursive
> call, the stack trace will be confusing.

This doesn't bother me as much as it apparently bothers you. But for
that matter, we hardly care about performance if we're going to be
generating a stack trace, so we could probably construct the stack
trace after-the-fact if needed.

Guido continues:
> Another reason is that I don't think it's a good idea to try to
> encourage a Scheme-ish "solve everything with recursion" programming
> style in Python.

I wouldn't say that tail recursion *encouraged* a functional style,
rather I would say that refusing to accept a working version of
tail recursion actively *discourages* the use of that style.

Now, I understand that a functional style is not for everybody. And
I realize that you want to design Python so that people will continue
to use it to write programs that are readable (and understandable)
by everyone. But I believe strongly in allowing different programming
paradigms -- using procedural, OOP, or functional programming styles
(and others) for the particular problems that they express best.
And I think that *discouraging* a functional style by intentionally
breaking it is a bad idea.

Rejecting an implementation of tail recursion because it doesn't
work well or it slows down other cases is perfectly reasonable. But
rejecting it because it allows some people's programs to work better...
that just seems wrongheaded.

-- Michael Chermside



From jhylton at gmail.com  Thu Jul 15 14:35:57 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Thu Jul 15 14:35:59 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <200407150136.i6F1aG928897@guido.python.org>
References: <Pine.LNX.4.44.0407141423310.20797-100000@ccc1.wpi.edu>
	<200407150136.i6F1aG928897@guido.python.org>
Message-ID: <e8bf7a530407150535869cf2d@mail.gmail.com>

On Wed, 14 Jul 2004 18:36:16 -0700, Guido van Rossum <guido@python.org> wrote:
> I'm not interested in adding this to the official Python release.
> 
> One reason is that if an exception happens in such a tail-recursive
> call, the stack trace will be confusing.
> 
> Another reason is that I don't think it's a good idea to try to
> encourage a Scheme-ish "solve everything with recursion" programming
> style in Python.
> 
> But feel free to maintain this as an independent modification, a la
> Stackless -- I'm sure there are people who would like to try this
> out.

It would be pretty interesting to create a Python-fork project that
combined all sorts of different ideas for a Python implementation:
Stackless + tagged integers + tail recursion.  There's more value to
having an actual SF project with its own CVS repository than to having
a collection of patches on various web pages.  We'd have more people
trying the changes and have a better chance of understanding their
effects (performance, stability, readability, &c).  Maybe we'll find
that some of the ideas would be good for Python.

I like Python-fork as the name; it captures the experimental nature for me.

Jeremy
From squirrel at WPI.EDU  Thu Jul 15 14:56:34 2004
From: squirrel at WPI.EDU (Christopher T King)
Date: Thu Jul 15 15:00:13 2004
Subject: [Python-Dev] Why is Bytecode the way it is?
In-Reply-To: <200407150147.i6F1lTt28974@guido.python.org>
Message-ID: <Pine.LNX.4.44.0407150838080.17601-100000@ccc2.wpi.edu>

On Wed, 14 Jul 2004, Guido van Rossum wrote:

> At the cost of an extra pointer dereference and jump, and usually for
> naught (the uses of LOAD_CONST not in a return statement must
> certainly vastly outnumber those in return statements).

Hm, I didn't think the extra three or so assembler opcodes needed would 
make that big of an impact on the loop, given all the other things it 
does.  Guess it was a just bad idea :)


From ncoghlan at iinet.net.au  Thu Jul 15 15:31:05 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu Jul 15 15:31:05 2004
Subject: [Python-Dev] PEP 292 for Python 2.4
In-Reply-To: <20040714122227.GA12473@titan.progiciels-bpi.ca>
References: <1087414523.7150.45.camel@localhost>
	<001f01c4538d$f1fcfec0$bfaf2c81@oemcomputer>
	<20040624142741.GC27922@panix.com>
	<20040713130439.GA2174@titan.progiciels-bpi.ca>
	<40F501EB.4080700@iinet.net.au>
	<20040714122227.GA12473@titan.progiciels-bpi.ca>
Message-ID: <40F68719.1050206@iinet.net.au>

Fran?ois Pinard wrote:

> [Nick Coghlan]
>>I would suggest that bare type names are rarely appropriate for use a
>>variable names, except in toy examples.
> 
> Or small enough functions.  Small functions are not necessarily toys.

Hmm, I hadn't considered that case. I guess I tend not to write too many 
support functions where generic names would be appropriate (most of my 
Python code is very domain specific).

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From michael.walter at gmail.com  Thu Jul 15 15:34:53 2004
From: michael.walter at gmail.com (Michael Walter)
Date: Thu Jul 15 15:34:55 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <1089894725.40f67945bc06d@login.werra.lunarpages.com>
References: <1089894294.40f677961fc20@login.werra.lunarpages.com>
	<1089894725.40f67945bc06d@login.werra.lunarpages.com>
Message-ID: <877e9a17040715063463791eb6@mail.gmail.com>

Herro,

On Thu, 15 Jul 2004 05:32:05 -0700, Michael Chermside <mcherm@mcherm.com> wrote:
> Drat!
> 
> I mis-clicked the "Send" button before finishing my previous email.
> 
> Here's the entire thing:
> 
> Christopher King writes:
> > To tell the truth, I don't really have any immediate use for this
> > functionality, either, but since it turned out to be so
> > easy to implement,
> > I ask, "why not?" ;)
> 
> Martin replies:
> > The traditional answer to that question is "because Jython cannot
> > support it".
> 
> No... that answer applies to *language features*, but not *implementation
> details*. The recursion limit (the value of it anyhow) is an
> implementation detail.
> 
> Think of it this way: if Iron Python, Jython, or PyPy were to implement
> a clever trick that made loops run 10x faster, that would be swell,
> even if the same trick couldn't be done in CPython. If CPython can find
> a way to run recursion far more efficiently, then that's swell too. Of
> course, the actual results from running the program must not differ,
> but that doesn't seem too difficult to achieve.
Well, tail call elimination (or "proper tail recursion", being a
subset of it?) is not only an implementation detail as it opens up
Python to a functional/recursion-oriented programming style (as you
also argue further down).

So either you would keep your patch purely as an implementation detail
(i.e. "Python has proper tail recursion!" in the "news" would be a
no-no), or it has to be possible for all Python implementations to
implement "proper" tail recursion (at least in my understanding).

Anyway there was no agreement yet that Jython couldn't support that
_feature_, too, IIRC - I would certainly be happy to have it in
Python.

> Guido writes:
> > I'm not interested in adding this to the official Python release.
> >
> > One reason is that if an exception happens in such a tail-recursive
> > call, the stack trace will be confusing.
Hmm. Maybe we could have some kind of indicator that tail recursion
occured (or even a recursion count -- of course this will only work
for tail recursion, not for general tail call elimination)?

> Guido continues:
> > Another reason is that I don't think it's a good idea to try to
> > encourage a Scheme-ish "solve everything with recursion" programming
> > style in Python.
> 
> I wouldn't say that tail recursion *encouraged* a functional style,
> rather I would say that refusing to accept a working version of
> tail recursion actively *discourages* the use of that style.
How about: Tail recursion "enables" recursion-oriented (functional) style? :)

Cheers,
Michael
From ncoghlan at iinet.net.au  Thu Jul 15 15:41:45 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu Jul 15 15:41:45 2004
Subject: [Python-Dev] Developer looking to help fix bugs
In-Reply-To: <20040714135557.GA24990@panix.com>
References: <1089688837.2657.5.camel@localhost.localdomain>
	<20040714135557.GA24990@panix.com>
Message-ID: <40F68999.3040905@iinet.net.au>

Aahz wrote:

> On Mon, Jul 12, 2004, Mike Mangino wrote:
> 
>>Earlier today I added a comment to http://www.python.org/sf/982679 to
>>tell the user how to fix the problem they were having (it was a bug in
>>their code, not python) How do I have somebody close that request? In
>>the future, if I fix one of the issues, is there somebody I should email
>>to make changes?
> 
> 
> BTW, have you read http://www.python.org/dev/dev_intro.html
> 
> Was it helpful?  If not, what was missing?

More prominent placement of the link to the developer pages on the 
python.org home page would be my suggestion. I *still* go looking for a 
link to the developer pages in the sidebar, and then go "D'oh" and look 
at the menu at the top of the page instead.

For instance, I think making the Developer pages the first link in the 
'Community' section in the side bar would make them much easier to find. 
(The actual developer pages I quite like - it was finding them in the 
first place that I found more difficult than it should have been)

Regards,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From barry at python.org  Thu Jul 15 16:26:52 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Jul 15 16:26:52 2004
Subject: [Python-Dev] Pie-thon at OSCON 2004?
In-Reply-To: <200407150149.i6F1nEh28985@guido.python.org>
References: <16629.43379.604258.804345@montanaro.dyndns.org>
	<200407150149.i6F1nEh28985@guido.python.org>
Message-ID: <1089901612.8322.96.camel@localhost>

On Wed, 2004-07-14 at 21:49, Guido van Rossum wrote:

> So I better start practice my throwing, and Dan better start practice
> his ducking. ;-)

Don't forget to "tape" it!  We'll make room on www.python.org and
display the link prominently. :)

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040715/77edee92/attachment.pgp
From barry at python.org  Thu Jul 15 16:30:32 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Jul 15 16:30:31 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <e8bf7a530407150535869cf2d@mail.gmail.com>
References: <Pine.LNX.4.44.0407141423310.20797-100000@ccc1.wpi.edu>
	<200407150136.i6F1aG928897@guido.python.org>
	<e8bf7a530407150535869cf2d@mail.gmail.com>
Message-ID: <1089901832.8314.98.camel@localhost>

On Thu, 2004-07-15 at 08:35, Jeremy Hylton wrote:

> I like Python-fork as the name; it captures the experimental nature for me.

I was thinking Py3K would be a good name <wink>.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040715/8d800edd/attachment.pgp
From guido at python.org  Thu Jul 15 16:34:46 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul 15 16:34:55 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: Your message of "Thu, 15 Jul 2004 15:34:53 +0200."
	<877e9a17040715063463791eb6@mail.gmail.com> 
References: <1089894294.40f677961fc20@login.werra.lunarpages.com>
	<1089894725.40f67945bc06d@login.werra.lunarpages.com> 
	<877e9a17040715063463791eb6@mail.gmail.com> 
Message-ID: <200407151434.i6FEYkt30174@guido.python.org>

> > Guido continues:
> > > Another reason is that I don't think it's a good idea to try to
> > > encourage a Scheme-ish "solve everything with recursion" programming
> > > style in Python.
> > 
> > I wouldn't say that tail recursion *encouraged* a functional style,
> > rather I would say that refusing to accept a working version of
> > tail recursion actively *discourages* the use of that style.

> How about: Tail recursion "enables" recursion-oriented (functional)
> style? :)

Still -1.  I was understating my case: I find the "recursion is the
basis of everything" attitude harmful.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From nbastin at opnet.com  Thu Jul 15 16:42:46 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Thu Jul 15 16:43:17 2004
Subject: [Python-Dev] What is test__locale supposed to test, anyhow?
In-Reply-To: <1DFE251E-D64F-11D8-8BD2-000D934FF6B4@cwi.nl>
References: <E4310D2D-D514-11D8-91CD-000D932927FE@opnet.com>	<40F45E18.3040209@v.loewis.de>	<35EBDEEC-D54F-11D8-91CD-000D932927FE@opnet.com>
	<40F4BF85.2030905@v.loewis.de> <40F5E7AA.7000003@ocf.berkeley.edu>
	<32387EE6-D61C-11D8-9B88-000D932927FE@opnet.com>
	<1DFE251E-D64F-11D8-8BD2-000D934FF6B4@cwi.nl>
Message-ID: <3CD55ABB-D66D-11D8-9B88-000D932927FE@opnet.com>


On Jul 15, 2004, at 7:07 AM, Jack Jansen wrote:

>
> On 15 Jul 2004, at 07:02, Nick Bastin wrote:
>>> But not on OS X.  It raises TestSkipped saying how locale support is 
>>> so minimal that it isn't worth testing.
>>
>> Yeah, I'm going remove that and test it out...that message may be 
>> bogus at this point - it was put there a year and a half ago by Jack, 
>> and I'm reasonably sure that no one's tested it since (and I don't 
>> believe it anyhow - it's more like Python's support is bad - MacOS 
>> X's support for locale is pretty good).
>
> MacOSX's support for locale was abysmal up to 10.2, a trait which it 
> inherited from FreeBSD. It is much better under 10.3 (but still 
> completely separate from the MacOSX native way to specify currency 
> signs and what have you).
>
> But it seems the C library doesn't use the locale info, here's the 
> output from test_locale (from cvs head, on 10.3.4):

No, this is something broken in Python - I just haven't figured out 
what.  If you were to write a small test program in C, you would see 
that the locale information you get from the C library is correct.  
However, something in Python is screwing this up, because calling 
setlocale() in python has almost *no* effect whatsoever on the data 
returned by localeconv().  (See previous messages in the thread for 
RADIXCHAR issues, specifically)

--
Nick

From ark-mlist at att.net  Thu Jul 15 16:53:37 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Thu Jul 15 16:53:34 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <200407151434.i6FEYkt30174@guido.python.org>
Message-ID: <01c701c46a7b$8291ee60$6602a8c0@arkdesktop>

> > How about: Tail recursion "enables" recursion-oriented (functional)
> > style? :)
> 
> Still -1.  I was understating my case: I find the "recursion is the
> basis of everything" attitude harmful.

I don't believe that recursion is the basis of everything.  Nevertheless, I
do think that there are some algorithms that are much more easily expressed
in recursive than in iterative terms, and I wish that Python did not impose
a fixed limit on recursion depth.  For example:

	def traverse(t, f):
		if t:
			f(t)
			traverse(t.left)
			traverse(t.right)

This code has the unfortunate limitation of being unable to traverse a tree
deeper than the recursion limit.  I can use tail-recursion elimination to
remove this restriction for one branch of each node:

	def traverse(t, f):
		while t:
			f(t)
			traverse(t.left)
			t = t.right

which makes the code much more useful if the "trees" are really lists
according to Lisp conventions (i.e. the left branch of each node is usually
short), but I would really rather that the compiler did this transformation
for me, as I find the first version much easier to understand than the
second.

If I really want to work around the recursion limit altogether, I have to
define a stack of my own:

	def traverse(t, f):
		s = [t]
		while s:
			t = s.pop()
			if t:
				f(t)
				s.append(t.right)
				s.append(t.left)

Now I am free of the recursion-depth limitation, but the code has become
complicated enough to reduce my confidence in its correctness.

So I agree that recursion is not the basis of everything--but it is the
basis of some things, and I would like to be able to express those things
recursively without having to worry about the implementation stopping me.  I
understand that the existence of Jython may make this wish impossible to
achieve in practice, but it's still there.

From squirrel at WPI.EDU  Thu Jul 15 16:55:23 2004
From: squirrel at WPI.EDU (Christopher T King)
Date: Thu Jul 15 17:00:17 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <200407151434.i6FEYkt30174@guido.python.org>
Message-ID: <Pine.LNX.4.44.0407151039150.20090-100000@ccc2.wpi.edu>

On Thu, 15 Jul 2004, Guido van Rossum wrote:

> > How about: Tail recursion "enables" recursion-oriented (functional)
> > style? :)
> 
> Still -1.  I was understating my case: I find the "recursion is the
> basis of everything" attitude harmful.

I think it's still best to "enable" such a style in some cases -- Python's 
syntax is designed to allow formulation of a problem in the most natural 
way.  Sometimes it's more natural to formulate a problem as a recursive 
function than a procedural one, but the current implementation restricts 
this.

I agree that "recursion is the basis of everything" is harmful, but I
think the same belief can apply to "procedularism is the basis of
everything" (why do we have classes, list comprehensions, etc.?).  Some 
things are better formulated in a procedural style, while others are 
better formulated in a recursive style.


From squirrel at WPI.EDU  Thu Jul 15 17:10:04 2004
From: squirrel at WPI.EDU (Christopher T King)
Date: Thu Jul 15 17:11:39 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <01c701c46a7b$8291ee60$6602a8c0@arkdesktop>
Message-ID: <Pine.LNX.4.44.0407151057060.20090-100000@ccc2.wpi.edu>

On Thu, 15 Jul 2004, Andrew Koenig wrote:

> [what I was trying to say, only better] :)

Just a note: because Python sticks an implicit 'return None' at the end of 
a function, rather than returning the result of the last expression, like 
Scheme, you have to have an explicit return to see any effect:

 	def traverse(t, f):
 		if t:
 			f(t)
 			traverse(t.left)
 			return traverse(t.right)


From bob at redivi.com  Thu Jul 15 17:27:07 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Jul 15 17:27:11 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <Pine.LNX.4.44.0407151057060.20090-100000@ccc2.wpi.edu>
References: <Pine.LNX.4.44.0407151057060.20090-100000@ccc2.wpi.edu>
Message-ID: <6F394265-D673-11D8-819B-000A95686CD8@redivi.com>

On Jul 15, 2004, at 11:10 AM, Christopher T King wrote:

> On Thu, 15 Jul 2004, Andrew Koenig wrote:
>
>> [what I was trying to say, only better] :)
>
> Just a note: because Python sticks an implicit 'return None' at the 
> end of
> a function, rather than returning the result of the last expression, 
> like
> Scheme, you have to have an explicit return to see any effect:
>
>  	def traverse(t, f):
>  		if t:
>  			f(t)
>  			traverse(t.left)
>  			return traverse(t.right)

But in this case what is tail-call optimization going to do for you?  
You still require a stack at least the size of the height of your tree 
because of traverse(t.left) since that can not be tail-call optimized 
away, with the proposed algorithm.

I think Guido is in the right here, if you want to work around the 
recursion limit, use Stackless..  It should already be able to go just 
about as deep as you want.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3589 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040715/6cddbb9a/smime.bin
From squirrel at WPI.EDU  Thu Jul 15 17:40:24 2004
From: squirrel at WPI.EDU (Christopher T King)
Date: Thu Jul 15 17:42:12 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <6F394265-D673-11D8-819B-000A95686CD8@redivi.com>
Message-ID: <Pine.LNX.4.44.0407151128300.20090-100000@ccc2.wpi.edu>

On Thu, 15 Jul 2004, Bob Ippolito wrote:

> On Jul 15, 2004, at 11:10 AM, Christopher T King wrote:
> 
> But in this case what is tail-call optimization going to do for you?  
> You still require a stack at least the size of the height of your tree 
> because of traverse(t.left) since that can not be tail-call optimized 
> away, with the proposed algorithm.

In Andrew's example, he noted that it would only help for list-like 
structures (i.e. those with mostly right nodes).

> I think Guido is in the right here, if you want to work around the 
> recursion limit, use Stackless..  It should already be able to go just 
> about as deep as you want.

You're right -- even if Stackless doesn't do tail call optimization,
implementation should be trivial.  But there's no guarantee when or even
if Stackless will be merged with CPython, and until that happens,
Stackless isn't an option for many (most?) people.


From bob at redivi.com  Thu Jul 15 18:02:41 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Jul 15 18:02:47 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <Pine.LNX.4.44.0407151128300.20090-100000@ccc2.wpi.edu>
References: <Pine.LNX.4.44.0407151128300.20090-100000@ccc2.wpi.edu>
Message-ID: <670FF77D-D678-11D8-819B-000A95686CD8@redivi.com>


On Jul 15, 2004, at 11:40 AM, Christopher T King wrote:

> On Thu, 15 Jul 2004, Bob Ippolito wrote:
>
>> On Jul 15, 2004, at 11:10 AM, Christopher T King wrote:
>>
>> But in this case what is tail-call optimization going to do for you?
>> You still require a stack at least the size of the height of your tree
>> because of traverse(t.left) since that can not be tail-call optimized
>> away, with the proposed algorithm.
>
> In Andrew's example, he noted that it would only help for list-like
> structures (i.e. those with mostly right nodes).

I think it's a misleading example nonetheless..

>> I think Guido is in the right here, if you want to work around the
>> recursion limit, use Stackless..  It should already be able to go just
>> about as deep as you want.
>
> You're right -- even if Stackless doesn't do tail call optimization,
> implementation should be trivial.  But there's no guarantee when or 
> even
> if Stackless will be merged with CPython, and until that happens,
> Stackless isn't an option for many (most?) people.

Why not?  Surely anyone who knows they want to do this kind of 
programming is capable of ./configure && make && sudo make install :)

Nearly all extension modules used with CPython should even be *binary* 
compatible with Stackless.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3589 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040715/a2cc4a50/smime.bin
From ark-mlist at att.net  Thu Jul 15 18:04:43 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Thu Jul 15 18:04:40 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <Pine.LNX.4.44.0407151057060.20090-100000@ccc2.wpi.edu>
Message-ID: <002c01c46a85$71529a50$6602a8c0@arkdesktop>

> Just a note: because Python sticks an implicit 'return None' at the end of
> a function, rather than returning the result of the last expression, like
> Scheme, you have to have an explicit return to see any effect:
> 
>  	def traverse(t, f):
>  		if t:
>  			f(t)
>  			traverse(t.left)
>  			return traverse(t.right)

Right.  I actually worked that out for myself not too long ago, and then
forgot it.

From ark-mlist at att.net  Thu Jul 15 18:07:53 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Thu Jul 15 18:07:49 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <6F394265-D673-11D8-819B-000A95686CD8@redivi.com>
Message-ID: <003c01c46a85$e2d2b1b0$6602a8c0@arkdesktop>

> But in this case what is tail-call optimization going to do for you?
> You still require a stack at least the size of the height of your tree
> because of traverse(t.left) since that can not be tail-call optimized
> away, with the proposed algorithm.

I thought I had pointed that out.  Lisp lists are really trees, the nodes of
which usually have very short left subtrees.  For example, (3 (4 5) 6 7 8)
is a tree in which the maximum left depth is 2 and maximum right depth is 5.
For such data structures, tail-call optimization would make a big difference
in practice even if it wouldn't matter in theory.

From mcherm at mcherm.com  Thu Jul 15 18:17:55 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu Jul 15 18:17:56 2004
Subject: [Python-Dev] Proper tail recursion
Message-ID: <1089908275.40f6ae33b98fd@login.werra.lunarpages.com>

Guido writes:
> Still -1.  I was understating my case: I find the "recursion is the
> basis of everything" attitude harmful.

Just for the record, I understand that is your objection, I simply
wanted to publicly disagree with it. I have a great deal of respect
for your judgement, but occasionally I think you're wrong <wink>.

Michael Walter writes:
> Anyway there was no agreement yet that Jython couldn't support that
> _feature_, too, IIRC - I would certainly be happy to have it in
> Python.

Well, I think I understand Jython well enough to provide the
assurance: Jython is (intentionally, and for good reason) designed
to utilize the JVM's call stack. Thus Jython cannot support tail
call elimination *unless* it is run on a JVM which does tail call
elimination. I don't believe any such JVM exists, but as far as I
know, the JVM specification *allows* it.

-- Michael Chermside

From squirrel at WPI.EDU  Thu Jul 15 18:26:19 2004
From: squirrel at WPI.EDU (Christopher T King)
Date: Thu Jul 15 18:28:13 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <1089908275.40f6ae33b98fd@login.werra.lunarpages.com>
Message-ID: <Pine.LNX.4.44.0407151221380.21569-100000@ccc2.wpi.edu>

On Thu, 15 Jul 2004, Michael Chermside wrote:

> > Anyway there was no agreement yet that Jython couldn't support that
> > _feature_, too, IIRC - I would certainly be happy to have it in
> > Python.
> 
> Well, I think I understand Jython well enough to provide the
> assurance: Jython is (intentionally, and for good reason) designed
> to utilize the JVM's call stack. Thus Jython cannot support tail
> call elimination *unless* it is run on a JVM which does tail call
> elimination.

I originally thought I'd need to use gcc's tail call optimization to
achieve my aim (since CPython works the same way), but it turned out a 
simple loop would suffice.  Unless Jython's implementation is radically 
different (i.e. it doesn't pass frame objects to a function that evaluates 
them) then it should be able to be implemented similarly.


From squirrel at WPI.EDU  Thu Jul 15 18:29:38 2004
From: squirrel at WPI.EDU (Christopher T King)
Date: Thu Jul 15 18:30:12 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <670FF77D-D678-11D8-819B-000A95686CD8@redivi.com>
Message-ID: <Pine.LNX.4.44.0407151214001.21569-100000@ccc2.wpi.edu>

On Thu, 15 Jul 2004, Bob Ippolito wrote:

> Why not?  Surely anyone who knows they want to do this kind of 
> programming is capable of ./configure && make && sudo make install :)

Many users might not be, though.  (Sorry, I don't have a better argument 
than that :P)

> Nearly all extension modules used with CPython should even be *binary* 
> compatible with Stackless.

I didn't know this -- that's quite interesting information.


From bob at redivi.com  Thu Jul 15 19:07:42 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Jul 15 19:07:46 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <Pine.LNX.4.44.0407151214001.21569-100000@ccc2.wpi.edu>
References: <Pine.LNX.4.44.0407151214001.21569-100000@ccc2.wpi.edu>
Message-ID: <7BD9F82A-D681-11D8-9739-000A95686CD8@redivi.com>


On Jul 15, 2004, at 12:29 PM, Christopher T King wrote:

> On Thu, 15 Jul 2004, Bob Ippolito wrote:
>
>> Why not?  Surely anyone who knows they want to do this kind of
>> programming is capable of ./configure && make && sudo make install :)
>
> Many users might not be, though.  (Sorry, I don't have a better 
> argument
> than that :P)

They they probably use Windows and can use Christian's binaries ;)

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3589 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040715/01f1181b/smime.bin
From mcherm at mcherm.com  Thu Jul 15 19:24:53 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu Jul 15 19:24:53 2004
Subject: [Python-Dev] Proper tail recursion
Message-ID: <1089912292.40f6bde500029@mcherm.com>

> > Well, I think I understand Jython well enough to provide the
> > assurance: Jython is (intentionally, and for good reason) designed
> > to utilize the JVM's call stack. Thus Jython cannot support tail
> > call elimination *unless* it is run on a JVM which does tail call
> > elimination.
>
> I originally thought I'd need to use gcc's tail call optimization to
> achieve my aim (since CPython works the same way), but it
> turned out a
> simple loop would suffice.  Unless Jython's implementation is
> radically
> different (i.e. it doesn't pass frame objects to a function
> that evaluates
> them) then it should be able to be implemented similarly.

Like I said, Jython uses the JVM's call stack. So it certainly doesn't
pass frame objects to a function that evaluates them... Jython
stack frames are actual Java stack frames. This is part of why Jython
does such an incredibly good job of integrating tightly with Java --
Java any Jython functions (methods) can call each other freely because
they're practically the same thing.

-- Michael Chermside

From martin at v.loewis.de  Thu Jul 15 21:08:24 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul 15 21:08:28 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <1089894294.40f677961fc20@login.werra.lunarpages.com>
References: <1089894294.40f677961fc20@login.werra.lunarpages.com>
Message-ID: <40F6D628.7010002@v.loewis.de>

Michael Chermside wrote:
> No... that answer applies to *language features*, but not *implementation
> details*. The recursion limit (the value of it anyhow) is an
> implementation detail.

No, it is not. In standard Python, the program

def rec(n):
   rec(n-1)

n=1
while 1:
   try:
     rec(n)
   except RuntimeError:
     break
print "The recursion limit is", n

will terminate. In the modified Python, it will not terminate.
It is a change in behaviour, and thus a language feature.

> This doesn't bother me as much as it apparently bothers you. But for
> that matter, we hardly care about performance if we're going to be
> generating a stack trace, so we could probably construct the stack
> trace after-the-fact if needed.

No, you can't: You forgot already what all the local variables where.

Regards,
Martin

From martin at v.loewis.de  Thu Jul 15 21:38:55 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul 15 21:38:59 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <Pine.LNX.4.44.0407151221380.21569-100000@ccc2.wpi.edu>
References: <Pine.LNX.4.44.0407151221380.21569-100000@ccc2.wpi.edu>
Message-ID: <40F6DD4F.6080707@v.loewis.de>

Christopher T King wrote:
> I originally thought I'd need to use gcc's tail call optimization to
> achieve my aim (since CPython works the same way), but it turned out a 
> simple loop would suffice.  Unless Jython's implementation is radically 
> different (i.e. it doesn't pass frame objects to a function that evaluates 
> them) then it should be able to be implemented similarly.

Yes, Jython is radically different. Each function gets its own Java byte 
code, and calls are implemented as Java "invoke*" instructions. So if 
you call a different function at the tail, you can't replace that with a
jump instruction.

Regards,
Martin

From dave at boost-consulting.com  Thu Jul 15 21:21:14 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Jul 15 21:40:21 2004
Subject: [Python-Dev] Re: Proper tail recursion
References: <200407151434.i6FEYkt30174@guido.python.org>
	<01c701c46a7b$8291ee60$6602a8c0@arkdesktop>
Message-ID: <u3c3t13h1.fsf@boost-consulting.com>

"Andrew Koenig" <ark-mlist@att.net> writes:

>> > How about: Tail recursion "enables" recursion-oriented (functional)
>> > style? :)
>> 
>> Still -1.  I was understating my case: I find the "recursion is the
>> basis of everything" attitude harmful.
>
> I don't believe that recursion is the basis of everything.  Nevertheless, I
> do think that there are some algorithms that are much more easily expressed
> in recursive than in iterative terms, and I wish that Python did not impose
> a fixed limit on recursion depth.

I just want to add a quiet +1 to Michael and Andrew's eloquent posts.

+1

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From martin at v.loewis.de  Thu Jul 15 21:45:00 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul 15 21:45:04 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <01c701c46a7b$8291ee60$6602a8c0@arkdesktop>
References: <01c701c46a7b$8291ee60$6602a8c0@arkdesktop>
Message-ID: <40F6DEBC.7020701@v.loewis.de>

Andrew Koenig wrote:
> So I agree that recursion is not the basis of everything--but it is the
> basis of some things, and I would like to be able to express those things
> recursively without having to worry about the implementation stopping me.  I
> understand that the existence of Jython may make this wish impossible to
> achieve in practice, but it's still there.

Interestingly enough, the .NET VM (i.e. MSIL) supports tail recursion 
explicitly by means of opcodes (e.g. tail.callvirt in addition
to callvirt).

Regards,
Martin

From jhylton at gmail.com  Thu Jul 15 22:04:00 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Thu Jul 15 22:04:02 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <40F6D628.7010002@v.loewis.de>
References: <1089894294.40f677961fc20@login.werra.lunarpages.com>
	<40F6D628.7010002@v.loewis.de>
Message-ID: <e8bf7a5304071513046311217@mail.gmail.com>

On Thu, 15 Jul 2004 21:08:24 +0200, "Martin v. L?wis"
<martin@v.loewis.de> wrote:
> Michael Chermside wrote:
> > No... that answer applies to *language features*, but not *implementation
> > details*. The recursion limit (the value of it anyhow) is an
> > implementation detail.
> 
> No, it is not. In standard Python, the program
> 
> def rec(n):
>    rec(n-1)
> 
> n=1
> while 1:
>    try:
>      rec(n)
>    except RuntimeError:
>      break
> print "The recursion limit is", n
> 
> will terminate. In the modified Python, it will not terminate.
> It is a change in behaviour, and thus a language feature.

I would describe this "feature" as an implementation-dependent one. 
An implementation is free to raise RuntimeError if the recursion limit
is exceeded.  The fact that setrecursionlimit is in sys suggests that
it isn't part of the language proper.

I could imagine a Stackless implementation that merely raised
MemoryError when it ran out of heap space for activation records.

> > This doesn't bother me as much as it apparently bothers you. But for
> > that matter, we hardly care about performance if we're going to be
> > generating a stack trace, so we could probably construct the stack
> > trace after-the-fact if needed.
> 
> No, you can't: You forgot already what all the local variables where.

Getting a useful traceback after tail call optimization seems like an
awfully important quality of implementation issue.

Jeremy
From art at wiktorsadowski.com  Thu Jul 15 22:09:04 2004
From: art at wiktorsadowski.com (Wiktor Sadowski)
Date: Thu Jul 15 22:09:00 2004
Subject: [Python-Dev] Absolute/Relative import switcher for packages (PEP
	328)
Message-ID: <04ed01c46aa7$980a6fc0$0201010a@kret>

FWIW:
Here is the solution for  Absolute/Relative problem that we are using
in our modified Python distribution...

We have added a few lines of code to to get_parent in import.c.
The code checks if there is "__absimport__=True" in parent dict
and if so: absolute import is used in "child" module (get_parent returns
Py_None)

Then putting "__absimport__=True" into an __init__.py is enough to
force absolute import for the whole package.
(The "__absimport__=True" switcher should be put into each subpackage
__init__.py,if desired)

Such addition is fully compatible with existing
Python import syntax and easy to use.

Also:
absolute import could be default,and "__relimport__=True"
would switch to relative import.

If nothing else,the switcher really helps with tracking absolute/relative
import
bugs in Python packages.

Wiktor Sadowski

From raymond.hettinger at verizon.net  Thu Jul 15 10:35:34 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Thu Jul 15 22:36:50 2004
Subject: [Python-Dev] Assignment to None
Message-ID: <000201c46a46$b327c700$582acb97@oemcomputer>

Python 2.4a1+ (#46, Jul 14 2004, 04:25:40) [MSC v.1200 32 bit (Intel)]
on win32
Type "copyright", "credits" or "license()" for more information.   
IDLE 1.1a2      
>>> None = 0
SyntaxError: assignment to None (<pyshell#0>, line 1)
>>> def f():
            None = 0
            
SyntaxError: assignment to None (<pyshell#2>, line 2)
 
 
Do this imply that the compiler can now treat None as a constant?
 
 
Raymond Hettinger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040715/7ffcad0d/attachment.htm
From mcherm at mcherm.com  Thu Jul 15 23:38:20 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu Jul 15 23:38:21 2004
Subject: [Python-Dev] Proper tail recursion
Message-ID: <1089927500.40f6f94c8b32f@mcherm.com>

I wrote:
> The recursion limit (the value of it anyhow) is an implementation detail.

Martin replies:
> No, it is not. In standard Python, the program
>
> def rec(n):
>    rec(n-1)
>
> n=1
> while 1:
>    try:
>      rec(n)
>    except RuntimeError:
>      break
> print "The recursion limit is", n
>
> will terminate. In the modified Python, it will not terminate.
> It is a change in behaviour, and thus a language feature.

First, I'm going to make a slight modification to your program, since I
want a version that actually does display the recursion limit. I tried
the following program instead:

    >>> def rec(n):
    ...     print n
    ...     rec(n+1)
    ...
    >>> rec(1)

Running this in Jython 2.1 (a well-known implementation of the Python
language) produced the numbers from 1 to 888, then immediately exited.

Running it on CPython 2.3 produced the numbers from 1 to 999, then a
traceback that looked like this:
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
      File "<stdin>", line 3, in rec
      File "<stdin>", line 3, in rec
      File "<stdin>", line 3, in rec
      File "<stdin>", line 3, in rec
      File "<stdin>", line 3, in rec
      File "<stdin>", line 3, in rec
  [... etc for a total of about 1000 lines ...]
      File "<stdin>", line 3, in rec
      File "<stdin>", line 3, in rec
      File "<stdin>", line 3, in rec
      File "<stdin>", line 3, in rec
    RuntimeError: maximum recursion depth exceeded

It then returned me to the interactive prompt.

I presume that since the behavior was different, CPython must not
actually be a correct implementation of the Python language, right?

Not-all-differences-are-language-differences lly yours,

Michael Chermside

From tim.peters at gmail.com  Fri Jul 16 00:01:25 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Jul 16 00:01:28 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <e8bf7a5304071513046311217@mail.gmail.com>
References: <1089894294.40f677961fc20@login.werra.lunarpages.com>
	<40F6D628.7010002@v.loewis.de>
	<e8bf7a5304071513046311217@mail.gmail.com>
Message-ID: <1f7befae040715150164b1481b@mail.gmail.com>

[Jeremy Hylton]
> ...
> Getting a useful traceback after tail call optimization seems like an
> awfully important quality of implementation issue.

Indeed, in delegation-heavy designs, when an error occurs the
delegation path you took to reach the failing code often *is* "the
error".

So let's optimize tail calls, but for each elided one we'll allocate a
record containing a pointer to its caller, the line number of the
optimized tail call, and the bindings of locals.  It will look pretty
much exactly like a frame object looks today, but we won't *call* it a
frame object, and then everyone will be happy <wink>.
From ark-mlist at att.net  Fri Jul 16 00:14:15 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Fri Jul 16 00:14:17 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <1f7befae040715150164b1481b@mail.gmail.com>
Message-ID: <00ee01c46ab9$14379ee0$6602a8c0@arkdesktop>

> So let's optimize tail calls, but for each elided one we'll allocate a
> record containing a pointer to its caller, the line number of the
> optimized tail call, and the bindings of locals.  It will look pretty
> much exactly like a frame object looks today, but we won't *call* it a
> frame object, and then everyone will be happy <wink>.

Especially because if we allocate such objects in the same way as we
allocate other objects, we no longer need to have an upper bound on the
number of such objects that can exist at one time.

For that matter, perhaps we could make such objects explicitly accessible by
user programs.  If we did that, of course, we would not want to restrict
such objects to the ones that correspond to tail calls--we might as well
create them for all calls.  We might even make it possible to "call" such an
object, the effect of which would be to cause execution to continue from the
state it was in when that object was created.  Gee, once we do that, we
don't need to keep that information around on the stack any more at all.

I guess we need a catchy name for such objects.  Because we can use them to
make execution continue from an arbitrary point, how about calling such an
object a "continuation?"  <very big wink>

From DavidA at ActiveState.com  Fri Jul 16 00:40:41 2004
From: DavidA at ActiveState.com (David Ascher)
Date: Fri Jul 16 00:40:48 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <e8bf7a530407150535869cf2d@mail.gmail.com>
References: <Pine.LNX.4.44.0407141423310.20797-100000@ccc1.wpi.edu>	<200407150136.i6F1aG928897@guido.python.org>
	<e8bf7a530407150535869cf2d@mail.gmail.com>
Message-ID: <40F707E9.80209@ActiveState.com>

Jeremy Hylton wrote:

> I like Python-fork as the name; it captures the experimental nature for me.

I don't.  I suspect people would see it as evidence of a (nonexistent) 
schism in the Python community.

--david
From tismer at stackless.com  Fri Jul 16 00:47:40 2004
From: tismer at stackless.com (Christian Tismer)
Date: Fri Jul 16 00:45:47 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to decref and
	set to NULL)
In-Reply-To: <40F3FD9F.2040808@zope.com>
References: <40F300DB.6050106@zope.com>	<1f7befae04071215506b00d40c@mail.gmail.com>	<40F3BD4C.70701@zope.com>	<1f7befae040713081023e68654@mail.gmail.com>
	<40F3FD9F.2040808@zope.com>
Message-ID: <40F7098C.7090906@stackless.com>

Jim Fulton wrote:

[visit macro that returns]

> I still don't like it, because it hides the return.
> 
> I'd rather do:
> 
>     if (self->foo != NULL && (vret = visit(self->foo, arg)))
>         return vret;
> 
> than
> 
>         VISIT(self->foo)

Just as a side note on semantically strange macros:

I did such a macro for easy error handling in Stackless months
ago, which sets an error and returns.
Before, the macro just had set an error.
The changed behavior was edited correctly in all but two places,
but I found the hidden error just by chance and very late.

Such side effects of changed macros can normally be avoided by
carefully avoiding type casts, to make sure that the compiler
recognizes incompatible changes. Unfortunately, an embedded
return statement bears *nothing* but changed semantics, and
this cannot be detected by a compiler.

This way, I managed to shoot myself seriously into the foot,
and that although I was working alone on the project for weeks.

I believe avoiding this is really a good idea.

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From tismer at stackless.com  Fri Jul 16 00:51:28 2004
From: tismer at stackless.com (Christian Tismer)
Date: Fri Jul 16 00:49:23 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to decref	and
	set to NULL)
In-Reply-To: <1f7befae040713083363a57a7b@mail.gmail.com>
References: <40F300DB.6050106@zope.com>	<1f7befae04071215506b00d40c@mail.gmail.com>	<40F3BD4C.70701@zope.com>	<1f7befae040713081023e68654@mail.gmail.com>	<40F3FD9F.2040808@zope.com>
	<1f7befae040713083363a57a7b@mail.gmail.com>
Message-ID: <40F70A70.7050303@stackless.com>

Tim Peters wrote:

> [Jim Fulton]
> 
>>I still don't like it, because it hides the return.
>>
>>I'd rather do:
>>
>>       if (self->foo != NULL && (vret = visit(self->foo, arg)))
>>               return vret;
>>
>>than
>>
>>        VISIT(self->foo)
> 
> 
> I don't believe you <0.5 wink>.  Here, rewrite this:
> 
> 	VISIT(self->readline);
> 	VISIT(self->read);
> 	VISIT(self->file);
> 	VISIT(self->memo);
> 	VISIT(self->stack);
> 	VISIT(self->pers_func);
> 	VISIT(self->arg);
> 	VISIT(self->last_string);
> 	VISIT(self->find_class);
>                 return 0;

How about

         if ( 0 ||
	       || VISIT(self->readline)
	       || VISIT(self->read)
	       || VISIT(self->file)
	    )
		return 1;

-- chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From dave at boost-consulting.com  Fri Jul 16 01:25:46 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Fri Jul 16 01:25:55 2004
Subject: [Python-Dev] Re: Proposal: C API Macro to decref and set to NULL
References: <40F300DB.6050106@zope.com>
	<1f7befae04071215506b00d40c@mail.gmail.com>
Message-ID: <u7jt4yhs5.fsf@boost-consulting.com>

Tim Peters <tim.peters@gmail.com> writes:

> [Jim Fulton]
> ...
>> If there are no objections I'll add the following definition to object.h,
>> after the definition for Py_DECREF:
>> 
>> #define Py_CLEAR(op)                            \
>>         do {                                   \
>>                 if (op) {                      \
>>                         PyObject *tmp = (op);  \
>>                         (op) = NULL;           \
>>                         Py_DECREF(tmp);                \
>>                 }                              \
>>         } while (0)
>> 
>> and update the docs and the tutorial on creating types in C.
>
> +1.  At least pystate.c can get rid of its functionally identical
> private ZAP macro then.

FWIW our experience in the C++ community suggests that
deallocate-and-NULL tends to hide bugs.  In general we prefer to use
deallocate-and-fill-pointer-with-garbage.  I'm not sure if the
experience translates here, of course.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From tismer at stackless.com  Fri Jul 16 01:33:09 2004
From: tismer at stackless.com (Christian Tismer)
Date: Fri Jul 16 01:31:02 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <00ee01c46ab9$14379ee0$6602a8c0@arkdesktop>
References: <00ee01c46ab9$14379ee0$6602a8c0@arkdesktop>
Message-ID: <40F71435.7040707@stackless.com>

Andrew Koenig wrote:

...

> I guess we need a catchy name for such objects.  Because we can use them to
> make execution continue from an arbitrary point, how about calling such an
> object a "continuation?"  <very big wink>

If it is an immutable object, you really *should* name it a
"continuation". But then you should ask yourself if you did
a very good job for the language :-)

<overly big wink> -- chris
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From tismer at stackless.com  Fri Jul 16 01:39:18 2004
From: tismer at stackless.com (Christian Tismer)
Date: Fri Jul 16 01:37:16 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <Pine.LNX.4.44.0407151214001.21569-100000@ccc2.wpi.edu>
References: <Pine.LNX.4.44.0407151214001.21569-100000@ccc2.wpi.edu>
Message-ID: <40F715A6.4080604@stackless.com>

Christopher T King wrote:

> On Thu, 15 Jul 2004, Bob Ippolito wrote:
...

>>Nearly all extension modules used with CPython should even be *binary* 
>>compatible with Stackless.
> 
> I didn't know this -- that's quite interesting information.

The only really incompatible module today to my knowledge that I use
is Psyco. I guess there is also one in the PyWin32 package...
Anyway, from today, Psyco is included in my supported module set,
and I'm about to tweak the code generator to support soft-switches
and pickling of Psyco-compiled functions.

Psyco also make a lot of use of frame->f_tstate, so I dropped my
removal of f_tstate.
Instead, I patched the generators to be consistent about f_tstate.
It will always be correct now, even if you call a generator from
different threads.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From tim.peters at gmail.com  Fri Jul 16 02:01:47 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Jul 16 02:01:49 2004
Subject: [Python-Dev] Re: Proposal: C API Macro to decref and set to NULL
In-Reply-To: <u7jt4yhs5.fsf@boost-consulting.com>
References: <40F300DB.6050106@zope.com>
	<1f7befae04071215506b00d40c@mail.gmail.com>
	<u7jt4yhs5.fsf@boost-consulting.com>
Message-ID: <1f7befae040715170142971b48@mail.gmail.com>

[David Abrahams, on the Py_CLEAR macro]
> FWIW our experience in the C++ community suggests that
> deallocate-and-NULL tends to hide bugs.  In general we prefer to use
> deallocate-and-fill-pointer-with-garbage.  I'm not sure if the
> experience translates here, of course.

I don't think so.  Along with the macro, Jim checked in a nice
explanation in the Extending and Embedding manual.  A decref can cause
anything to happen if the decref'ed containee is now trash, including
fresh accesses to the container object holding the now-trash containee
that was decrefed.  If the container's pointer to the containee isn't
NULLed out first, the entire implementation of the container has to be
aware that this particular containee may hold an insane value.  Ditto
if garbage is put in the pointer.  Ditto for NULL too, but NULL is a
single, obvious, and easy-to-test way to say "and this containee no
longer exists".  This isn't really so much about deallocation as about
horrid side effects triggered by deallocation.
From tismer at stackless.com  Fri Jul 16 02:09:04 2004
From: tismer at stackless.com (Christian Tismer)
Date: Fri Jul 16 02:07:06 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <Pine.LNX.4.44.0407151128300.20090-100000@ccc2.wpi.edu>
References: <Pine.LNX.4.44.0407151128300.20090-100000@ccc2.wpi.edu>
Message-ID: <40F71CA0.5030904@stackless.com>

Christopher T King wrote:

> On Thu, 15 Jul 2004, Bob Ippolito wrote:
> 
>>On Jul 15, 2004, at 11:10 AM, Christopher T King wrote:
>>
>>But in this case what is tail-call optimization going to do for you?  
>>You still require a stack at least the size of the height of your tree 
>>because of traverse(t.left) since that can not be tail-call optimized 
>>away, with the proposed algorithm.
> 
> In Andrew's example, he noted that it would only help for list-like 
> structures (i.e. those with mostly right nodes).
> 
>>I think Guido is in the right here, if you want to work around the 
>>recursion limit, use Stackless..  It should already be able to go just 
>>about as deep as you want.

Right.

> You're right -- even if Stackless doesn't do tail call optimization,
> implementation should be trivial.  But there's no guarantee when or even
> if Stackless will be merged with CPython, and until that happens,
> Stackless isn't an option for many (most?) people.

Wrong.

Stackless will always be there, one or two weeks later but will
be there.

There is a guarantee: It will never be part of CPython.
Proof: Just follow the postings in python-dev over the last five
years, and how Guido handled these.
If the proposals seemed to have a tiny chance, it was finally
shut down by some "And however you argue, I don't want it" like
statement. "If you need it, use it, that's where Stackless is for".
In a sense, I like this Groucho Marx approach of
"whatever it is, I'm against it". It is just not coherent with
all the feature-creaping-ins of the past years, this is
not consistent.

Some way I can accept this. But the whole story is partially like
cheating.
First, I was dissed because of Guido disliking continuations. OK!
Next, I was dissed because of rewriting lots of the core is
not feasible. OK!
Btw., I have almost solved this by some automation which could
apply sensible patches to almost every extension module.
But who cares.
Afterwards, I was dissed because compatibility to Jython
was made into a problem. OK!
Nice that I could turn a not-so-beloved kid into a really loved one.

After all (as said many times, already), I don't care, and I continue
with Stackless. Also by supporting more supportive languages like
Prothon (although I still don't understand why prototypes are better
than classes).

Anyway, removing recursive-descend evaluation from Python would
mean to remove Guido from Python, which is certainly not an
option. At least I won't try this for his life-time, and it
is also by no means my wish. I prefer to stick within my role as
the dark side of Python. :-)

If I only could *understand* him, that would really help.

ciao -- chris

p.s.: since yesterday, Psyco belongs to the "well supported special
modules for Stackless". Psyco is the only module that I know of that
heavily depends on frame layout. You have to recompile it, but then
it works really fine. I hope not to impose resentments on Psyco
by that :)
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From tismer at stackless.com  Fri Jul 16 02:24:06 2004
From: tismer at stackless.com (Christian Tismer)
Date: Fri Jul 16 02:21:58 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <200407151434.i6FEYkt30174@guido.python.org>
References: <1089894294.40f677961fc20@login.werra.lunarpages.com>	<1089894725.40f67945bc06d@login.werra.lunarpages.com>
	<877e9a17040715063463791eb6@mail.gmail.com>
	<200407151434.i6FEYkt30174@guido.python.org>
Message-ID: <40F72026.6030207@stackless.com>

Guido van Rossum wrote:

...

> Still -1.  I was understating my case: I find the "recursion is the
> basis of everything" attitude harmful.

How can you then still implement ceval.c in a recursive fashion,
since ceval is the "basis of everything" ???

In that sense, I'd like to discuss the "the Python interpreter
implementaton is harmful" issue.

But why should I bother. You have already reached more than you thought
to reach with your language, so why should I care about reaching you?
You probably wont even consider, so I don't even try, again.
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From tim.peters at gmail.com  Fri Jul 16 02:30:33 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Jul 16 02:30:35 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <00ee01c46ab9$14379ee0$6602a8c0@arkdesktop>
References: <00ee01c46ab9$14379ee0$6602a8c0@arkdesktop>
Message-ID: <1f7befae04071517306debf2d6@mail.gmail.com>

[Andrew Koenig]
> Especially because if we allocate such objects in the same way as we
> allocate other objects, we no longer need to have an upper bound on the
> number of such objects that can exist at one time.
>
> For that matter, perhaps we could make such objects explicitly accessible by
> user programs.  If we did that, of course, we would not want to restrict
> such objects to the ones that correspond to tail calls--we might as well
> create them for all calls.  We might even make it possible to "call" such an
> object, the effect of which would be to cause execution to continue from the
> state it was in when that object was created.  Gee, once we do that, we
> don't need to keep that information around on the stack any more at all.
>
> I guess we need a catchy name for such objects.  Because we can use them to
> make execution continue from an arbitrary point, how about calling such an
> object a "continuation?"  <very big wink>

Every sufficiently advanced implementation of Python will indeed
reinvent Stackless, but continuations are a "don't go there" topic
even for long-suffering Christian now <wink>.
From tim.peters at gmail.com  Fri Jul 16 02:45:20 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Jul 16 02:45:24 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to decref
	and set to NULL)
In-Reply-To: <40F70A70.7050303@stackless.com>
References: <40F300DB.6050106@zope.com>	<1f7befae04071215506b00d40c@mail.gmail.com>	<40F3BD4C.70701@zope.com>	<1f7befae040713081023e68654@mail.gmail.com>	<40F3FD9F.2040808@zope.com>
	<1f7befae040713083363a57a7b@mail.gmail.com>
	<40F70A70.7050303@stackless.com>
Message-ID: <1f7befae0407151745763a320d@mail.gmail.com>

[Christian Tismer]
> How about
> 
>         if ( 0 ||
>               || VISIT(self->readline)
>               || VISIT(self->read)
>               || VISIT(self->file)
>            )
>                return 1;

Well, "0 ||" doesn't do anything at all, it doesn't return the return
value of the visit() callback if a non-zero one is returned, it's
clumsier to write and read than what Jim checked in, and since
Py_VISIT does exactly one thing in exactly one context (tp_traverse
implementations), I have no fear at all of the "hidden" return. 
Almost all tp_traverse implementations can (and should) look exactly
alike:

Py_VISIT(member1);
Py_VISIT(member2);
...
Py_VISIT(member_n);
return 0;

and Py_VISIT has no other use case.  Why complicate it?  Sticking a
"return" in a general purpose macro could be horrid indeed; Py_VISIT
is so specific, and of such limited use, there's simply nothing to
fear here.  It makes writing "correct by casual inspection"
tp_traverse implementations dead easy, and has no effect on anything
else.  A continued +1 for the simplest thing that could possibly work
here (which is happily a 100% solution in this case too).
From ark-mlist at att.net  Fri Jul 16 02:53:09 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Fri Jul 16 02:53:09 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <1f7befae04071517306debf2d6@mail.gmail.com>
Message-ID: <010a01c46acf$442ad9d0$6602a8c0@arkdesktop>

> Every sufficiently advanced implementation of Python will indeed
> reinvent Stackless, but continuations are a "don't go there" topic
> even for long-suffering Christian now <wink>.

I will completely agree with you that programming with continuations
requires a special state of mind.

From tim.peters at gmail.com  Fri Jul 16 02:57:26 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Jul 16 02:57:29 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <40F656A2.70803@stackless.com>
References: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
	<40F4FF98.2000204@sweetapp.com> <40F656A2.70803@stackless.com>
Message-ID: <1f7befae040715175725c86abf@mail.gmail.com>

[Christian Tismer]
> ...
> Boehm GC might become feasible.
> If we then can write C extensions without having to think
> of reference counts, code would be simplified quite
> much, and I would save 80 percent of debugging time.

There's no escape from gc debugging nightmares at the C level,
regardless of gc strategy.  With Boehm gc (Bgc) you have to ensure
that all live objects are reachable from what Bgc guesses is the
current root set, and screwing that up is just as deadly as screwing
up refcounts.  It's harder to debug, though, because refcounting is
very simple to picture, while Bgc is a relatively huge pile of
complicated code in its own right.  I'm sure Neil Schemenauer still
has fond memories of the pleasant days he spent trying to account for
Tk crashes last time he tried hooking Bgc to Python <heh>.  Bgc has a
special entry point to call, to register roots that Bgc can't find on
its own -- guess how *you* find out which roots those are?  One crash
at a time, and in the presence of threads you're damned lucky to get a
repeatable test case.
From python at rcn.com  Thu Jul 15 15:17:48 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Jul 16 03:19:10 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to
	decrefand set to NULL)
In-Reply-To: <1f7befae0407151745763a320d@mail.gmail.com>
Message-ID: <000401c46a6e$25e49260$bb19c797@oemcomputer>

> Almost all tp_traverse implementations can (and should) look exactly
> alike:
> 
> Py_VISIT(member1);
> Py_VISIT(member2);
> ...
> Py_VISIT(member_n);
> return 0;

It does get a little more complicated for containers that loop over
their contents and possibly have other PyObject * fields also.

Still, I applied the macro to collections and itertools and found it to
be straight-forward and easy to use.


Raymond



From greg at cosc.canterbury.ac.nz  Fri Jul 16 03:41:04 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Jul 16 03:41:09 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <7B67AEB4-D629-11D8-B5AB-000A95A50FB2@fuhm.net>
Message-ID: <200407160141.i6G1f4hn012216@cosc353.cosc.canterbury.ac.nz>

James Y Knight <foom@fuhm.net>:

> The only thing you can do with an arbitrary PyObject * is access its
> ob_type or ob_refcnt. Anything else will break with objects today.

One other thing that would break is testing whether
an object is an integer and then accessing fields of
the integer object directly. Not that this seems a
likely thing for people to do, though.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From guido at python.org  Fri Jul 16 04:21:00 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Jul 16 04:21:07 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: Your message of "Thu, 15 Jul 2004 10:55:23 EDT."
	<Pine.LNX.4.44.0407151039150.20090-100000@ccc2.wpi.edu> 
References: <Pine.LNX.4.44.0407151039150.20090-100000@ccc2.wpi.edu> 
Message-ID: <200407160221.i6G2L0Q30988@guido.python.org>

> > > How about: Tail recursion "enables" recursion-oriented (functional)
> > > style? :)
> > 
> > Still -1.  I was understating my case: I find the "recursion is the
> > basis of everything" attitude harmful.
> 
> I think it's still best to "enable" such a style in some cases --
> Python's syntax is designed to allow formulation of a problem in the
> most natural way.  Sometimes it's more natural to formulate a
> problem as a recursive function than a procedural one, but the
> current implementation restricts this.
> 
> I agree that "recursion is the basis of everything" is harmful, but
> I think the same belief can apply to "procedularism is the basis of
> everything" (why do we have classes, list comprehensions, etc.?).
> Some things are better formulated in a procedural style, while
> others are better formulated in a recursive style.

And how should we decide whether something is better done without
recursion?  I currently have an easy answer: if it exceeds the
stack limit, you're probably spending too much time on function
calls...

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Fri Jul 16 04:29:31 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Jul 16 04:29:39 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to decref and
	set to NULL)
In-Reply-To: Your message of "Fri, 16 Jul 2004 00:51:28 +0200."
	<40F70A70.7050303@stackless.com> 
References: <40F300DB.6050106@zope.com>
	<1f7befae04071215506b00d40c@mail.gmail.com>
	<40F3BD4C.70701@zope.com>
	<1f7befae040713081023e68654@mail.gmail.com>
	<40F3FD9F.2040808@zope.com>
	<1f7befae040713083363a57a7b@mail.gmail.com> 
	<40F70A70.7050303@stackless.com> 
Message-ID: <200407160229.i6G2TVS31086@guido.python.org>

>          if ( 0 ||
> 	       || VISIT(self->readline)
> 	       || VISIT(self->read)
> 	       || VISIT(self->file)
> 	    )
> 		return 1;

Yuck.  Feels like double negatives, wheras the original felt "juuuuust
right" :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Fri Jul 16 04:35:24 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Jul 16 04:36:24 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: Your message of "Fri, 16 Jul 2004 02:24:06 +0200."
	<40F72026.6030207@stackless.com> 
References: <1089894294.40f677961fc20@login.werra.lunarpages.com>
	<1089894725.40f67945bc06d@login.werra.lunarpages.com>
	<877e9a17040715063463791eb6@mail.gmail.com>
	<200407151434.i6FEYkt30174@guido.python.org> 
	<40F72026.6030207@stackless.com> 
Message-ID: <200407160235.i6G2ZOm31140@guido.python.org>

> > Still -1.  I was understating my case: I find the "recursion is
> > the basis of everything" attitude harmful.
> 
> How can you then still implement ceval.c in a recursive fashion,
> since ceval is the "basis of everything" ???
> 
> In that sense, I'd like to discuss the "the Python interpreter
> implementaton is harmful" issue.
> 
> But why should I bother. You have already reached more than you
> thought to reach with your language, so why should I care about
> reaching you?  You probably wont even consider, so I don't even try,
> again.

Oh Christian, did you have a bad day again?

I didn't say "all recursion is evil".  For a language interpreter
recursion is perfect.

My dig was at the Schemers who would rather implement an array as a
linked list and then optimize tail-recursion in the hope of getting
some performance back -- in vain because they still can't get O(1)
indexing.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Fri Jul 16 04:37:09 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Jul 16 04:37:20 2004
Subject: [Python-Dev] Assignment to None
In-Reply-To: Your message of "Thu, 15 Jul 2004 04:35:34 EDT."
	<000201c46a46$b327c700$582acb97@oemcomputer> 
References: <000201c46a46$b327c700$582acb97@oemcomputer> 
Message-ID: <200407160237.i6G2b9h31171@guido.python.org>

> >>> None = 0
> SyntaxError: assignment to None (<pyshell#0>, line 1)
> >>> def f():
>             None = 0
>             
> SyntaxError: assignment to None (<pyshell#2>, line 2)
>  
> Does this imply that the compiler can now treat None as a constant?

I'd hope so!

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Fri Jul 16 04:37:21 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Jul 16 04:37:30 2004
Subject: [Python-Dev] Re: Proposal: C API Macro to decref and set to NULL
In-Reply-To: Your message of "Thu, 15 Jul 2004 20:01:47 EDT."
	<1f7befae040715170142971b48@mail.gmail.com> 
References: <40F300DB.6050106@zope.com>
	<1f7befae04071215506b00d40c@mail.gmail.com>
	<u7jt4yhs5.fsf@boost-consulting.com> 
	<1f7befae040715170142971b48@mail.gmail.com> 
Message-ID: <200407160237.i6G2bLJ31180@guido.python.org>

> [David Abrahams, on the Py_CLEAR macro]
> > FWIW our experience in the C++ community suggests that
> > deallocate-and-NULL tends to hide bugs.  In general we prefer to use
> > deallocate-and-fill-pointer-with-garbage.  I'm not sure if the
> > experience translates here, of course.

[Tim]
> I don't think so.  Along with the macro, Jim checked in a nice
> explanation in the Extending and Embedding manual.  A decref can cause
> anything to happen if the decref'ed containee is now trash, including
> fresh accesses to the container object holding the now-trash containee
> that was decrefed.  If the container's pointer to the containee isn't
> NULLed out first, the entire implementation of the container has to be
> aware that this particular containee may hold an insane value.  Ditto
> if garbage is put in the pointer.  Ditto for NULL too, but NULL is a
> single, obvious, and easy-to-test way to say "and this containee no
> longer exists".  This isn't really so much about deallocation as about
> horrid side effects triggered by deallocation.

Plus, in many cases NULL is *already* an expected value.

We even have macros like Py_XDECREF() that are NULL-aware.

--Guido van Rossum (home page: http://www.python.org/~guido/)

--i6G2WKR31130.1089945140/guido.python.org--
From greg at cosc.canterbury.ac.nz  Fri Jul 16 05:55:32 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Jul 16 05:55:38 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <00ee01c46ab9$14379ee0$6602a8c0@arkdesktop>
Message-ID: <200407160355.i6G3tWke012433@cosc353.cosc.canterbury.ac.nz>

> > So let's optimize tail calls, but for each elided one we'll allocate a
> > record containing a pointer to its caller, the line number of the
> > optimized tail call, and the bindings of locals.  It will look pretty
> > much exactly like a frame object looks today, but we won't *call* it a
> > frame object, and then everyone will be happy <wink>.
> 
> Especially because if we allocate such objects in the same way as we
> allocate other objects, we no longer need to have an upper bound on the
> number of such objects that can exist at one time.

We could get the benefits of that just by eliminating recursive calls
to ceval() when a Python function calls a Python function. I think
that would be a worthwhile thing to do on its own, because it would
mean recursion in pure Python would be limited only by available
memory and not an arbitrary recursion limit, which has always seemed
like a kludge to me.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Fri Jul 16 06:00:34 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Jul 16 06:00:40 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <2mwu15vdow.fsf@starship.python.net>
Message-ID: <200407160400.i6G40YBW012441@cosc353.cosc.canterbury.ac.nz>

Michael Hudson <mwh@python.net>:

> It's kinda hard to work seriously with lists and strings without
> integers :-)

That may be true in C, but I think it's far less true in languages
like Python that have high-level iteration facilities and string and
list manipulation functions.  One still uses *some* integers, but my
feeling is they're a much smaller proportion of the mix.

> This was, at the very least, a worthwhile experiment...

Certainly, but it needs to me measured on typical code as well, not
just code artificially designed to exercise integer operations, before
judging whether it's worth the effort.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From pje at telecommunity.com  Fri Jul 16 06:08:22 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Jul 16 06:05:05 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <200407160141.i6G1f4hn012216@cosc353.cosc.canterbury.ac.nz>
References: <7B67AEB4-D629-11D8-B5AB-000A95A50FB2@fuhm.net>
Message-ID: <5.1.1.6.0.20040715234833.02262910@mail.telecommunity.com>

At 01:41 PM 7/16/04 +1200, Greg Ewing wrote:
>James Y Knight <foom@fuhm.net>:
>
> > The only thing you can do with an arbitrary PyObject * is access its
> > ob_type or ob_refcnt. Anything else will break with objects today.
>
>One other thing that would break is testing whether
>an object is an integer and then accessing fields of
>the integer object directly. Not that this seems a
>likely thing for people to do, though.

I guess we'd need to XXX-ify ob_ival as well, then.  There's already a 
PyInt_AS_LONG macro that should be used for that purpose.

So far, the -1000 from Guido makes it seem unlikely this'll get accepted, 
though.

I am sort of curious, though, as to how many extension modules would 
actually require changes to support this.  A bit of grepping through the 
source of several I use, seems to indicate that it isn't that common to use 
ob_type, and it's most likely to appear:

1) in the setup of a type object, setting the type's type to type  :)  (by 
far the most common, and not a usage that needs to be changed)

2) in the definition of a macro that verifies the type of an extension object

3) in allocation/deallocation routines

Only uses 2 and 3 would need to be changed, and they tend to be quite isolated.

By contrast, I didn't find any ob_ival uses, and the only place I saw 
direct use of ob_refcnt was in some Pyrex-generated tp_dealloc code.

Of course, my sampling was hardly statistically valid.  I imagine the real 
extensions to check out would be things like SciPy, Numeric, PIL, wxPython, 
Boost::Python, win32all, PyCrytpo, etc.  That is, widely used packages of 
very large size.

If I understand correctly, extensions have to be recompiled to move between 
Python versions anyway, and compared to some previous C API changes, this 
one actually seems pretty minor.  Extensions like 'kjbuckets' that were 
written for early 1.x Python versions definitely needed work to make them 
build with the later 1.x and 2.x versions.


From pje at telecommunity.com  Fri Jul 16 06:13:46 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Jul 16 06:10:23 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <200407160355.i6G3tWke012433@cosc353.cosc.canterbury.ac.nz>
References: <00ee01c46ab9$14379ee0$6602a8c0@arkdesktop>
Message-ID: <5.1.1.6.0.20040716000900.01e9bd40@mail.telecommunity.com>

At 03:55 PM 7/16/04 +1200, Greg Ewing wrote:

>We could get the benefits of that just by eliminating recursive calls
>to ceval() when a Python function calls a Python function. I think
>that would be a worthwhile thing to do on its own, because it would
>mean recursion in pure Python would be limited only by available
>memory and not an arbitrary recursion limit, which has always seemed
>like a kludge to me.

Yeah, but it's a *useful* kludge to have a recursion limit.  Most 
algorithms that are "sensibly" recursive have some fan-out at each 
recursion level, such that the total recursion needed is something like 
log2N.  So as N grows, the relative amount of recursion shrinks.  A 
4-billion element binary tree traversal only needs 32 recursion levels.

So, realistically speaking, if you have hundreds of levels of recursion 
going on, you're probably doing something that should be expressed 
iteratively, or you're using 64-bit Python, in which case you probably have 
the stack space and CPU power to spare anyway.  ;)

From dave at boost-consulting.com  Fri Jul 16 06:33:19 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Fri Jul 16 06:33:34 2004
Subject: [Python-Dev] Re: Proposal: C API Macro to decref and set to NULL
References: <40F300DB.6050106@zope.com>
	<1f7befae04071215506b00d40c@mail.gmail.com>
	<u7jt4yhs5.fsf@boost-consulting.com>
	<1f7befae040715170142971b48@mail.gmail.com>
	<200407160237.i6G2bLJ31180@guido.python.org>
Message-ID: <un020vaeo.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

>> [David Abrahams, on the Py_CLEAR macro]
>> > FWIW our experience in the C++ community suggests that
>> > deallocate-and-NULL tends to hide bugs.  In general we prefer to use
>> > deallocate-and-fill-pointer-with-garbage.  I'm not sure if the
>> > experience translates here, of course.
>
> [Tim]
>> I don't think so.  Along with the macro, Jim checked in a nice
>> explanation in the Extending and Embedding manual.  A decref can cause
>> anything to happen if the decref'ed containee is now trash, including
>> fresh accesses to the container object holding the now-trash containee
>> that was decrefed.  If the container's pointer to the containee isn't
>> NULLed out first, the entire implementation of the container has to be
>> aware that this particular containee may hold an insane value.  Ditto
>> if garbage is put in the pointer.  Ditto for NULL too, but NULL is a
>> single, obvious, and easy-to-test way to say "and this containee no
>> longer exists".  This isn't really so much about deallocation as about
>> horrid side effects triggered by deallocation.
>
> Plus, in many cases NULL is *already* an expected value.
>
> We even have macros like Py_XDECREF() that are NULL-aware.

That's exactly the sort of case that covers up bugs in C++.

Our operator delete basically just ignores NULL values.  When people
write

   delete p; p = 0;

It can cover up bugs where p is unintentionally getting deleted
twice.  It might still not be applicable here, though.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From eppstein at ics.uci.edu  Fri Jul 16 07:36:54 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Fri Jul 16 07:37:01 2004
Subject: [Python-Dev] Re: Proper tail recursion
References: <00ee01c46ab9$14379ee0$6602a8c0@arkdesktop>
	<200407160355.i6G3tWke012433@cosc353.cosc.canterbury.ac.nz>
	<5.1.1.6.0.20040716000900.01e9bd40@mail.telecommunity.com>
Message-ID: <eppstein-1543D6.22365415072004@sea.gmane.org>

In article <5.1.1.6.0.20040716000900.01e9bd40@mail.telecommunity.com>,
 "Phillip J. Eby" <pje@telecommunity.com> wrote:

> At 03:55 PM 7/16/04 +1200, Greg Ewing wrote:
> 
> >We could get the benefits of that just by eliminating recursive calls
> >to ceval() when a Python function calls a Python function. I think
> >that would be a worthwhile thing to do on its own, because it would
> >mean recursion in pure Python would be limited only by available
> >memory and not an arbitrary recursion limit, which has always seemed
> >like a kludge to me.
> 
> Yeah, but it's a *useful* kludge to have a recursion limit.  Most 
> algorithms that are "sensibly" recursive have some fan-out at each 
> recursion level, such that the total recursion needed is something like 
> log2N.  So as N grows, the relative amount of recursion shrinks.  A 
> 4-billion element binary tree traversal only needs 32 recursion levels.
> 
> So, realistically speaking, if you have hundreds of levels of recursion 
> going on, you're probably doing something that should be expressed 
> iteratively, or you're using 64-bit Python, in which case you probably have 
> the stack space and CPU power to spare anyway.  ;)

Where the recursion limit really bites me is the inability to do 
recursive depth first search on big graphs.  Of course, I can simulate 
the stack myself and write the rest iteratively, but if I wanted to do 
that I'd go back to writing in assembly language.

-- 
David Eppstein                      http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science

From vinay_sajip at red-dove.com  Fri Jul 16 08:02:37 2004
From: vinay_sajip at red-dove.com (Vinay Sajip)
Date: Fri Jul 16 08:00:28 2004
Subject: [Python-Dev] Planned changes to logging
References: <000f01c45c99$5e644dc0$652b6992@alpha>
	<d7cee4a704071422523dc9173c@mail.gmail.com>
Message-ID: <007201c46afa$7fb8c400$652b6992@alpha>

> One is a usability bug: calling fileConfig() disables any logger
> objects already created that are not explicitly specified in the log
> config file. Since loggers are often created when a module is

It's not a bug, it's by design. The semantics of fileConfig() is that it
completely replaces the existing configuration (in some scenarios, that's
what you want). I'm thinking about a better way to configure the system -
incremental rather than all-or-nothing. I will update the docstring to
indicate this, as it's perhaps not clear enough.

> imported, it's hard to keep track of the order in which loggers are
> created, leading to unexpected behavior because an unspecifed logger
> created after the call to fileConfig behave differently -- it will
> inherit its settings from its parent logger.
> Here's an example illustrating the problem:
>
> import logging, logging.config
> import mymodule #this module contains a line like log =
> logging.getLogger("mymodule")
>
> if __name__ == '__main__':
>   loggging.config.fileConfig("log.config")
>   #mymodule.log is now disabled!
>   #this seems like a bad design -- difficult to figure out what's going on
>   #work-around: re-enable the loggers
>   for logger in logging.Logger.manager.loggerDict.itervalues():
>       logger.disabled = 0

It'd be easier to comment out the line in config.py which sets disabled to
1, if this is behaviour you want in all your applications.

> Second, i'd recommend a minor change that would greatly increase the
> flexibility of the architecture: store the entire keyword dictionary
> in the LogRecord, not just exc_info -- that way arbitrary objects
> could be passed to particular handlers or formatters that know how to
> use them.  Some use cases for this:
> * pass a unique error number.
> * pass structured information about the message, e.g. for writing to a
> database table
> * for a localized message formatter (LogRecord already stores the
> argument list for string interpolation but this is cumbersome for
> localization since they depends on the order they appear in the
> message).
>
The whole area of LogRecord construction needs to be made more extensible.
I'm in discussions with Fred Drake about this, and hopefully soon will be
able to implement a better mechanism.

Fred - if you're listening - :-)

Thanks for the feedback.

Regards,

Vinay

From bac at OCF.Berkeley.EDU  Fri Jul 16 08:03:45 2004
From: bac at OCF.Berkeley.EDU (Brett Cannon)
Date: Fri Jul 16 08:03:49 2004
Subject: [Python-Dev] Adding directories to configure.in for fink?
Message-ID: <40F76FC1.3040908@ocf.berkeley.edu>

I discovered that when compiling for readline against a Fink install a 
warning was spit out about the lack of a function definition.  Turned 
out a function that #ifdef'ed a macro was not being triggered because 
configure.in was not catching the header file in /sw/lib for readline.

Since /sw is already looked at by setup.py, is there any way to have 
configure.in do it as well so as to have some consistency?  Or is 
prepending ``LDFLAGS=/sw/lib`` the only way?  If so, does it warrant 
mentioning in the README?

-Brett
From adamsz at gmail.com  Fri Jul 16 09:18:01 2004
From: adamsz at gmail.com (Adam Souzis)
Date: Fri Jul 16 09:18:04 2004
Subject: [Python-Dev] Planned changes to logging
In-Reply-To: <007201c46afa$7fb8c400$652b6992@alpha>
References: <000f01c45c99$5e644dc0$652b6992@alpha>
	<d7cee4a704071422523dc9173c@mail.gmail.com>
	<007201c46afa$7fb8c400$652b6992@alpha>
Message-ID: <d7cee4a704071600185e6fba05@mail.gmail.com>

On Fri, 16 Jul 2004 07:02:37 +0100, Vinay Sajip
<vinay_sajip@red-dove.com> wrote:
> > One is a usability bug: calling fileConfig() disables any logger
> > objects already created that are not explicitly specified in the log
> > config file. Since loggers are often created when a module is
> 
> It's not a bug, it's by design. The semantics of fileConfig() is that it
> completely replaces the existing configuration (in some scenarios, that's
> what you want). I'm thinking about a better way to configure the system -
> incremental rather than all-or-nothing. I will update the docstring to
> indicate this, as it's perhaps not clear enough.
> 

OK, but a better implementation of these semantics would be to have
logging to one of these "replaced" log objects raise an exception
instead of continuing to accept log messages but silently have them
disappear into the ether -- that's what I mean by a "usability bug".

Better still, it seems entirely consistent with these semantics to
have the existing log objects reset to their parent settings -- just
like unspecified log objects created after the confguration load do. 
I think this is more consistent, more intuitive, and doesn't require
you to figure out what logs may or exist throughout the system when
loading a configuration.

regards,
adam
From jcarlson at uci.edu  Fri Jul 16 09:52:07 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Fri Jul 16 09:58:21 2004
Subject: [Python-Dev] Re: Proper tail recursion
In-Reply-To: <eppstein-1543D6.22365415072004@sea.gmane.org>
References: <5.1.1.6.0.20040716000900.01e9bd40@mail.telecommunity.com>
	<eppstein-1543D6.22365415072004@sea.gmane.org>
Message-ID: <20040715235400.6990.JCARLSON@uci.edu>


> Where the recursion limit really bites me is the inability to do 
> recursive depth first search on big graphs.  Of course, I can simulate 
> the stack myself and write the rest iteratively, but if I wanted to do 
> that I'd go back to writing in assembly language.



If there were a practical method to remove recursion limits (seems like
one is possible), that also preserved stack traces (seems like it can be
done), with a method to set and get a synthetic recursion depth limit
that gets checked before any call is made (seems like one already exists),
and is set to a reasonable depth initally (like 1000), but can be set to
None (no limit) by experienced users, then we would have the best of
both worlds: a limit for mortals, no limit for experienced users.  Heck,
we could even require setting the recursion limit twice before it takes
effect (those that want to use it must read the manual about how its use
is discouraged).


Now, offering people enough rope to hang themselves with*, does not mean
that it would be encouraged, and maybe allowing unlimited recursion with
tracebacks should be a compile-time option (same internals, only one
allows the recursion limit to be set, the other doesn't).

If unlimited recursion were allowed, perhaps limited tracebacks (first
and last 500 stack frame traces), RLE tracebacks (though a clever
programmer could easily destroy the generator with fewer than 64
functions, but we probably shouldn't do that), or combined limited and
RLE tracebacks would be sufficient. What do I mean by an RLE traceback?

>>> def f(n):
...     if n > 1:
...             f(n-1)
... 
>>> f(1000)
'Traceback (most recent call last):'
  File "<stdin>", line 1, in f
999x:
      File "<stdin>", line 3, in f
RuntimeError: maximum recursion depth exceeded
>>> def g(n):
...     if n > 1:
...             h(n)
...
>>> def h(n):
...     if n > 1:
...             g(n)
...
>>> g(1000)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
499x:
       File "<stdin>", line 3, in g
       File "<stdin>", line 3, in h
  File "<stdin>", line 3, in g
RuntimeError: maximum recursion depth exceeded
>>> #the following lines are a combination
... 
>>> f(2000)
'Traceback (most recent call last):'
  File "<stdin>", line 1, in f
499x:
      File "<stdin>", line 3, in f
***1000 levels of traceback clipped for your protection***
500x:
      File "<stdin>", line 3, in f
RuntimeError: maximum recursion depth exceeded
>>> 


Heck, the above would be nice even without unlimited recursion.


 - Josiah

*
1. giant tracebacks caused by running out of memory, being constructed
in memory, causing annother out of memory error, which then causes a
traceback to be printed while printing the traceback, which loses the
real traceback
2. giant tracebacks being printed to the logging module
...others I'm sure...

From Jack.Jansen at cwi.nl  Fri Jul 16 10:20:40 2004
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Fri Jul 16 10:20:00 2004
Subject: [Python-Dev] Adding directories to configure.in for fink?
In-Reply-To: <40F76FC1.3040908@ocf.berkeley.edu>
References: <40F76FC1.3040908@ocf.berkeley.edu>
Message-ID: <06559C87-D701-11D8-A243-000A958D1666@cwi.nl>


On 16-jul-04, at 8:03, Brett Cannon wrote:

> I discovered that when compiling for readline against a Fink install a 
> warning was spit out about the lack of a function definition.  Turned 
> out a function that #ifdef'ed a macro was not being triggered because 
> configure.in was not catching the header file in /sw/lib for readline.
>
> Since /sw is already looked at by setup.py, is there any way to have 
> configure.in do it as well so as to have some consistency?  Or is 
> prepending ``LDFLAGS=/sw/lib`` the only way?  If so, does it warrant 
> mentioning in the README?

Hmm.
I guess we should really be consistent about this: if we look in /sw 
for setup.py I guess we should also do it for configure.

But I'm not too thrilled with fink and /sw support in general: it's 
just too easy to build an executable or plugin module that doesn't work 
on other machines (if you have fink stuff installed and the other 
machine doesn't). And while this is uncommon on most unix machines it's 
much more common on the Mac.

Could we add a configure flag --with[out]-fink that governs adding /sw 
to search paths in both configure and setup.py, or is that overkill?
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman

From ronaldoussoren at mac.com  Fri Jul 16 10:56:17 2004
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Fri Jul 16 10:56:29 2004
Subject: [Python-Dev] Adding directories to configure.in for fink?
In-Reply-To: <06559C87-D701-11D8-A243-000A958D1666@cwi.nl>
References: <40F76FC1.3040908@ocf.berkeley.edu>
	<06559C87-D701-11D8-A243-000A958D1666@cwi.nl>
Message-ID: <FFC54B0E-D705-11D8-922A-0003931CFE24@mac.com>


On Jul 16, 2004, at 10:20 AM, Jack Jansen wrote:

>
> On 16-jul-04, at 8:03, Brett Cannon wrote:
>
>> I discovered that when compiling for readline against a Fink install 
>> a warning was spit out about the lack of a function definition.  
>> Turned out a function that #ifdef'ed a macro was not being triggered 
>> because configure.in was not catching the header file in /sw/lib for 
>> readline.
>>
>> Since /sw is already looked at by setup.py, is there any way to have 
>> configure.in do it as well so as to have some consistency?  Or is 
>> prepending ``LDFLAGS=/sw/lib`` the only way?  If so, does it warrant 
>> mentioning in the README?
>
> Hmm.
> I guess we should really be consistent about this: if we look in /sw 
> for setup.py I guess we should also do it for configure.
>
> But I'm not too thrilled with fink and /sw support in general: it's 
> just too easy to build an executable or plugin module that doesn't 
> work on other machines (if you have fink stuff installed and the other 
> machine doesn't). And while this is uncommon on most unix machines 
> it's much more common on the Mac.
>
> Could we add a configure flag --with[out]-fink that governs adding /sw 
> to search paths in both configure and setup.py, or is that overkill?

And what about DarwinPorts? They use /opt/local by default.

Ronald

From stephen at xemacs.org  Fri Jul 16 11:25:39 2004
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri Jul 16 11:25:50 2004
Subject: [Python-Dev] Adding directories to configure.in for fink?
In-Reply-To: <06559C87-D701-11D8-A243-000A958D1666@cwi.nl> (Jack Jansen's
	message of "Fri, 16 Jul 2004 10:20:40 +0200")
References: <40F76FC1.3040908@ocf.berkeley.edu>
	<06559C87-D701-11D8-A243-000A958D1666@cwi.nl>
Message-ID: <878ydks3qk.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Jack" == Jack Jansen <Jack.Jansen@cwi.nl> writes:

    Jack> But I'm not too thrilled with fink and /sw support in
    Jack> general: it's just too easy to build an executable or plugin
    Jack> module that doesn't work on other machines (if you have fink
    Jack> stuff installed and the other machine doesn't). And while
    Jack> this is uncommon on most unix machines it's much more common
    Jack> on the Mac.

If you're saying that you think Python should ignore Fink in both
configure and setup.py, I tend to agree, although I'm a Fink user.
Mostly I use Fink sources; they take care of it.

    Jack> Could we add a configure flag --with[out]-fink that governs
    Jack> adding /sw to search paths in both configure and setup.py,
    Jack> or is that overkill?

If you're going to do this for Fink, don't you want to do it for
DarwinPorts too?  XEmacs's configure has a --site-prefixes switch such
that --site-prefixes=/sw does both --site-includes=/sw/include and
--site-libraries=/sw/lib.  This handles both Fink and DarwinPorts (and
anythign else that might come along) in a sensible way.  Isn't that a
generic autoconf feature?

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.
From dan.gass at gmail.com  Fri Jul 16 11:39:00 2004
From: dan.gass at gmail.com (Dan Gass)
Date: Fri Jul 16 11:39:03 2004
Subject: [Python-Dev] [Python Dev] HTML side by side diff - can it go in a2?
Message-ID: <bc15ba280407160239935c7ca@mail.gmail.com>

Python Patch #914575
(https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914575&group_id=5470)

The patch now includes documentation, code, and test changes (in
contextual diff format) per a request by Martin v. L?wis in the patch
followups.  I have also tweaked the user interface and am now
comfortable with it.  I have seen positive comments from Tim Peters
and others and have implemented all suggestions to date.

Can this patch be considered for the alpha 2 release?

This new feature could be useful for doing full Python build
comparisons (generating an HTML page listing all files that changed
with hyper links to pages showing the side by side differences).  If
there would be a use for such a tool to make your release review
easier or to get a good feeling in your gut before it goes out the
door I would be more than happy to assist in writing one.  The tool
would be relatively easy to write.

Dan Gass
From theller at python.net  Fri Jul 16 12:19:38 2004
From: theller at python.net (Thomas Heller)
Date: Fri Jul 16 12:19:54 2004
Subject: [Python-Dev] bdist_wininst
Message-ID: <8ydkuudh.fsf@python.net>

bdist_wininst has seen a lot of bug fixes and improvements, mostly by
Mark, partly by me.

I'd like to 'improve' the visual appearance, too.

First, I would like to add an XP style manifest as resource in the exe,
which displays the gui elements in the XP style when running under XP.
I don't think this will create problems on older windows versions, but
if anyone knows better please speak up.

Second, the default bitmap it displays is the old Python Powered logo by
Just (that with the cool license ;-).  A much better bitmap would be
that from <http://tinyurl.com/595xd>, which seems to have been made for
the PSF.  There are two problems with this image: The size is 122x261
pixels, while bdist_wininst would prefer 152x261 pixels.  Maybe someone
can recreate the icon with a changed size?
The file size, when it is converted to 24-bit color bmp, is
nearly 100 kB, which is a bit large imo for an optical gimmik - but this
can probably be fixed by compressing the image in the installer itself.
Finally, I'm not sure I'm allowed to use and redistribute this image.

Third, the installer executable doesn't have an icon.  Personally I
don't care too much about that, but Phil Hornby made two icons which he
posted to the distutils list (the gmane archives do a nice job to show
the icons on the web page):

<http://article.gmane.org/gmane.comp.python.distutils.devel/852>
<http://article.gmane.org/gmane.comp.python.distutils.devel/855>

Note that with standard Python it is not possible to change the default
icon to something else, although it could be done with pywin32, or with
some tools included in py2exe.  So, distutils bdist_wininst probably can
not provide an option to change the icon to something else.
Also I know that some people build bdist_wininst installers on Linux,
which is possible for pure python package distributions, but it wouldn't
be possible to change the icon there anyway.  So, the question is: is
this icon good enough to have it in basically every distutils' created
windows installer?

Thomas

From jhylton at gmail.com  Fri Jul 16 13:50:30 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Fri Jul 16 13:50:33 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <200407160235.i6G2ZOm31140@guido.python.org>
References: <1089894294.40f677961fc20@login.werra.lunarpages.com>
	<1089894725.40f67945bc06d@login.werra.lunarpages.com>
	<877e9a17040715063463791eb6@mail.gmail.com>
	<200407151434.i6FEYkt30174@guido.python.org> 
	<40F72026.6030207@stackless.com>
	<200407160235.i6G2ZOm31140@guido.python.org>
Message-ID: <e8bf7a53040716045044c8170d@mail.gmail.com>

On Thu, 15 Jul 2004 19:35:24 -0700, Guido van Rossum <guido@python.org> wrote:
> My dig was at the Schemers who would rather implement an array as a
> linked list and then optimize tail-recursion in the hope of getting
> some performance back -- in vain because they still can't get O(1)
> indexing.

In all fairness, a serious Schemer would probabliy use the builtin
vector type for an array rather than cons cells.

Jeremy
From colanderman at gmail.com  Fri Jul 16 14:55:08 2004
From: colanderman at gmail.com (Chris King)
Date: Fri Jul 16 14:55:11 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <200407160355.i6G3tWke012433@cosc353.cosc.canterbury.ac.nz>
References: <200407160355.i6G3tWke012433@cosc353.cosc.canterbury.ac.nz>
Message-ID: <875c7e0704071605556a22d022@mail.gmail.com>

On Fri, 16 Jul 2004 15:55:32 +1200, Greg Ewing
<greg@cosc.canterbury.ac.nz> wrote:
> We could get the benefits of that just by eliminating recursive calls
> to ceval() when a Python function calls a Python function. I think
> that would be a worthwhile thing to do on its own, because it would
> mean recursion in pure Python would be limited only by available
> memory and not an arbitrary recursion limit, which has always seemed
> like a kludge to me.

The recursion limit isn't imposed so much by the C stack as it is by
an an actual, arbitrary limit enforced by the interpreter (presumably
to protect the C stack), one which can be effectively nullified in
Python by feeding a proper value to sys.setrecursionlimit().

The point of my patch is not to eliminate the recursive nature of the
interpreter (that's what Stackless is for), but to eliminate the
memory usage required by recursive functions -- after 100,000
recursions, normal Python eats up nearly 80MB, whereas with this
patch, it just eats up its standard 4 or 5MB or what-have-you.  An
unfortunate side effect of this is, of course, the elimination of
correct stack traces.
From amk at amk.ca  Fri Jul 16 15:18:11 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Fri Jul 16 15:19:08 2004
Subject: [Python-Dev] 
	Re: [Python-checkins] python/dist/src/Python compile.c, 2.306, 2.307
In-Reply-To: <E1BlRav-0007WL-R7@sc8-pr-cvs1.sourceforge.net>
References: <E1BlRav-0007WL-R7@sc8-pr-cvs1.sourceforge.net>
Message-ID: <20040716131811.GA27072@rogue.amk.ca>

On Fri, Jul 16, 2004 at 05:13:41AM -0700, 
	rhettinger@users.sourceforge.net wrote:
> Treat None as a constant.

This is probably something stupid on my part, but in my copy of the
built tree it's still a syntax warning:

Python 2.4a1+ (#13, Jul 16 2004, 09:11:38)
[GCC 3.3.2 (Debian)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> None = 0
<stdin>:1: SyntaxWarning: assignment to None
>>> print None   # But assignment has no effect... perhaps due to this optim.?
None
>>> def f():
...    None = 0
...
<stdin>:2: SyntaxWarning: assignment to None
>>> f()

My CVS tree is up-to-date, and I did a "make clean".  Is everyone else
getting the SyntaxError for this?

--amk

From python at rcn.com  Fri Jul 16 03:22:25 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Jul 16 15:23:39 2004
Subject: [Python-Dev] Prototypes  [was: Proper tail recursion]
In-Reply-To: <40F71CA0.5030904@stackless.com>
Message-ID: <004001c46ad3$5a1b5180$bb19c797@oemcomputer>

 [Michael Chermside] 
> I wouldn't say that tail recursion *encouraged* a functional style,
> rather I would say that refusing to accept a working version of
> tail recursion actively *discourages* the use of that style.
> 
> Now, I understand that a functional style is not for everybody. And
> I realize that you want to design Python so that people will continue
> to use it to write programs that are readable (and understandable)
> by everyone. But I believe strongly in allowing different programming
> paradigms -- using procedural, OOP, or functional programming styles
> (and others) for the particular problems that they express best.
> And I think that *discouraging* a functional style by intentionally
> breaking it is a bad idea.
> 
> Rejecting an implementation of tail recursion because it doesn't
> work well or it slows down other cases is perfectly reasonable. But
> rejecting it because it allows some people's programs to work
better...
> that just seems wrongheaded.


All of this has gotten me thinking again about a pending (currently out
of favor) feature request to loosen a syntax rule, with the goal of
helping out people who want to experiment with prototype OO.  The
request is make it possible to write:

    def Account.deposit(self, v):
        self.balance += v

Currently, it has to be written:

    def _deposit(self, v):
        self.balance += v
    Account.deposit = _deposit
    del _deposit

While it seems minor, this little relaxation of rules is essential for
working with prototype OO.  On comp.lang.py, I remember someone posting
a recipe for an object that supported prototype style cloning.  The only
thing missing to make it useable was this syntax change.

Outside of prototype OO, the Lua programming language also offers this
syntax as a way to define functions in a specified namespace.  Their
experiences with it have been favorable.

The reason is the feature request is currently out of favor is the
possibility that the syntax would be abused and people would start
defining their methods outside of class definitions.  

My feeling is that we document a little discouragement and then let
people do whatever they want.  After all, they can already define
methods outside of a class definition, it just takes an extra line or
two as shown above.  We can also take a lesson from the for-loop syntax
which currently allows "for i, a[i] in enumerate(b)".  Though it is
available and easily abused, it has not corrupted the masses.  The
lesson is that the prototype style can be facilitated without mucking up
anything else.


Raymond

From colanderman at gmail.com  Fri Jul 16 15:23:15 2004
From: colanderman at gmail.com (Chris King)
Date: Fri Jul 16 15:23:52 2004
Subject: [Python-Dev] Re: Proper tail recursion
In-Reply-To: <20040715235400.6990.JCARLSON@uci.edu>
References: <5.1.1.6.0.20040716000900.01e9bd40@mail.telecommunity.com>
	<eppstein-1543D6.22365415072004@sea.gmane.org>
	<20040715235400.6990.JCARLSON@uci.edu>
Message-ID: <875c7e0704071606231c8f5795@mail.gmail.com>

On Fri, 16 Jul 2004 00:52:07 -0700, Josiah Carlson <jcarlson@uci.edu> wrote:

> If unlimited recursion were allowed, perhaps limited tracebacks (first
> and last 500 stack frame traces), RLE tracebacks (though a clever
> programmer could easily destroy the generator with fewer than 64
> functions, but we probably shouldn't do that), or combined limited and
> RLE tracebacks would be sufficient. What do I mean by an RLE traceback?

I thought of doing RLE tracebacks, but compression fails in the case
of cooperative recursive calls.  I think perhaps along with the
sys.setrecursionlimit(None) option you suggest, developers should be
allowed to turn recursive tracebacks off entirely in the case of
cooperative recursive calls.

The other problem with RLE tracebacks is that a traceback object also
keeps a reference to each function's locals (by virtue of keeping a
reference to its frame obejct).  Throwing this info out makes RLE
tracebacks no more useful to debuggers than having no traceback at
all.

Keeping the first and last X frames in a traceback seems reasonable,
but this would similarly cripple debuggers (what happens if the bug is
in the (X+1)th frame?).  Implementation would also be complicated.

IMHO it should be an all-or-nothing deal: either the programmer turns
tail-call optimizations on to nullify memory uses, or turns them off
to facilitate debugging.
From amk at amk.ca  Fri Jul 16 15:33:14 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Fri Jul 16 15:34:11 2004
Subject: [Python-Dev] Prototypes  [was: Proper tail recursion]
In-Reply-To: <004001c46ad3$5a1b5180$bb19c797@oemcomputer>
References: <40F71CA0.5030904@stackless.com>
	<004001c46ad3$5a1b5180$bb19c797@oemcomputer>
Message-ID: <20040716133314.GA27141@rogue.amk.ca>

eeOn Thu, Jul 15, 2004 at 09:22:25PM -0400, Raymond Hettinger wrote:
> helping out people who want to experiment with prototype OO.  The
> request is make it possible to write:
> 
>     def Account.deposit(self, v):
>         self.balance += v

Could function decorators be used for this?  Something like:

@attach_to_object(Account)
def deposit (self, v):
    ...
del deposit

The decorator couldn't delete the binding of 'deposit' in the current
namespace, however.  Decorators could return a magic value that meant
"don't create a binding for the function", but that seems only useful
for this one use case.  Or maybe the decorator would record the name
internally, and after defining all the methods you'd call a cleanup
function that deleted them from the module namespace.

--amk
From python at rcn.com  Fri Jul 16 03:43:32 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Jul 16 15:44:48 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	compile.c, 2.306, 2.307
In-Reply-To: <20040716131811.GA27072@rogue.amk.ca>
Message-ID: <004601c46ad6$4d786d20$bb19c797@oemcomputer>

> Python 2.4a1+ (#13, Jul 16 2004, 09:11:38)
> [GCC 3.3.2 (Debian)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> None = 0
> <stdin>:1: SyntaxWarning: assignment to None
> >>> print None   # But assignment has no effect... perhaps due to this
> optim.?
> None
> >>> def f():
> ...    None = 0
> ...
> <stdin>:2: SyntaxWarning: assignment to None
> >>> f()
> 
> My CVS tree is up-to-date, and I did a "make clean".  Is everyone else
> getting the SyntaxError for this?


Hmm, it looks like I got burnt by IDLE which is converting the
SyntaxWarning into a SyntaxError:

Python 2.4a1 (#54, Jul  8 2004, 11:30:13) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> None = 0
<stdin>:1: SyntaxWarning: assignment to None


Python 2.4a1 (#54, Jul  8 2004, 11:30:13) [MSC v.1310 32 bit (Intel)] on
win32
Type "copyright", "credits" or "license()" for more information.
IDLE 1.1a0      
>>> None = 0
SyntaxError: assignment to None (<pyshell#0>, line 1)


That raises the question.  Since there was a SyntaxWarning in Py2.3,
should it have been escalated to an error for Py2.4?



Raymond

From amk at amk.ca  Fri Jul 16 15:52:56 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Fri Jul 16 15:53:52 2004
Subject: [Python-Dev] Constancy of None
In-Reply-To: <004601c46ad6$4d786d20$bb19c797@oemcomputer>
References: <20040716131811.GA27072@rogue.amk.ca>
	<004601c46ad6$4d786d20$bb19c797@oemcomputer>
Message-ID: <20040716135256.GA27286@rogue.amk.ca>

On Thu, Jul 15, 2004 at 09:43:32PM -0400, Raymond Hettinger wrote:
> That raises the question.  Since there was a SyntaxWarning in Py2.3,
> should it have been escalated to an error for Py2.4?

+1.

(And is it time to make assignments to True and False trigger a
SyntaxWarning, or should that wait until 2.5?)

--amk
From mwh at python.net  Fri Jul 16 15:57:11 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Jul 16 15:57:13 2004
Subject: [Python-Dev] Adding directories to configure.in for fink?
In-Reply-To: <40F76FC1.3040908@ocf.berkeley.edu> (Brett Cannon's message of
	"Thu, 15 Jul 2004 23:03:45 -0700")
References: <40F76FC1.3040908@ocf.berkeley.edu>
Message-ID: <2m7jt4ukaw.fsf@starship.python.net>

Brett Cannon <bac@OCF.Berkeley.EDU> writes:

> I discovered that when compiling for readline against a Fink install a
> warning was spit out about the lack of a function definition.  Turned
> out a function that #ifdef'ed a macro was not being triggered because
> configure.in was not catching the header file in /sw/lib for readline.
>
> Since /sw is already looked at by setup.py, is there any way to have
> configure.in do it as well so as to have some consistency?  Or is
> prepending ``LDFLAGS=/sw/lib`` the only way?  If so, does it warrant
> mentioning in the README?

If you take /sw/lib out of setup.py and then run

$ LDFLAGS=-L/sw/lib ./configure

does setup.py still find goodies in /sw?  If not, it should <wink>.

I'm definitely in the 'if I want fink I'll ask for it' camp (and 99%
of the time I don't and won't).  Every time I check something into
Python I have to be careful to not check in my local changes to
setup.py that comment this stuff out...

Cheers,
mwh

-- 
  ARTHUR:  Why should a rock hum?
    FORD:  Maybe it feels good about being a rock.
                    -- The Hitch-Hikers Guide to the Galaxy, Episode 8
From mwh at python.net  Fri Jul 16 16:04:20 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Jul 16 16:04:27 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <Pine.LNX.4.44.0407151039150.20090-100000@ccc2.wpi.edu>
	(Christopher
	T. King's message of "Thu, 15 Jul 2004 10:55:23 -0400 (EDT)")
References: <Pine.LNX.4.44.0407151039150.20090-100000@ccc2.wpi.edu>
Message-ID: <2m3c3sujyz.fsf@starship.python.net>

Christopher T King <squirrel@WPI.EDU> writes:

> On Thu, 15 Jul 2004, Guido van Rossum wrote:
>
>> > How about: Tail recursion "enables" recursion-oriented (functional)
>> > style? :)
>> 
>> Still -1.  I was understating my case: I find the "recursion is the
>> basis of everything" attitude harmful.
>
> I think it's still best to "enable" such a style in some cases -- Python's 
> syntax is designed to allow formulation of a problem in the most natural 
> way.  Sometimes it's more natural to formulate a problem as a recursive 
> function than a procedural one, but the current implementation restricts 
> this.

I think it's pretty hard to come up with a problem where the natural
solution is tail recursive.  There are problems where the natural
solution is *recursive*, sure -- but if the natural recursive solution
is tail recursive, I'd expect the iterative solution to be even more
natural.

Cheers,
mwh

-- 
  NUTRIMAT:  That drink was individually tailored to meet your
             personal requirements for nutrition and pleasure.
    ARTHUR:  Ah.  So I'm a masochist on a diet am I?
                    -- The Hitch-Hikers Guide to the Galaxy, Episode 9
From mwh at python.net  Fri Jul 16 16:08:25 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Jul 16 16:08:29 2004
Subject: [Python-Dev] Assignment to None
In-Reply-To: <000201c46a46$b327c700$582acb97@oemcomputer> (Raymond
	Hettinger's message of "Thu, 15 Jul 2004 04:35:34 -0400")
References: <000201c46a46$b327c700$582acb97@oemcomputer>
Message-ID: <2mr7rct57q.fsf@starship.python.net>

"Raymond Hettinger" <raymond.hettinger@verizon.net> writes:

<something in multipart MIME>

Raymond!  *PLEASE* stop doing that!

Cheers,
mwh

-- 
  There are 'infinite' number of developed artifacts and one cannot
  develop appreciation for them all.  It would be all right to not
  understand something, but it would be imbecilic to put judgements
  on things one don't understand.               -- Xah, comp.lang.lisp
From tim.peters at gmail.com  Fri Jul 16 16:08:27 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Jul 16 16:08:34 2004
Subject: [Python-Dev] Re: Proposal: C API Macro to decref and set to NULL
In-Reply-To: <un020vaeo.fsf@boost-consulting.com>
References: <40F300DB.6050106@zope.com>
	<1f7befae04071215506b00d40c@mail.gmail.com>
	<u7jt4yhs5.fsf@boost-consulting.com>
	<1f7befae040715170142971b48@mail.gmail.com>
	<200407160237.i6G2bLJ31180@guido.python.org>
	<un020vaeo.fsf@boost-consulting.com>
Message-ID: <1f7befae04071607082d56c798@mail.gmail.com>

[David Abrahams]
> That's exactly the sort of case that covers up bugs in C++.
> 
> Our operator delete basically just ignores NULL values.  When people
> write
> 
>   delete p; p = 0;
> 
> It can cover up bugs where p is unintentionally getting deleted
> twice.  It might still not be applicable here, though.

Understood.  Py_CLEAR isn't about deallocation, it's about a container
releasing *a* reference it owns to a containee.  The container doesn't
know, and doesn't care, whether it's releasing the last reference to
the containee.  Once a container no longer refers to a containee,
that's all the NULL means -- "I no longer refer to anything here", not
necessarily "the thing I referenced is no longer referenced by anyone
and has been recycled".  The former containee *may* get recycled at
this point, and it's important for the container to record that it no
longer references it before any destructors associated with the
containee get executed.  This is harmless.  But it's something people
often do in a wrong order, which is why it's a Good Idea to have a
macro capturing the right order.  A wrong order can lead to horridly
subtle bugs.
From ark-mlist at att.net  Fri Jul 16 16:11:52 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Fri Jul 16 16:11:56 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <2m3c3sujyz.fsf@starship.python.net>
Message-ID: <004a01c46b3e$db08d4f0$6602a8c0@arkdesktop>

> I think it's pretty hard to come up with a problem where the natural
> solution is tail recursive.

I might agree with you that it isn't easy to come up with a problem where
the natural problem is *entirely* tail recursive -- but my original
tree-traversal example is tail-recursive and I think it's more natural to
write it that way than it is to eliminate the tail recursion (leaving the
other recursion) by hand.

I've also seen examples of functions that begin like this:

	def foo(x, y):
		if x < 0:
			return foo(-x, -y)
		...

which I think is clearer than

	def foo(x, y)
		if x < 0:
			x = -x
			y = -y
		...

especially if there are several such cases to consider.  Of course there
isn't much to gain by removing this particular tail recursion, but
nevertheless it is a tail recursion that I think is more natural to leave in
place.

From barry at python.org  Fri Jul 16 16:14:42 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Jul 16 16:14:40 2004
Subject: [Python-Dev] Prototypes  [was: Proper tail recursion]
In-Reply-To: <20040716133314.GA27141@rogue.amk.ca>
References: <40F71CA0.5030904@stackless.com>
	<004001c46ad3$5a1b5180$bb19c797@oemcomputer>
	<20040716133314.GA27141@rogue.amk.ca>
Message-ID: <1089987282.8409.157.camel@localhost>

On Fri, 2004-07-16 at 09:33, A.M. Kuchling wrote:
> eeOn Thu, Jul 15, 2004 at 09:22:25PM -0400, Raymond Hettinger wrote:
> > helping out people who want to experiment with prototype OO.  The
> > request is make it possible to write:
> > 
> >     def Account.deposit(self, v):
> >         self.balance += v

I've wanted this for a long time, but I don't think Guido's ever been in
favor of this.

> Could function decorators be used for this?  Something like:

Possibly so, and that's kind of neat.

> @attach_to_object(Account)
> def deposit (self, v):
>     ...
> del deposit
> 
> The decorator couldn't delete the binding of 'deposit' in the current
> namespace, however.

Couldn't you play sys._getframe() tricks inside the decorator to do
this?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040716/da2d4985/attachment.pgp
From ark-mlist at att.net  Fri Jul 16 16:15:22 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Fri Jul 16 16:15:32 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <e8bf7a53040716045044c8170d@mail.gmail.com>
Message-ID: <004b01c46b3f$5b6872e0$6602a8c0@arkdesktop>

> > My dig was at the Schemers who would rather implement an array as a
> > linked list and then optimize tail-recursion in the hope of getting
> > some performance back -- in vain because they still can't get O(1)
> > indexing.

> In all fairness, a serious Schemer would probabliy use the builtin
> vector type for an array rather than cons cells.

People who insist on using linked lists when they want fast random-access
indexing are weird.  But my experience is that random-access indexing isn't
nearly as useful as people seem to think it is, and that many people use
arrays as pure sequences.  Not all the time, of course, but often.

From ark-mlist at att.net  Fri Jul 16 16:18:20 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Fri Jul 16 16:18:15 2004
Subject: [Python-Dev] Re: Proper tail recursion
In-Reply-To: <eppstein-1543D6.22365415072004@sea.gmane.org>
Message-ID: <005201c46b3f$bf736920$6602a8c0@arkdesktop>

> Where the recursion limit really bites me is the inability to do
> recursive depth first search on big graphs.  Of course, I can simulate
> the stack myself and write the rest iteratively, but if I wanted to do
> that I'd go back to writing in assembly language.

+1.

If you think you don't care about recursive depth-first searches on big
graphs, think again: Pickling is an example of such an algorithm.  I haven't
looked at how it's actually implemented, but it seems to me that either the
implementation must simulate recursion manually or it will fail on linked
data structures that are bigger than the recursion limit.  Even if the
standard library takes care to avoid unbounded recursion, anyone else who
wants to traverse data structures in similar ways faces similar problems.

From ark-mlist at att.net  Fri Jul 16 16:20:04 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Fri Jul 16 16:19:59 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <5.1.1.6.0.20040716000900.01e9bd40@mail.telecommunity.com>
Message-ID: <005301c46b3f$fe3da120$6602a8c0@arkdesktop>

 
> Yeah, but it's a *useful* kludge to have a recursion limit.  Most
> algorithms that are "sensibly" recursive have some fan-out at each
> recursion level, such that the total recursion needed is something like
> log2N.  So as N grows, the relative amount of recursion shrinks.

Not if the data structure is lopsided -- perhaps intentionally so.  Remember
the typical usage of a Lisp list, which is really a binary tree in which the
depth of each left-hand subtree is usually very small (typically 1).

From Jack.Jansen at cwi.nl  Fri Jul 16 16:23:20 2004
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Fri Jul 16 16:22:38 2004
Subject: [Python-Dev] Adding directories to configure.in for fink?
In-Reply-To: <878ydks3qk.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <40F76FC1.3040908@ocf.berkeley.edu>
	<06559C87-D701-11D8-A243-000A958D1666@cwi.nl>
	<878ydks3qk.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <B00CF432-D733-11D8-A243-000A958D1666@cwi.nl>


On 16-jul-04, at 11:25, Stephen J. Turnbull wrote:
> If you're going to do this for Fink, don't you want to do it for
> DarwinPorts too?  XEmacs's configure has a --site-prefixes switch such
> that --site-prefixes=/sw does both --site-includes=/sw/include and
> --site-libraries=/sw/lib.  This handles both Fink and DarwinPorts (and
> anythign else that might come along) in a sensible way.  Isn't that a
> generic autoconf feature?

This would be very nice to have. How much work would it be to implement 
it?
We don't currently support site-includes and site-libraries either (I 
think) so that
would need to be added too.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman

From tanzer at swing.co.at  Thu Jul 15 13:08:14 2004
From: tanzer at swing.co.at (tanzer@swing.co.at)
Date: Fri Jul 16 16:30:41 2004
Subject: [Python-Dev] RELEASED Python 2.4, alpha 1
In-Reply-To: Your message of "Fri, 09 Jul 2004 17:11:34 +1000."
	<40EE4526.5060306@python.org>
Message-ID: <E1Bl462-0007dg-00@swing.co.at>

In February, I reported a doctest bug against 2.3.3 (request id
902628) and attached a patch to fix the problem.

2.4 alpha 1 still shows this behavior:

Traceback (most recent call last):
  File "/tmp/confuse_doctest.py", line 14, in ?
    import doctest, confuse_doctest
  File "/tmp/confuse_doctest.py", line 15, in ?
    doctest.testmod(confuse_doctest)
  File "/usr/local/lib/python2.4/doctest.py", line 1155, in testmod
    f, t = tester.rundict(m.__dict__, name, m)
  File "/usr/local/lib/python2.4/doctest.py", line 915, in rundict
    f2, t2 = self.__runone(value, name + "." + thisname)
  File "/usr/local/lib/python2.4/doctest.py", line 1076, in __runone
    return self.rundoc(target, name)
  File "/usr/local/lib/python2.4/doctest.py", line 836, in rundoc
    f2, t2 = self.run__test__(d, name)
  File "/usr/local/lib/python2.4/doctest.py", line 944, in run__test__
    raise TypeError("Tester.run__test__: values in "
TypeError: Tester.run__test__: values in dict must be strings, functions, methods, or classes; <super: <class 'A'>, NULL>

What can I do to help getting this fixed?

-- 
Christian Tanzer                                    http://www.c-tanzer.at/

From paul at prescod.net  Fri Jul 16 16:31:13 2004
From: paul at prescod.net (Paul Prescod)
Date: Fri Jul 16 16:30:54 2004
Subject: [Python-Dev] Re: Proper tail recursion
In-Reply-To: <005201c46b3f$bf736920$6602a8c0@arkdesktop>
References: <005201c46b3f$bf736920$6602a8c0@arkdesktop>
Message-ID: <40F7E6B1.90108@prescod.net>

Andrew Koenig wrote:
>>Where the recursion limit really bites me is the inability to do
>>recursive depth first search on big graphs.  Of course, I can simulate
>>the stack myself and write the rest iteratively, but if I wanted to do
>>that I'd go back to writing in assembly language.
> 
> 
> +1.
> 
> If you think you don't care about recursive depth-first searches on big
> graphs, think again: Pickling is an example of such an algorithm.  I haven't
> looked at how it's actually implemented, but it seems to me that either the
> implementation must simulate recursion manually or it will fail on linked
> data structures that are bigger than the recursion limit. 

http://mail.python.org/pipermail/python-list/2003-November/193953.html
http://mail.python.org/pipermail/python-list/2003-November/193956.html

  Paul Prescod

From amk at amk.ca  Fri Jul 16 16:35:02 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Fri Jul 16 16:35:57 2004
Subject: [Python-Dev] Prototypes  [was: Proper tail recursion]
In-Reply-To: <1089987282.8409.157.camel@localhost>
References: <40F71CA0.5030904@stackless.com>
	<004001c46ad3$5a1b5180$bb19c797@oemcomputer>
	<20040716133314.GA27141@rogue.amk.ca>
	<1089987282.8409.157.camel@localhost>
Message-ID: <20040716143502.GA27617@rogue.amk.ca>

On Fri, Jul 16, 2004 at 10:14:42AM -0400, Barry Warsaw wrote:
> Couldn't you play sys._getframe() tricks inside the decorator to do
> this?

When the decorator code is running, I'll bet the function hasn't
been bound to its name yet, so the binding can't be deleted.  

--amk
From python at rcn.com  Fri Jul 16 04:43:27 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Jul 16 16:44:46 2004
Subject: [Python-Dev] Prototypes  [was: Proper tail recursion]
In-Reply-To: <1089987282.8409.157.camel@localhost>
Message-ID: <001801c46ade$aea43ea0$c622c797@oemcomputer>

[Raymond]:
> > > helping out people who want to experiment with prototype OO.  The
> > > request is make it possible to write:
> > >
> > >     def Account.deposit(self, v):
> > >         self.balance += v

[Barry]
> I've wanted this for a long time, but I don't think Guido's ever been
in
> favor of this.

Every now and then he gives a little gift if it makes us happy, benefits
a class of users, and doesn't muck up anything else. 



[AMK]
> > Could function decorators be used for this?

[Barry]
> Possibly so, and that's kind of neat.

Perhaps, but why make something hard out of something simple.  Regular
assignments show that no special magic is required:  acct.bal = 0.  The
def statement could benefit from the same courtesy.



Raymond

From ark-mlist at att.net  Fri Jul 16 16:47:55 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Fri Jul 16 16:47:49 2004
Subject: [Python-Dev] Re: Proper tail recursion
In-Reply-To: <40F7E6B1.90108@prescod.net>
Message-ID: <005401c46b43$e130f740$6602a8c0@arkdesktop>

> http://mail.python.org/pipermail/python-list/2003-November/193953.html
> http://mail.python.org/pipermail/python-list/2003-November/193956.html

As an interesting example of writing such algorithms without explicit
recursion:  The garbage collector for Spitbol/360 uses a mark-and-sweep
algorithm, which it implements in such a way as to use O(1) auxiliary space,
including the stack!  It does do by using the following trick:

Every block of memory begins with a word that usually contains the block's
type.  The memory-management system requires that every pointer must point
to the beginning of a block (i.e. to the type word) -- there are no pointers
to memory within blocks.

The mark phase of the garbage collector replaces each type word with the
head of a singly linked through the pointers that originally pointed to that
block.  The tail of the list is the original type word with the low-order
bit turned on to indicate that it is the end.

When garbage collection is complete, then, every pointer that originally
pointed to a block is now either a copy of that block's original type word
(with the low-order bit turned on) or is a pointer to another pointer that
originally pointed to that block.  Every live block's type word is a pointer
to a pointer that originally pointed to it.  Every dead block's type word
retains its original value.

The sweep phase is done in two passes.  The first one goes through every
block.  If the block is dead, it does nothing.  If it is live, it goes
through the chain of pointers that are now linked to that block, setting
every pointer to the address that the block will occupy *after* the second
pass is complete.  Once it has done this, it can restore the block's type
word.

The second pass of the sweep phase relocates every live block to its final
position.  There is no need to change any pointers in this pass, because the
first pass changed them all.

The result is that all live blocks are now contiguous, and all pointers
point to the blocks' new location.  There is also the nice side effect that
all live blocks' addresses remain in the same relative order, a property
that is used by other parts of the compiler.

I mention this rather long example to show that it is possible to eliminate
recursion even in fairly complicated algorithms, but that you may not
actually want to do it unless you're Robert Dewar :-)

From barry at python.org  Fri Jul 16 17:18:21 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Jul 16 17:18:19 2004
Subject: [Python-Dev] Prototypes  [was: Proper tail recursion]
In-Reply-To: <001801c46ade$aea43ea0$c622c797@oemcomputer>
References: <001801c46ade$aea43ea0$c622c797@oemcomputer>
Message-ID: <1089991101.8409.166.camel@localhost>

On Thu, 2004-07-15 at 22:43, Raymond Hettinger wrote:

> Perhaps, but why make something hard out of something simple.  Regular
> assignments show that no special magic is required:  acct.bal = 0.  The
> def statement could benefit from the same courtesy.

You're preaching to the choir here!

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040716/cb3a5836/attachment.pgp
From pje at telecommunity.com  Fri Jul 16 18:15:00 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Jul 16 18:11:36 2004
Subject: [Python-Dev] Prototypes  [was: Proper tail recursion]
In-Reply-To: <004001c46ad3$5a1b5180$bb19c797@oemcomputer>
References: <40F71CA0.5030904@stackless.com>
Message-ID: <5.1.1.6.0.20040716120310.02a26cb0@mail.telecommunity.com>

At 09:22 PM 7/15/04 -0400, Raymond Hettinger wrote:
>The reason is the feature request is currently out of favor is the
>possibility that the syntax would be abused and people would start
>defining their methods outside of class definitions.

That sounds like a feature to me.  :)  I've just finished drafting an 
extensible generic function framework using decorators, like this stupid 
trivial non-motivating example:

     [when("x<2")]
     def fact(x): return 1

     [when("True")]
     def fact(x): return x * fact(x-1)

Generic functions are a great way to structure an extensible framework, 
where the base function is defined in one module, and then other modules 
add cases to the function, like:

     from storage_framework import save_ob

     [when("ob in Invoice and db in Oracle")]
     def save_ob(db, ob):
         # code to write Invoice to Oracle

So, with your syntax, you could instead do:

     import storage_framework

     [when("ob in Invoice and db in Oracle")]
     def storage_framework.save_ob(db, ob):
         # code to write Invoice to Oracle

which is much clearer as to intent, although I must confess I have no clue 
how I'd make it work with my current decorator implemenation, which looks 
for changes in the frame's locals.

As for declaring methods outside a class, this is actually a good idea for 
certain generic functions.  Say for example that 'save_ob' was actually a 
method of the database type, rather than being a "free range" generic 
function.  Then, the above would be written as:

     [when("ob in Invoice")]
     def Oracle.save_ob(self, ob):
         # code to write Invoice to Oracle

which is even clearer as to intention.

But, I think that there'd have to be integrated decorator support for it to 
be really useful, at least for me.

From guido at python.org  Fri Jul 16 18:19:07 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Jul 16 18:19:12 2004
Subject: [Python-Dev] Constancy of None
In-Reply-To: Your message of "Fri, 16 Jul 2004 09:52:56 EDT."
	<20040716135256.GA27286@rogue.amk.ca> 
References: <20040716131811.GA27072@rogue.amk.ca>
	<004601c46ad6$4d786d20$bb19c797@oemcomputer> 
	<20040716135256.GA27286@rogue.amk.ca> 
Message-ID: <200407161619.i6GGJ7402512@guido.python.org>

> (And is it time to make assignments to True and False trigger a
> SyntaxWarning, or should that wait until 2.5?)

We should be much more conservative with that.  There's tons of code
out there that assigns these (conditionally, if it's 2.3-aware) to
truth values and we don't want to break that.  The situation with None
is much better, since it was always there; AFAIK only a few speed
freaks in Zope-land have ever assigned to it (and always using the
form "None = None").

--Guido van Rossum (home page: http://www.python.org/~guido/)
From mcherm at mcherm.com  Fri Jul 16 18:22:38 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Fri Jul 16 18:22:40 2004
Subject: [Python-Dev] Planned changes to logging
Message-ID: <1089994958.40f800cecf88b@mcherm.com>

> Better still, it seems entirely consistent with these semantics to
> have the existing log objects reset to their parent settings -- just
> like unspecified log objects created after the confguration load do.
> I think this is more consistent, more intuitive, and doesn't require
> you to figure out what logs may or exist throughout the system when
> loading a configuration.

+1 (based on my experiences with java.logging, not with the Python
    module)

-- Michael Chermside

From skip at pobox.com  Fri Jul 16 18:25:44 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Jul 16 18:25:54 2004
Subject: [Python-Dev] RELEASED Python 2.4, alpha 1
In-Reply-To: <E1Bl462-0007dg-00@swing.co.at>
References: <40EE4526.5060306@python.org> <E1Bl462-0007dg-00@swing.co.at>
Message-ID: <16632.392.703114.742388@montanaro.dyndns.org>


    Christian> In February, I reported a doctest bug against 2.3.3 (request
    Christian> id 902628) and attached a patch to fix the problem.

    Christian> 2.4 alpha 1 still shows this behavior:

    ...

    Christian> TypeError: Tester.run__test__: values in dict must be strings, functions, methods, or classes; <super: <class 'A'>, NULL>

    Christian> What can I do to help getting this fixed?

Hey Christian...  Andrew Kuchling has been holding bug days recently, where
a group of folks get on irc and coordinate activity whittling down the
bug/patch backlog.  There was one last weekend.  Some python-dev folks
besides Andrew have participated (Brett Cannon comes to mind).  If you can
participate or at least drop one of them a note the day before the next one
you can probably convince one of them to look at it then.  I don't recall
when the next one is, probably around the release of 2.4a2.  Check c.l.py or
c.l.py.a archives for announcements.

Skip

From walter at livinglogic.de  Fri Jul 16 18:36:19 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Fri Jul 16 18:36:23 2004
Subject: [Python-Dev] bdist_wininst
In-Reply-To: <8ydkuudh.fsf@python.net>
References: <8ydkuudh.fsf@python.net>
Message-ID: <40F80403.9030904@livinglogic.de>

Thomas Heller wrote:

> bdist_wininst has seen a lot of bug fixes and improvements, mostly by
> Mark, partly by me.
> 
> I'd like to 'improve' the visual appearance, too.

One improvement would be proper treatment of Unicode. Currently it
doesn't seem to be possible to use unicode objects with umlauts in the
arguments to distutils.core.setup(). Some of my Windows packages say
"Author: Walter Dxf6rwald"

Bye,
    Walter D?rwald


From theller at python.net  Fri Jul 16 18:54:02 2004
From: theller at python.net (Thomas Heller)
Date: Fri Jul 16 18:54:18 2004
Subject: [Python-Dev] bdist_wininst
In-Reply-To: <40F80403.9030904@livinglogic.de> (
	=?iso-8859-1?q?Walter_D=F6rwald's_message_of?= "Fri,
	16 Jul 2004 18:36:19 +0200")
References: <8ydkuudh.fsf@python.net> <40F80403.9030904@livinglogic.de>
Message-ID: <r7rbsxjp.fsf@python.net>

Walter D?rwald <walter@livinglogic.de> writes:

> Thomas Heller wrote:
>
>> bdist_wininst has seen a lot of bug fixes and improvements, mostly by
>> Mark, partly by me.
>> I'd like to 'improve' the visual appearance, too.
>
> One improvement would be proper treatment of Unicode. Currently it
> doesn't seem to be possible to use unicode objects with umlauts in the
> arguments to distutils.core.setup(). Some of my Windows packages say
> "Author: Walter Dxf6rwald"

Can you point me to some of your setup scripts?

Thanks,

Thomas

From walter at livinglogic.de  Fri Jul 16 18:57:43 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Fri Jul 16 18:57:45 2004
Subject: [Python-Dev] bdist_wininst
In-Reply-To: <r7rbsxjp.fsf@python.net>
References: <8ydkuudh.fsf@python.net> <40F80403.9030904@livinglogic.de>
	<r7rbsxjp.fsf@python.net>
Message-ID: <40F80907.7040606@livinglogic.de>

Thomas Heller wrote:

> Walter D?rwald <walter@livinglogic.de> writes:
> 
> 
>>Thomas Heller wrote:
>>
>>
>>>bdist_wininst has seen a lot of bug fixes and improvements, mostly by
>>>Mark, partly by me.
>>>I'd like to 'improve' the visual appearance, too.
>>
>>One improvement would be proper treatment of Unicode. Currently it
>>doesn't seem to be possible to use unicode objects with umlauts in the
>>arguments to distutils.core.setup(). Some of my Windows packages say
>>"Author: Walter Dxf6rwald"
> 
> 
> Can you point me to some of your setup scripts?

http://www.livinglogic.de/viewcvs/index.cgi/LivingLogic/Python/ansistyle/setup.py?rev=1.15&content-type=text/vnd.viewcvs-markup

changing the strs to unicode objects doesn't fix the problem.

Bye,
    Walter D?rwald


From bac at OCF.Berkeley.EDU  Fri Jul 16 19:32:31 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Jul 16 19:32:45 2004
Subject: [Python-Dev] Adding directories to configure.in for fink?
In-Reply-To: <878ydks3qk.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <40F76FC1.3040908@ocf.berkeley.edu>	<06559C87-D701-11D8-A243-000A958D1666@cwi.nl>
	<878ydks3qk.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <40F8112F.40107@ocf.berkeley.edu>

Stephen J. Turnbull wrote:

> If you're going to do this for Fink, don't you want to do it for
> DarwinPorts too?  XEmacs's configure has a --site-prefixes switch such
> that --site-prefixes=/sw does both --site-includes=/sw/include and
> --site-libraries=/sw/lib.  This handles both Fink and DarwinPorts (and
> anythign else that might come along) in a sensible way.  Isn't that a
> generic autoconf feature?
> 

I would love to have this as well.  And chances are that people using 
Fink would know how to use this option properly.

And if there was a way to have setup.py be able to get at this 
information that could help eliminate the hard-coded search paths it uses.

And if we could take this even further and have the stuff configure.in 
is passed and discovers available to Fink we could customize the install 
even further.  I bring this up because if you do a 
--without-toolbox-glue build on OS X you get a ton of failed links of 
various modules that want Carbon stuff that they don't get since they 
were turned off.  If that flag was available in setup.py then we could 
just add a conditional making sure that the flag was not flipped to off.

-Brett
From barry at python.org  Fri Jul 16 19:33:10 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Jul 16 19:33:09 2004
Subject: [Python-Dev] Constancy of None
In-Reply-To: <200407161619.i6GGJ7402512@guido.python.org>
References: <20040716131811.GA27072@rogue.amk.ca>
	<004601c46ad6$4d786d20$bb19c797@oemcomputer>
	<20040716135256.GA27286@rogue.amk.ca>
	<200407161619.i6GGJ7402512@guido.python.org>
Message-ID: <1089999190.8408.188.camel@localhost>

On Fri, 2004-07-16 at 12:19, Guido van Rossum wrote:
> > (And is it time to make assignments to True and False trigger a
> > SyntaxWarning, or should that wait until 2.5?)
> 
> We should be much more conservative with that

Agreed!

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040716/ec0eee95/attachment.pgp
From tismer at stackless.com  Fri Jul 16 19:51:02 2004
From: tismer at stackless.com (Christian Tismer)
Date: Fri Jul 16 19:50:13 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <200407160235.i6G2ZOm31140@guido.python.org>
References: <1089894294.40f677961fc20@login.werra.lunarpages.com>
	<1089894725.40f67945bc06d@login.werra.lunarpages.com>
	<877e9a17040715063463791eb6@mail.gmail.com>
	<200407151434.i6FEYkt30174@guido.python.org>
	<40F72026.6030207@stackless.com>
	<200407160235.i6G2ZOm31140@guido.python.org>
Message-ID: <40F81586.7030505@stackless.com>

Guido van Rossum wrote:

>>>Still -1.  I was understating my case: I find the "recursion is
>>>the basis of everything" attitude harmful.
>>
>>How can you then still implement ceval.c in a recursive fashion,
>>since ceval is the "basis of everything" ???

[and more such crap]

> Oh Christian, did you have a bad day again?

Yes, I had. Negotiations with the EU guys concerning
PyPy funding really puts me down. It moves on and on.

Please ignore the nonsense I wrote. I hope to avoid incidents
like these, but sometimes... ;-)

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From jcarlson at uci.edu  Fri Jul 16 19:45:38 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Fri Jul 16 19:51:41 2004
Subject: [Python-Dev] Re: Proper tail recursion
In-Reply-To: <875c7e0704071606231c8f5795@mail.gmail.com>
References: <20040715235400.6990.JCARLSON@uci.edu>
	<875c7e0704071606231c8f5795@mail.gmail.com>
Message-ID: <20040716094428.4EC1.JCARLSON@uci.edu>


On Fri, 16 Jul 2004 09:23:15 -0400, Chris King <colanderman@gmail.com> wrote:

> On Fri, 16 Jul 2004 00:52:07 -0700, Josiah Carlson <jcarlson@uci.edu> wrote:
> 
> > If unlimited recursion were allowed, perhaps limited tracebacks (first
> > and last 500 stack frame traces), RLE tracebacks (though a clever
> > programmer could easily destroy the generator with fewer than 64
> > functions, but we probably shouldn't do that), or combined limited and
> > RLE tracebacks would be sufficient. What do I mean by an RLE traceback?
> 
> I thought of doing RLE tracebacks, but compression fails in the case
> of cooperative recursive calls.  I think perhaps along with the
> sys.setrecursionlimit(None) option you suggest, developers should be
> allowed to turn recursive tracebacks off entirely in the case of
> cooperative recursive calls.

They already can by writing your own sys.excepthook callable. I think
the following would be sufficient for removing tracebacks...

def myexcepthook(type, value, traceback):
    print >>sys.stderr, "Exception:", type.__name__, ":", value


> The other problem with RLE tracebacks is that a traceback object also
> keeps a reference to each function's locals (by virtue of keeping a
> reference to its frame obejct).  Throwing this info out makes RLE
> tracebacks no more useful to debuggers than having no traceback at
> all.

Which is precisely why (in general) frames should not be tossed.  Even a
partial traceback is better than no traceback.


> Keeping the first and last X frames in a traceback seems reasonable,
> but this would similarly cripple debuggers (what happens if the bug is
> in the (X+1)th frame?).  Implementation would also be complicated.

Impementation is 99% done in the traceback module.

As for bugs appearing in frame X+1, I'm not concerned.  20, 30, 40
levels of traceback, yeah, I (and others I'm sure) have gone through
those.  But as soon as you start hitting 500 levels of traceback, all
but the first and last few dozen are noise.

Honestly, I'd like to see a real application that has a bug only in
recursion level 501.  However, I'm not sure one would ever come to exist
if the programmer used any sort of reasonable modern development methods
(test-driven, design by contract, etc.), and were remotely competant.

To me, the point of limiting tracebacks to 1000 calls was to reduce
and/or remove the liklihood of someone getting a multi-meg email via
logging.SMTPHandler, or filling up their filesystem with a
logging.FileHandler.


> IMHO it should be an all-or-nothing deal: either the programmer turns
> tail-call optimizations on to nullify memory uses, or turns them off
> to facilitate debugging.

IMO it shouldn't be only about tail-call optimizations.  Andrew Koenig
suggested that frames be allocated from the heap, which if it is
possible (is there anything that gets executed directly by the processor
so we have to worry about processors supporting NX?), seems to remove
the C stack limit.

In the case of tail-calls, more optimizations could be done (optionally),
to remove the overhead on tail calls (also removing information
containing tracebacks), but the user should be warned that while it
would reduce memory usage, all they get from the traceback are lines are
like so:
***recursive tail call frames removed as requested***

Now, because it seems as though one could easily mix regular and tail
calls, the following traceback seems reasonable (if optional tail call
optimizations were done)...

Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "<stdin>", line 3, in g
  File "<stdin>", line 3, in h
***recursive tail call frames removed as requested***
  File "<stdin>", line 3, in i
...


 - Josiah

From pje at telecommunity.com  Fri Jul 16 19:56:09 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Jul 16 19:52:44 2004
Subject: [Python-Dev] "as" to be a keyword?
Message-ID: <5.1.1.6.0.20040716135337.01ec94c0@mail.telecommunity.com>

Is it true that 'as' is going to become a keyword in future?  I ask, 
because I  was thinking about implementing an 'as' function, such that:

     [as(classmethod)]
     def foo(...):
         ...

will work for Python 2.2 and up.

Actually, I've already written and tested the function itself, it's just 
that I would like to be sure that I shouldn't try to come up with another 
name.  :)

From nidoizo at yahoo.com  Fri Jul 16 19:55:59 2004
From: nidoizo at yahoo.com (Nicolas Fleury)
Date: Fri Jul 16 19:56:12 2004
Subject: [Python-Dev] Re: Absolute/Relative import switcher for packages
	(PEP 328)
In-Reply-To: <04ed01c46aa7$980a6fc0$0201010a@kret>
References: <04ed01c46aa7$980a6fc0$0201010a@kret>
Message-ID: <cd94rj$ebf$1@sea.gmane.org>

Wiktor Sadowski wrote:
> FWIW:
> Here is the solution for  Absolute/Relative problem that we are using
> in our modified Python distribution...
> 
> We have added a few lines of code to to get_parent in import.c.
> The code checks if there is "__absimport__=True" in parent dict
> and if so: absolute import is used in "child" module (get_parent returns
> Py_None)
> 
> Then putting "__absimport__=True" into an __init__.py is enough to
> force absolute import for the whole package.
> (The "__absimport__=True" switcher should be put into each subpackage
> __init__.py,if desired)
> 
> Such addition is fully compatible with existing
> Python import syntax and easy to use.
> 
> Also:
> absolute import could be default,and "__relimport__=True"
> would switch to relative import.

Personally, I would like very much to have a way to specify as default 
absolute imports in packages, instead of in modules.  I would not mind 
if I have to specify it in all packages.  The environment where I work 
uses a lot modules in few packages.  A way to specify it in packages 
would ease our development until Python 2.6.

Regards,
Nicolas

From pje at telecommunity.com  Fri Jul 16 20:03:25 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Jul 16 20:00:02 2004
Subject: [Python-Dev] Re: Proper tail recursion
In-Reply-To: <20040716094428.4EC1.JCARLSON@uci.edu>
References: <875c7e0704071606231c8f5795@mail.gmail.com>
	<20040715235400.6990.JCARLSON@uci.edu>
	<875c7e0704071606231c8f5795@mail.gmail.com>
Message-ID: <5.1.1.6.0.20040716135901.026cbab0@mail.telecommunity.com>

At 10:45 AM 7/16/04 -0700, Josiah Carlson wrote:
>IMO it shouldn't be only about tail-call optimizations.  Andrew Koenig
>suggested that frames be allocated from the heap, which if it is
>possible (is there anything that gets executed directly by the processor
>so we have to worry about processors supporting NX?), seems to remove
>the C stack limit.

Frames *are* allocated on the heap.  That's not the problem.  The problem 
is that the interpreter *has* to use the C stack to call C functions, and 
Python objects are implemented using C functions.  Therefore, invoking 
operations on Python objects uses the C stack, in the general case.

(Which is why some versions of Stackless made extensive modifications to 
the Python core, in order to maximize the number of specific cases where C 
function calls could be avoided.)

From martin at v.loewis.de  Fri Jul 16 20:06:41 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Jul 16 20:06:44 2004
Subject: [Python-Dev] "as" to be a keyword?
In-Reply-To: <5.1.1.6.0.20040716135337.01ec94c0@mail.telecommunity.com>
References: <5.1.1.6.0.20040716135337.01ec94c0@mail.telecommunity.com>
Message-ID: <40F81931.3040007@v.loewis.de>

Phillip J. Eby wrote:
> Is it true that 'as' is going to become a keyword in future? 

I doubt it. If this was meant to happen, it would have happened
long ago.

Regards,
Martin

From tismer at stackless.com  Fri Jul 16 20:10:28 2004
From: tismer at stackless.com (Christian Tismer)
Date: Fri Jul 16 20:09:38 2004
Subject: WHUI! (was Re: [Python-Dev] Proposal: C API Macro to decref and
	set to NULL)
In-Reply-To: <200407160229.i6G2TVS31086@guido.python.org>
References: <40F300DB.6050106@zope.com>
	<1f7befae04071215506b00d40c@mail.gmail.com>
	<40F3BD4C.70701@zope.com>
	<1f7befae040713081023e68654@mail.gmail.com>
	<40F3FD9F.2040808@zope.com>
	<1f7befae040713083363a57a7b@mail.gmail.com>
	<40F70A70.7050303@stackless.com>
	<200407160229.i6G2TVS31086@guido.python.org>
Message-ID: <40F81A14.7030503@stackless.com>

Guido van Rossum wrote:

>>      if ( 0 ||
>>	       || VISIT(self->readline)
>>	       || VISIT(self->read)
>>	       || VISIT(self->file)
>>	    )
>>		return 1;
> 
> 
> Yuck.  Feels like double negatives, wheras the original felt "juuuuust
> right" :-)

I should have left out the "0 ||" which is nothing but
a way to format all relevant lines the same.
What I meant was just the or-ing together of the VISITs.

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From colanderman at gmail.com  Fri Jul 16 20:11:24 2004
From: colanderman at gmail.com (Chris King)
Date: Fri Jul 16 20:11:28 2004
Subject: [Python-Dev] Re: Proper tail recursion
In-Reply-To: <20040716094428.4EC1.JCARLSON@uci.edu>
References: <20040715235400.6990.JCARLSON@uci.edu>
	<875c7e0704071606231c8f5795@mail.gmail.com>
	<20040716094428.4EC1.JCARLSON@uci.edu>
Message-ID: <875c7e07040716111149c2018a@mail.gmail.com>

On Fri, 16 Jul 2004 10:45:38 -0700, Josiah Carlson <jcarlson@uci.edu> wrote:
> 
> On Fri, 16 Jul 2004 09:23:15 -0400, Chris King <colanderman@gmail.com> wrote:
> 
> > I thought of doing RLE tracebacks, but compression fails in the case
> > of cooperative recursive calls.  I think perhaps along with the
> > sys.setrecursionlimit(None) option you suggest, developers should be
> > allowed to turn recursive tracebacks off entirely in the case of
> > cooperative recursive calls.
> 
> They already can by writing your own sys.excepthook callable. I think
> the following would be sufficient for removing tracebacks...
> 
> def myexcepthook(type, value, traceback):
>     print >>sys.stderr, "Exception:", type.__name__, ":", value

I wasn't clear enough -- by "turn off" I meant "throw out"; redefining
sys.excepthook will still store them (and thus take up memory).

The problem isn't whether or not tracebacks should be cut -- it's
whether huge lists of recursively entered stack frames should be kept
around.  If they are kept (a necessity for a proper traceback),
optimizing tail calls is almost worthless; indeed, it would be
entirely worthless in Stackless, which already allocates frames from
the heap.

> IMO it shouldn't be only about tail-call optimizations.  Andrew Koenig
> suggested that frames be allocated from the heap, which if it is
> possible (is there anything that gets executed directly by the processor
> so we have to worry about processors supporting NX?), seems to remove
> the C stack limit.

The stack limit isn't the problem (it's very large, as far as I can
tell).  The memory used by holding onto stack frames that have no use
other than in tracebacks is the problem.

> In the case of tail-calls, more optimizations could be done (optionally),
> to remove the overhead on tail calls (also removing information
> containing tracebacks), but the user should be warned that while it
> would reduce memory usage, all they get from the traceback are lines are
> like so:
> ***recursive tail call frames removed as requested***

This is exactly what the current patch does (minus the warning
message): it removes the memory overhead by tossing out extraneous
stack frames, at the cost of the availability of proper traceback
information.  Adding a warning would be trivial.
From theller at python.net  Fri Jul 16 20:11:35 2004
From: theller at python.net (Thomas Heller)
Date: Fri Jul 16 20:11:53 2004
Subject: [Python-Dev] bdist_wininst
In-Reply-To: <40F80907.7040606@livinglogic.de> (
	=?iso-8859-1?q?Walter_D=F6rwald's_message_of?= "Fri,
	16 Jul 2004 18:57:43 +0200")
References: <8ydkuudh.fsf@python.net> <40F80403.9030904@livinglogic.de>
	<r7rbsxjp.fsf@python.net> <40F80907.7040606@livinglogic.de>
Message-ID: <iscnstyg.fsf@python.net>


>>>>bdist_wininst has seen a lot of bug fixes and improvements, mostly by
>>>>Mark, partly by me.
>>>>I'd like to 'improve' the visual appearance, too.
>>>
>>>One improvement would be proper treatment of Unicode. Currently it
>>>doesn't seem to be possible to use unicode objects with umlauts in the
>>>arguments to distutils.core.setup(). Some of my Windows packages say
>>>"Author: Walter Dxf6rwald"

Walter, can you try out the attached patch?  It seems to work for me.
Both for unicode and ascii strings as author name, containing an umlaut.

Thanks,

Thomas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: bdist_wininst.diff
Type: text/x-patch
Size: 4164 bytes
Desc: patch for distutils/command/bdist_wininst.py
Url : http://mail.python.org/pipermail/python-dev/attachments/20040716/f1c71415/bdist_wininst.bin
From theller at python.net  Fri Jul 16 20:15:51 2004
From: theller at python.net (Thomas Heller)
Date: Fri Jul 16 20:16:08 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/distutils
 __init__.py, 1.23, 1.24
In-Reply-To: <E1BlXEG-0000vQ-GF@sc8-pr-cvs1.sourceforge.net>
	(theller@users.sourceforge.net's
	message of "Fri, 16 Jul 2004 11:14:40 -0700")
References: <E1BlXEG-0000vQ-GF@sc8-pr-cvs1.sourceforge.net>
Message-ID: <d62vstrc.fsf@python.net>

theller@users.sourceforge.net writes:

> Update of /cvsroot/python/python/dist/src/Lib/distutils
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3551
>
> Modified Files:
> 	__init__.py 
> Log Message:
> The new distutils features justify a new version number, imo.
>
> If someone has other ideas for the numbering scheme, please change to
> something else (1.1.0 ?).

After I committed this, it ocurred to me: certainly 2.4.0 ?

Thomas

From jcarlson at uci.edu  Fri Jul 16 20:23:46 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Fri Jul 16 20:29:42 2004
Subject: [Python-Dev] Re: Proper tail recursion
In-Reply-To: <5.1.1.6.0.20040716135901.026cbab0@mail.telecommunity.com>
References: <20040716094428.4EC1.JCARLSON@uci.edu>
	<5.1.1.6.0.20040716135901.026cbab0@mail.telecommunity.com>
Message-ID: <20040716111713.4EC6.JCARLSON@uci.edu>


On Fri, 16 Jul 2004 14:03:25 -0400, "Phillip J. Eby" <pje@telecommunity.com> wrote:
> At 10:45 AM 7/16/04 -0700, Josiah Carlson wrote:
> >IMO it shouldn't be only about tail-call optimizations.  Andrew Koenig
> >suggested that frames be allocated from the heap, which if it is
> >possible (is there anything that gets executed directly by the processor
> >so we have to worry about processors supporting NX?), seems to remove
> >the C stack limit.
> 
> Frames *are* allocated on the heap.  That's not the problem.  The problem 
> is that the interpreter *has* to use the C stack to call C functions, and 
> Python objects are implemented using C functions.  Therefore, invoking 
> operations on Python objects uses the C stack, in the general case.

Ahh, now I understand.  Thank you for the clarification.  I'll go back
to lurking now.

 - Josiah

From walter at livinglogic.de  Fri Jul 16 22:04:32 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Fri Jul 16 22:04:37 2004
Subject: [Python-Dev] bdist_wininst
In-Reply-To: <iscnstyg.fsf@python.net>
References: <8ydkuudh.fsf@python.net>
	<40F80403.9030904@livinglogic.de>	<r7rbsxjp.fsf@python.net>
	<40F80907.7040606@livinglogic.de> <iscnstyg.fsf@python.net>
Message-ID: <40F834D0.50903@livinglogic.de>

Thomas Heller wrote:
>>>>>bdist_wininst has seen a lot of bug fixes and improvements, mostly by
>>>>>Mark, partly by me.
>>>>>I'd like to 'improve' the visual appearance, too.
>>>>
>>>>One improvement would be proper treatment of Unicode. Currently it
>>>>doesn't seem to be possible to use unicode objects with umlauts in the
>>>>arguments to distutils.core.setup(). Some of my Windows packages say
>>>>"Author: Walter Dxf6rwald"
> 
> Walter, can you try out the attached patch?  It seems to work for me.
> Both for unicode and ascii strings as author name, containing an umlaut.

Installing http://www.python.org/ftp/python/2.4/Python-2.4a1.msi,
applying your patch to the CVS version and copying the distutils
directory over C:\Programme\Python24\Lib\distutils gives me the
following:

c:\programme\python24\python setup.py bdist --formats=wininst
running bdist
running bdist_wininst
running build
running build_py
Traceback (most recent call last):
   File "setup.py", line 50, in ?
     package_dir={"ll": ""}
   File "c:\programme\python24\lib\distutils\core.py", line 150, in setup
     dist.run_commands()
   File "c:\programme\python24\lib\distutils\dist.py", line 951, in 
run_commands
     self.run_command(cmd)
   File "c:\programme\python24\lib\distutils\dist.py", line 971, in 
run_command
     cmd_obj.run()
   File "c:\programme\python24\lib\distutils\command\bdist.py", line 
146, in run
     self.run_command(cmd_name)
   File "c:\programme\python24\lib\distutils\cmd.py", line 333, in 
run_command
     self.distribution.run_command(command)
   File "c:\programme\python24\lib\distutils\dist.py", line 971, in 
run_command
     cmd_obj.run()
   File "c:\programme\python24\lib\distutils\command\bdist_wininst.py", 
line 127, in run
     assert self.skip_build, "Should have already checked this"
AssertionError: Should have already checked this

Bye,
    Walter D?rwald

From fdrake at acm.org  Fri Jul 16 23:01:10 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri Jul 16 23:01:21 2004
Subject: [Python-Dev] Planned changes to logging
In-Reply-To: <007201c46afa$7fb8c400$652b6992@alpha>
References: <000f01c45c99$5e644dc0$652b6992@alpha>
	<d7cee4a704071422523dc9173c@mail.gmail.com>
	<007201c46afa$7fb8c400$652b6992@alpha>
Message-ID: <200407161701.10341.fdrake@acm.org>

On Friday 16 July 2004 02:02 am, Vinay Sajip wrote:
 > The whole area of LogRecord construction needs to be made more extensible.
 > I'm in discussions with Fred Drake about this, and hopefully soon will be
 > able to implement a better mechanism.
 >
 > Fred - if you're listening - :-)

I've not forgotten; I need to get back to explaining what I've come up with.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>

From python at rcn.com  Fri Jul 16 11:47:05 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Jul 16 23:48:23 2004
Subject: [Python-Dev] Constancy of None
In-Reply-To: <200407161619.i6GGJ7402512@guido.python.org>
Message-ID: <005501c46b19$da8c51c0$c622c797@oemcomputer>

[Raymond]
> > That raises the question.  Since there was a SyntaxWarning in Py2.3,
> > should it have been escalated to an error for Py2.4?

[AMK]
> > +1.
> >
> > (And is it time to make assignments to True and False trigger a
> > SyntaxWarning, or should that wait until 2.5?)

[Guido]
> We should be much more conservative with that.  There's tons of code
> out there that assigns these (conditionally, if it's 2.3-aware) to
> truth values and we don't want to break that.

We may never (prior to Py3.0) be able to block assignments to
True/False; however, some lesser restriction could be feasible.  For
example, it may be possible to require that the assignment have the same
boolean value so that "True=x" rejects when bool(x)!=True.  The would
allow "True=1" and
"True=(1==1)" statements to work while still enabling the compiler to
optimize the test out of "while True".

While I'm not sure this is a great idea, I would have no remorse for
wrecking a piece of code that assigned "True=0"    ;-)



[Guido]
>  The situation with None
> is much better, since it was always there; AFAIK only a few speed
> freaks in Zope-land have ever assigned to it (and always using the
> form "None = None").

Guido's reasoning here is confirmed by Py2.3's SyntaxWarning not causing
the slightest stir.  With respect to the constancy of None, we are on
firm ground.

If there are no objections, I would like to leave in yesterday's patch
and upgrade the Py2.4 None assignment SyntaxWarning to a SyntaxError.


Raymond

From guido at python.org  Sat Jul 17 01:14:41 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Jul 17 01:14:47 2004
Subject: [Python-Dev] "as" to be a keyword?
In-Reply-To: Your message of "Fri, 16 Jul 2004 20:06:41 +0200."
	<40F81931.3040007@v.loewis.de> 
References: <5.1.1.6.0.20040716135337.01ec94c0@mail.telecommunity.com>  
	<40F81931.3040007@v.loewis.de> 
Message-ID: <200407162314.i6GNEfd03082@guido.python.org>

> > Is it true that 'as' is going to become a keyword in future? 
> 
> I doubt it. If this was meant to happen, it would have happened
> long ago.

It will be a keyword in Python 3.0.

Before then, it's probably not worth facing the comaptibility issues.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tim.peters at gmail.com  Sat Jul 17 03:29:42 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Jul 17 03:29:44 2004
Subject: [Python-Dev] bdist_wininst
In-Reply-To: <8ydkuudh.fsf@python.net>
References: <8ydkuudh.fsf@python.net>
Message-ID: <1f7befae040716182915e4c82e@mail.gmail.com>

[Thomas Heller]
> ...
> Second, the default bitmap it displays is the old Python Powered logo by
> Just (that with the cool license ;-).  A much better bitmap would be
> that from <http://tinyurl.com/595xd>, which seems to have been made for
> the PSF.  There are two problems with this image: The size is 122x261
> pixels, while bdist_wininst would prefer 152x261 pixels.  Maybe someone
> can recreate the icon with a changed size?
> The file size, when it is converted to 24-bit color bmp, is
> nearly 100 kB, which is a bit large imo for an optical gimmik - but this
> can probably be fixed by compressing the image in the installer itself.
> Finally, I'm not sure I'm allowed to use and redistribute this image.

That looks like a minor variant of the graphic Erik van Blokland-- a
colleague of Just's --created for Python's Wise installer.  Erik asked
for an acknowledgement  in return for the work, and that's why the
Wise installer displayed this on its last (Installation Complete)
screen:

Special Windows thanks to:

    Wise Solutions, for the use of InstallMaster 8.1.
        http://www.wisesolutions.com/

    LettError, Erik van Blokland, for the Python for Windows graphic.
        http://www.letterror.com/

    Mark Hammond, without whose years of freely shared Windows expertise,
    Python for Windows would still be Python for DOS.

So ask Just about this.  I don't have a current email address for Erik.

OTOH, the Python-Powered logo is quite recognizable now, and it's fine
by me if bsdist_wininst just leaves this alone!

> Third, the installer executable doesn't have an icon.  Personally I
> don't care too much about that, but Phil Hornby made two icons which he
> posted to the distutils list (the gmane archives do a nice job to show
> the icons on the web page):
>
> <http://article.gmane.org/gmane.comp.python.distutils.devel/852>
> <http://article.gmane.org/gmane.comp.python.distutils.devel/855>

For me, those both show little boxes with a red X, i.e. the image
couldn't be loaded.  Or maybe that's what they're supposed to look
like <wink>.
From tim.one at comcast.net  Sat Jul 17 06:12:48 2004
From: tim.one at comcast.net (Tim Peters)
Date: Sat Jul 17 06:12:56 2004
Subject: [Python-Dev] Does CVS Python work for anyone else?
Message-ID: <20040717041255.5B00A1E4002@bag.python.org>

On WinXP, it segfaults instantly, while trying to import site.py.  I'm able
to get into the interpreter by passing -S (skips importing site.py).  Any
attempt to run the test suite also segfaults instantly, even with -S.


From tim.peters at gmail.com  Sat Jul 17 06:15:42 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Jul 17 06:15:45 2004
Subject: [Python-Dev] Does CVS Python work for anyone else?
In-Reply-To: <20040717041255.5B00A1E4002@bag.python.org>
References: <20040717041255.5B00A1E4002@bag.python.org>
Message-ID: <1f7befae040716211569d89746@mail.gmail.com>

[Tim Peters]
> On WinXP, it segfaults instantly, while trying to import site.py.  I'm able
> to get into the interpreter by passing -S (skips importing site.py).  Any
> attempt to run the test suite also segfaults instantly, even with -S.

FYI, reverting the most recent checkin to compile.c repairs all this:

C:\Code\python\Python>cvs up -j 2.307 -j 2.306 compile.c
From ncoghlan at iinet.net.au  Sat Jul 17 06:31:34 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sat Jul 17 06:32:09 2004
Subject: [Python-Dev] Developer looking to help fix bugs
In-Reply-To: <40F68999.3040905@iinet.net.au>
References: <1089688837.2657.5.camel@localhost.localdomain>	<20040714135557.GA24990@panix.com>
	<40F68999.3040905@iinet.net.au>
Message-ID: <40F8ABA6.3000105@iinet.net.au>

Nick Coghlan wrote:
> For instance, I think making the Developer pages the first link in the 
> 'Community' section in the side bar would make them much easier to find. 
> (The actual developer pages I quite like - it was finding them in the 
> first place that I found more difficult than it should have been)

Ah, my bad. The 'Python Project' title takes you to the developer pages. 
For some reason, I'd only ever checked the three Source Forge links that 
appear there.

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From tim.peters at gmail.com  Sat Jul 17 07:08:09 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Jul 17 07:08:16 2004
Subject: [Python-Dev] Does CVS Python work for anyone else?
In-Reply-To: <1f7befae040716211569d89746@mail.gmail.com>
References: <20040717041255.5B00A1E4002@bag.python.org>
	<1f7befae040716211569d89746@mail.gmail.com>
Message-ID: <1f7befae04071622081c557f3e@mail.gmail.com>

[Tim]
> On WinXP, it segfaults instantly, while trying to import site.py.  I'm able
> to get into the interpreter by passing -S (skips importing site.py).  Any
> attempt to run the test suite also segfaults instantly, even with -S.

I believe this is repaired now (gross error in new code; missing a
break stmt before a new case in a switch stmt, so an old case fell
into the new code by mistake, leading to a variety of insane
behaviors).

Everyone, please run the test suite before checking code in!  This
isn't what I wanted to spend my smidgen of Python time on tonight.
From tim.one at comcast.net  Sat Jul 17 07:50:45 2004
From: tim.one at comcast.net (Tim Peters)
Date: Sat Jul 17 07:50:58 2004
Subject: [Python-Dev] Kill FCNTL.py
Message-ID: <20040717055056.F0F791E4006@bag.python.org>

Can we get rid of Lib/FCNTL.py for 2.4?  AFAICT, its existence does nothing
but create obscure problems for Windows users.  For an old example, in
tempfile.py:

try:
    import fcntl as _fcntl
    # If PYTHONCASEOK is set on Windows, stinking FCNTL.py gets
    # imported, and we don't get an ImportError then.  Provoke
    # an AttributeError instead in that case.
    _fcntl.fcntl
except (ImportError, AttributeError):


For an example from yesterday:

    http://mail.zope.org/pipermail/zope-dev/2004-July/023463.html

I don't know all the ways this happens, but a surprising number of Windows
users end up with a file named:

    fcntl.pyc

in their Lib directories, and then tons of things break, because imports of
fcntl pick that up instead of getting the ImportError cross-platform code
expects on Windows.

I know that one persistent user traced this to one of the Python-aware
installer-builders creating allupper.pyc files from ALLUPPER.py files on
Windows, but it's not worth the effort of tracking this down:  since the day
it was introduced, FCNTL.py has generated a deprecation warning telling you
to use fcntl instead.  The builtin fcntl was introduced in 1.5a3.  Enough
already <wink>.


From s.percivall at chello.se  Sat Jul 17 13:24:34 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Sat Jul 17 13:24:36 2004
Subject: [Python-Dev] None as constant. Still SyntaxWarning
Message-ID: <E1448088-D7E3-11D8-9368-0003934AD54A@chello.se>

After the recent change in CVS None in is treated like a constant, and 
assigment to it fails. But it _seems_ to pass with a SyntaxWarning just 
like in 2.3.

From ncoghlan at iinet.net.au  Sat Jul 17 13:59:35 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sat Jul 17 13:59:32 2004
Subject: [Python-Dev] Non-Visual Studio builds on Windows XP?
Message-ID: <40F914A7.5070804@iinet.net.au>

I'm probably missing something obvious, but what can I use to interpret 
the vcproj build files?

The free toolkit only seems to contain the compiler and the linker, 
neither of which has a clue about .vcproj or .sln files.

And MinGW needs a normal make file, rather than the Microsoft custom 
formats.

Any pointers to more info would be appreciated. . .

(If no-one else is trying to do this, my next step will be to write my 
own .vcproj -> makefile Python script to get something I can feed to 
MinGW, since my Google searches haven't turned up anything useful. I'd 
just hate to do that unnecessarily)

Regards,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From ncoghlan at iinet.net.au  Sat Jul 17 17:24:39 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sat Jul 17 17:24:37 2004
Subject: [Python-Dev] Non-Visual Studio builds on Windows XP?
In-Reply-To: <877e9a17040717063138470602@mail.gmail.com>
References: <40F914A7.5070804@iinet.net.au>
	<877e9a17040717063138470602@mail.gmail.com>
Message-ID: <40F944B7.9060202@iinet.net.au>

Michael Walter wrote:

> I was under the impression that nmake worked with .sln or .vcproj files.
> 
> On Sat, 17 Jul 2004 21:59:35 +1000, Nick Coghlan <ncoghlan@iinet.net.au> wrote:
> 
>>I'm probably missing something obvious, but what can I use to interpret
>>the vcproj build files?
>>
> 

That something obvious I mentioned? The main problem I had was that 
nmake isn't in the free VC Toolkit.

It eventually occured to me that it isn't in the VC toolkit because its 
already in some of the other free downloads from MS. . .

Thanks, anyway :)

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From ncoghlan at iinet.net.au  Sat Jul 17 18:53:22 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sat Jul 17 18:53:18 2004
Subject: [Python-Dev] Non-Visual Studio builds on Windows XP?
In-Reply-To: <40F944B7.9060202@iinet.net.au>
References: <40F914A7.5070804@iinet.net.au>	<877e9a17040717063138470602@mail.gmail.com>
	<40F944B7.9060202@iinet.net.au>
Message-ID: <40F95982.7060903@iinet.net.au>

And one download of the .Net SDK later. . . I'm still stuck. From 
Microsoft's web page:

".vcproj files are not compatible with NMAKE. You must export a makefile 
to build with NMAKE."

(http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore/html/_core_file_types_created_for_visual_c.2b2b_.projects.asp).

Which brings me back to needing a .vcproj -> makefile conversion tool. 
At the moment, I can't find any references to anything other than the 
full version of Visual Studio being able to do this.

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From python at rcn.com  Sat Jul 17 07:15:27 2004
From: python at rcn.com (Raymond Hettinger)
Date: Sat Jul 17 19:16:55 2004
Subject: [Python-Dev] None as constant. Still SyntaxWarning
In-Reply-To: <E1448088-D7E3-11D8-9368-0003934AD54A@chello.se>
Message-ID: <009a01c46bbd$12dd4ae0$c622c797@oemcomputer>

> After the recent change in CVS None in is treated like a constant, and
> assigment to it fails. But it _seems_ to pass with a SyntaxWarning
just
> like in 2.3.

That is forthcoming.  Remain calm.


Raymond

From nhodgson at bigpond.net.au  Sun Jul 18 00:22:20 2004
From: nhodgson at bigpond.net.au (Neil Hodgson)
Date: Sun Jul 18 00:22:27 2004
Subject: [Python-Dev] Non-Visual Studio builds on Windows XP?
References: <40F914A7.5070804@iinet.net.au>
	<877e9a17040717063138470602@mail.gmail.com>
	<40F944B7.9060202@iinet.net.au> <40F95982.7060903@iinet.net.au>
Message-ID: <002901c46c4c$869da850$ee4a8890@neil>

Nick Coghlan:

> Which brings me back to needing a .vcproj -> makefile conversion tool.
> At the moment, I can't find any references to anything other than the
> full version of Visual Studio being able to do this.

   It is possible that a 2 stage process may get you something.

   prjconverter converts from VS .NET .vcproj to VC 6 .dsp:
http://www.codeproject.com/tools/prjconverter.asp

   dsw2mak converts from .dsp to makefile
http://cvs.sourceforge.net/viewcvs.py/mingw/utils/scripts/dsw2mak.in

   I experimented with prjconverter for my projects a while back but decided
it was easier to maintain the different build files in parallel by hand
instead. I haven't tried dsw2mak.

   Here is someone trying to build Python using dsw2mak:
http://uucode.com/texts/python-mingw/python-mingw.html

   Neil

From martin at v.loewis.de  Sun Jul 18 01:05:17 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Jul 18 01:05:21 2004
Subject: [Python-Dev] Non-Visual Studio builds on Windows XP?
In-Reply-To: <40F914A7.5070804@iinet.net.au>
References: <40F914A7.5070804@iinet.net.au>
Message-ID: <40F9B0AD.8010109@v.loewis.de>

Nick Coghlan wrote:
> I'm probably missing something obvious, but what can I use to interpret 
> the vcproj build files?

devenv.exe, and only devenv.exe. You need Microsoft Visual Studio .NET
2003 to interpret these files.

That said: they are XML files, so you can interpret them with 
notepad.exe also.

Garth wrote a set of Python scripts that attempt to interpret these
files as a build process:

http://mail.python.org/pipermail/python-dev/2004-February/042595.html

Regards,
Martin

From guido at python.org  Sun Jul 18 03:20:55 2004
From: guido at python.org (Guido van Rossum)
Date: Sun Jul 18 03:21:02 2004
Subject: [Python-Dev] Kill FCNTL.py
In-Reply-To: Your message of "Sat, 17 Jul 2004 01:50:45 EDT."
	<20040717055056.F0F791E4006@bag.python.org> 
References: <20040717055056.F0F791E4006@bag.python.org> 
Message-ID: <200407180120.i6I1Kth04897@guido.python.org>

> Can we get rid of Lib/FCNTL.py for 2.4?  AFAICT, its existence does
> nothing but create obscure problems for Windows users.

+1

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tim.one at comcast.net  Sun Jul 18 08:13:06 2004
From: tim.one at comcast.net (Tim Peters)
Date: Sun Jul 18 08:13:10 2004
Subject: [Python-Dev] Fun with whitespace
Message-ID: <20040718061309.DCAA01E4002@bag.python.org>

Heads up!  Using reindent.py to normalize whitespace in the Tools/scripts
directory uncovered a file with bad syntax problems.  So I repaired that
file's errors, then ran reindent over the entire source tree.

Using reindent once creates a fixed-point:  running reindent again on the
output file won't change it.  That makes reindent an easy way to ensure
files in the core follow the Python coding guidelines.

Before today, I only ran it on hand-selected subdirectories, to minimize the
pain if it broke anything (it can break code that relies on trailing
whitespace being preserved in Python files, and a few cases of that have
occurred over the years; no other case of damage has ever been reported).

All the tests still pass on my box, but perhaps there's damage on other
platforms.

Checking in all the changes is proving to be a challenge, and I'm not sure
checkin msgs are getting sent out for all the changes.  I'm getting lots of
these tracebacks from SF:

Traceback (most recent call last):
  File "/cvsroot/python/CVSROOT/syncmail", line 329, in ?
    main()
  File "/cvsroot/python/CVSROOT/syncmail", line 322, in main
    blast_mail(subject, people, specs[1:], contextlines, fromhost)
  File "/cvsroot/python/CVSROOT/syncmail", line 247, in blast_mail
    resp = conn.sendmail(author, people, s.getvalue())
  File "/usr/lib/python2.2/smtplib.py", line 654, in sendmail
    if not (200 <= self.ehlo()[0] <= 299):
  File "/usr/lib/python2.2/smtplib.py", line 384, in ehlo
    (code,msg)=self.getreply()
  File "/usr/lib/python2.2/smtplib.py", line 335, in getreply
    raise SMTPServerDisconnected("Connection unexpectedly closed")

I think I'm also bumping into never-before-noticed stale CVS locks on files
nobody has touched in years (e.g., Guido has a lock on
Demo/threads/Generator.py).


From hyeshik at gmail.com  Sun Jul 18 10:16:10 2004
From: hyeshik at gmail.com (Hye-Shik Chang)
Date: Sun Jul 18 10:16:15 2004
Subject: [Python-Dev] 
	Re: [Python-checkins] python/dist/src/PC config.c, 1.47, 1.48
In-Reply-To: <E1Bm3sf-0006Az-JA@sc8-pr-cvs1.sourceforge.net>
References: <E1Bm3sf-0006Az-JA@sc8-pr-cvs1.sourceforge.net>
Message-ID: <4f0b69dc04071801161e931297@mail.gmail.com>

On Sat, 17 Jul 2004 22:06:33 -0700, tim_one@users.sourceforge.net
<tim_one@users.sourceforge.net> wrote:
> Update of /cvsroot/python/python/dist/src/PC
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23589/PC
> 
> Modified Files:
>         config.c
> Log Message:
> Woo hoo!  All the encoding tests pass on Windows (& I downloaded the
> 14 input files needed to run all of them).
> 

Thanks for fixing cjkcodecs to build in Windows again.  I'll rearrange
cjkcodecs.h to get rid of the problem you described, soon.  :)

Hye-Shik
From ncoghlan at iinet.net.au  Sun Jul 18 11:41:06 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sun Jul 18 11:41:14 2004
Subject: [Python-Dev] Non-Visual Studio builds on Windows XP?
In-Reply-To: <40F9B0AD.8010109@v.loewis.de>
References: <40F914A7.5070804@iinet.net.au> <40F9B0AD.8010109@v.loewis.de>
Message-ID: <40FA45B2.8050804@iinet.net.au>

Martin v. L?wis wrote:
> Nick Coghlan wrote:
> 
> Garth wrote a set of Python scripts that attempt to interpret these
> files as a build process:
> 
> http://mail.python.org/pipermail/python-dev/2004-February/042595.html

Which is also in my local py-dev archive (now that I know what to look 
for). . . *slaps forehead*

It didn't work first try, but I'm sure it will be fairly straightforward 
to sort out once I can put some more time into it. It looks like Garth 
has done most of the heavy lifting (which is the work I was trying to 
avoid duplicating!).

Thanks for the help, folks.

Cheers,
Nick.

(./configure, make test on the Mandrake box was so much easier. . . *g*)

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From tim.peters at gmail.com  Sun Jul 18 19:13:28 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sun Jul 18 19:13:31 2004
Subject: [Python-Dev] Fun with whitespace
In-Reply-To: <20040718061309.DCAA01E4002@bag.python.org>
References: <20040718061309.DCAA01E4002@bag.python.org>
Message-ID: <1f7befae0407181013306eb5d8@mail.gmail.com>

[Tim[
...
> Checking in all the changes is proving to be a challenge, ...

Using "cvs admin -uREV file" managed to break the mysterious version
locks that were holding this up, and it's done now.  Running "reindent
-r ." from the root of a checkout now leaves everything alone.  That
in turns means that all the .py files in the checkout:

- Use 4-space indent levels.
- Contain no hard tabs.
- Have no trailing whitespace on any line.
- End with a newline character.

The "-u all" tests pass on WinXP, in release and debug builds.

Apologies in advance if breakage occurred on other platforms, but code
relying on trailing whitespace-- or hard tab characters --in .py files
is so non-WYSIWYG it needed to be repaired anyway.  The good news is
that it's dead easy to keep the tree in whitespace-pristine shape now.
From Jack.Jansen at cwi.nl  Sun Jul 18 22:53:47 2004
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Sun Jul 18 22:53:29 2004
Subject: [Python-Dev] Adding directories to configure.in for fink?
In-Reply-To: <40F8112F.40107@ocf.berkeley.edu>
References: <40F76FC1.3040908@ocf.berkeley.edu>	<06559C87-D701-11D8-A243-000A958D1666@cwi.nl>
	<878ydks3qk.fsf@tleepslib.sk.tsukuba.ac.jp>
	<40F8112F.40107@ocf.berkeley.edu>
Message-ID: <90B18417-D8FC-11D8-99F7-000D934FF6B4@cwi.nl>


On 16 Jul 2004, at 19:32, Brett C. wrote:
> And if we could take this even further and have the stuff configure.in 
> is passed and discovers available to Fink we could customize the 
> install even further.  I bring this up because if you do a 
> --without-toolbox-glue build on OS X you get a ton of failed links of 
> various modules that want Carbon stuff that they don't get since they 
> were turned off.  If that flag was available in setup.py then we could 
> just add a conditional making sure that the flag was not flipped to 
> off.

This is actually a slightly different problem: the information is in 
pyconfig.h (USE_TOOLBOX_OBJECT_GLUE), but distutils doesn't seem to 
have a way to get at it.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman

From garth at garthy.com  Sun Jul 18 23:45:49 2004
From: garth at garthy.com (Garth)
Date: Sun Jul 18 23:46:19 2004
Subject: [Python-Dev] Non-Visual Studio builds on Windows XP?
In-Reply-To: <40FA45B2.8050804@iinet.net.au>
References: <40F914A7.5070804@iinet.net.au> <40F9B0AD.8010109@v.loewis.de>
	<40FA45B2.8050804@iinet.net.au>
Message-ID: <40FAEF8D.6020202@garthy.com>

Nick Coghlan wrote:

> Martin v. L?wis wrote:
>
>> Nick Coghlan wrote:
>>
>> Garth wrote a set of Python scripts that attempt to interpret these
>> files as a build process:
>>
>> http://mail.python.org/pipermail/python-dev/2004-February/042595.html
>
>
> Which is also in my local py-dev archive (now that I know what to look 
> for). . . *slaps forehead*
>
> It didn't work first try, but I'm sure it will be fairly 
> straightforward to sort out once I can put some more time into it. It 
> looks like Garth has done most of the heavy lifting (which is the work 
> I was trying to avoid duplicating!).
>
> Thanks for the help, folks.
>
> Cheers,
> Nick.
>
> (./configure, make test on the Mandrake box was so much easier. . . *g*)
>
If these scripts don't work I'm happy to help out.

You can also try the free version of the IDE downloadable from 
http://lab.msdn.microsoft.com/express/visualc/default.aspx. I haven't 
given it a go but it may work.

Garth
From stephen at xemacs.org  Mon Jul 19 11:18:12 2004
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon Jul 19 11:18:29 2004
Subject: [Python-Dev] Adding directories to configure.in for fink?
In-Reply-To: <B00CF432-D733-11D8-A243-000A958D1666@cwi.nl> (Jack Jansen's
	message of "Fri, 16 Jul 2004 16:23:20 +0200")
References: <40F76FC1.3040908@ocf.berkeley.edu>
	<06559C87-D701-11D8-A243-000A958D1666@cwi.nl>
	<878ydks3qk.fsf@tleepslib.sk.tsukuba.ac.jp>
	<B00CF432-D733-11D8-A243-000A958D1666@cwi.nl>
Message-ID: <87pt6s8iej.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Jack" == Jack Jansen <Jack.Jansen@cwi.nl> writes:

    Jack> On 16-jul-04, at 11:25, Stephen J. Turnbull wrote:

    >> If you're going to do this for Fink, don't you want to do it
    >> for DarwinPorts too?  XEmacs's configure has a --site-prefixes
    >> switch such that --site-prefixes=/sw does both
    >> --site-includes=/sw/include and --site-libraries=/sw/lib.  This
    >> handles both Fink and DarwinPorts (and anythign else that might
    >> come along) in a sensible way.  Isn't that a generic autoconf
    >> feature?

    Jack> This would be very nice to have. How much work would it be
    Jack> to implement it?

I'm not sure offhand, because I don't know the Python build system.
Of course argument parsing is trivial.  If we can just add those to
the generic compile and link commands once each (which is effectively
how it works in XEmacs) and once to setup.py, use is trivial, too.
I'll take a look; it'll take a few days, though.  Is there a time
constraint (like are we shooting for the next 2.4 alpha)?

Likely the biggest problem is licensing (XEmacs is GPL, of course),
but I bet I know who coded our configure.in, and he would give
permission for me to do the work. <wink>

    Jack> We don't currently support site-includes and site-libraries
    Jack> either (I think) so that would need to be added too.

Same deal; it's just a question of prepending to the compile and link
commands in such a way as to implement the documented include and load
paths.

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.
From theller at python.net  Mon Jul 19 11:48:50 2004
From: theller at python.net (Thomas Heller)
Date: Mon Jul 19 11:48:51 2004
Subject: [Python-Dev] bdist_wininst
In-Reply-To: <40F834D0.50903@livinglogic.de> (
	=?iso-8859-1?q?Walter_D=F6rwald's_message_of?= "Fri,
	16 Jul 2004 22:04:32 +0200")
References: <8ydkuudh.fsf@python.net> <40F80403.9030904@livinglogic.de>
	<r7rbsxjp.fsf@python.net> <40F80907.7040606@livinglogic.de>
	<iscnstyg.fsf@python.net> <40F834D0.50903@livinglogic.de>
Message-ID: <u0w48gzh.fsf@python.net>

Walter D?rwald <walter@livinglogic.de> writes:

> Thomas Heller wrote:
>>>>>>bdist_wininst has seen a lot of bug fixes and improvements, mostly by
>>>>>>Mark, partly by me.
>>>>>>I'd like to 'improve' the visual appearance, too.
>>>>>
>>>>>One improvement would be proper treatment of Unicode. Currently it
>>>>>doesn't seem to be possible to use unicode objects with umlauts in the
>>>>>arguments to distutils.core.setup(). Some of my Windows packages say
>>>>>"Author: Walter Dxf6rwald"
>> Walter, can you try out the attached patch?  It seems to work for me.
>> Both for unicode and ascii strings as author name, containing an umlaut.
>
> Installing http://www.python.org/ftp/python/2.4/Python-2.4a1.msi,
> applying your patch to the CVS version and copying the distutils
> directory over C:\Programme\Python24\Lib\distutils gives me the
> following:
[...]


I doubt this traceback has to do with my patch.  Can you try again,
first without the patch, after doing 'cvs update -A', to make sure you
have the most recent version?  (We should probably move this to the SF tracker)

Thomas

From theller at python.net  Mon Jul 19 11:56:54 2004
From: theller at python.net (Thomas Heller)
Date: Mon Jul 19 12:02:24 2004
Subject: [Python-Dev] bdist_wininst
In-Reply-To: <1f7befae040716182915e4c82e@mail.gmail.com> (Tim Peters's
	message of "Fri, 16 Jul 2004 21:29:42 -0400")
References: <8ydkuudh.fsf@python.net>
	<1f7befae040716182915e4c82e@mail.gmail.com>
Message-ID: <oemc8gm1.fsf@python.net>

Tim Peters <tim.peters@gmail.com> writes:

> [Thomas Heller]
>> ...
>> Second, the default bitmap it displays is the old Python Powered logo by
>> Just (that with the cool license ;-).  A much better bitmap would be
>> that from <http://tinyurl.com/595xd>, which seems to have been made for
>> the PSF.  There are two problems with this image: The size is 122x261
>> pixels, while bdist_wininst would prefer 152x261 pixels.  Maybe someone
>> can recreate the icon with a changed size?
>> The file size, when it is converted to 24-bit color bmp, is
>> nearly 100 kB, which is a bit large imo for an optical gimmik - but this
>> can probably be fixed by compressing the image in the installer itself.
>> Finally, I'm not sure I'm allowed to use and redistribute this image.
>
> That looks like a minor variant of the graphic Erik van Blokland-- a
> colleague of Just's --created for Python's Wise installer.  Erik asked
> for an acknowledgement  in return for the work, and that's why the
> Wise installer displayed this on its last (Installation Complete)
> screen:
>
> Special Windows thanks to:
>
>     Wise Solutions, for the use of InstallMaster 8.1.
>         http://www.wisesolutions.com/
>
>     LettError, Erik van Blokland, for the Python for Windows graphic.
>         http://www.letterror.com/
>
>     Mark Hammond, without whose years of freely shared Windows expertise,
>     Python for Windows would still be Python for DOS.
>
> So ask Just about this.  I don't have a current email address for Erik.
>
> OTOH, the Python-Powered logo is quite recognizable now, and it's fine
> by me if bsdist_wininst just leaves this alone!

The reason I was looking into this is that including an XP style
manifest into wininst.exe was that the logo looks somewhat different
then.  It didn't fill any longer whole area of the static field.  I will
take a look if this can be fixed.

>> Third, the installer executable doesn't have an icon.  Personally I
>> don't care too much about that, but Phil Hornby made two icons which he
>> posted to the distutils list (the gmane archives do a nice job to show
>> the icons on the web page):
>>
>> <http://article.gmane.org/gmane.comp.python.distutils.devel/852>
>> <http://article.gmane.org/gmane.comp.python.distutils.devel/855>
>
> For me, those both show little boxes with a red X, i.e. the image
> couldn't be loaded.  Or maybe that's what they're supposed to look
> like <wink>.

The boxes with the red X's mean 'upgrade to Mozilla' ;-).  (For me, the
icons are displayed in Mozilla, but I see the red X's in IE as well).
The pipermail urls are these

<http://mail.python.org/pipermail/distutils-sig/2004-February/003704.html>
<http://mail.python.org/pipermail/distutils-sig/2004-February/003707.html>

they allow to download the icons.

Thomas

From colanderman at gmail.com  Fri Jul 16 02:10:59 2004
From: colanderman at gmail.com (Chris King)
Date: Mon Jul 19 12:48:37 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <1f7befae040715150164b1481b@mail.gmail.com>
References: <1089894294.40f677961fc20@login.werra.lunarpages.com>
	<40F6D628.7010002@v.loewis.de>
	<e8bf7a5304071513046311217@mail.gmail.com>
	<1f7befae040715150164b1481b@mail.gmail.com>
Message-ID: <875c7e0704071517101e10f27a@mail.gmail.com>

On Thu, 15 Jul 2004 18:01:25 -0400, Tim Peters <tim.peters@gmail.com> wrote:
> [Jeremy Hylton]
> > ...
> > Getting a useful traceback after tail call optimization seems like an
> > awfully important quality of implementation issue.
> 
> Indeed, in delegation-heavy designs, when an error occurs the
> delegation path you took to reach the failing code often *is* "the
> error".

Well, since optimizing only when -O is specified was rejected, and
from __future__ import tail_call seems like a similarly bad idea, why
not something like 'sys.optimize_tail_calls()' to enable the alternate
mechanism? This can be done in such a way that programs not using it
won't even notice a change in execution speed (e.g. using a different
eval loop when it's enabled).

I'd then place this mechanism in the same category as such other
implementation-altering functions as gc.disable(), psyco.full(), and
sys.setrecursionlimt(1e9). Since it's explicitly enabled by the
programmer, he'd know the consequences of enabling it. The only
downside is a little extra glue code would be needed to replace it
with an appropriate sys.setrecursionlimit() call on Pythons that don't
support it:

import sys
try:
    sys.optimize_tail_calls()
except AttributeError:
    sys.setrecursionlimit(10000)
From mwh at python.net  Mon Jul 19 17:32:39 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Jul 19 17:32:41 2004
Subject: [Python-Dev] Constancy of None
In-Reply-To: <005501c46b19$da8c51c0$c622c797@oemcomputer> (Raymond
	Hettinger's message of "Fri, 16 Jul 2004 05:47:05 -0400")
References: <005501c46b19$da8c51c0$c622c797@oemcomputer>
Message-ID: <2m4qo4t3l4.fsf@starship.python.net>

"Raymond Hettinger" <python@rcn.com> writes:

> [Raymond]
>> > That raises the question.  Since there was a SyntaxWarning in Py2.3,
>> > should it have been escalated to an error for Py2.4?
>
> [AMK]
>> > +1.
>> >
>> > (And is it time to make assignments to True and False trigger a
>> > SyntaxWarning, or should that wait until 2.5?)
>
> [Guido]
>> We should be much more conservative with that.  There's tons of code
>> out there that assigns these (conditionally, if it's 2.3-aware) to
>> truth values and we don't want to break that.
>
> We may never (prior to Py3.0) be able to block assignments to
> True/False; however, some lesser restriction could be feasible.  For
> example, it may be possible to require that the assignment have the same
> boolean value so that "True=x" rejects when bool(x)!=True.  The would
> allow "True=1" and
> "True=(1==1)" statements to work while still enabling the compiler to
> optimize the test out of "while True".

Sounds, um, challenging to implement.  I guess you could emit
different code for an assignment to True than for other identifiers...

> While I'm not sure this is a great idea, I would have no remorse for
> wrecking a piece of code that assigned "True=0"    ;-)

But what about "True=1" or even "True=[1]"?

Cheers,
mwh

-- 
  If I didn't have my part-time performance art income to help pay
  the  bills, I could never afford to support my programming 
  lifestyle.                                -- Jeff Bauer, 21 Apr 2000
From mwh at python.net  Mon Jul 19 17:36:17 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Jul 19 17:36:19 2004
Subject: [Python-Dev] Re: Proper tail recursion
In-Reply-To: <20040716094428.4EC1.JCARLSON@uci.edu> (Josiah Carlson's
	message of "Fri, 16 Jul 2004 10:45:38 -0700")
References: <20040715235400.6990.JCARLSON@uci.edu>
	<875c7e0704071606231c8f5795@mail.gmail.com>
	<20040716094428.4EC1.JCARLSON@uci.edu>
Message-ID: <2mzn5wroum.fsf@starship.python.net>

Josiah Carlson <jcarlson@uci.edu> writes:

> Honestly, I'd like to see a real application that has a bug only in
> recursion level 501.

I'm sure PyPy has managed it :-)

Cheers,
mwh

-- 
3. Syntactic sugar causes cancer of the semicolon.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html
From Paul.Moore at atosorigin.com  Mon Jul 19 17:54:01 2004
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Mon Jul 19 17:54:01 2004
Subject: [Python-Dev] Thread safety of deques
Message-ID: <16E1010E4581B049ABC51D4975CEDB8803060F1A@UKDCX001.uk.int.atosorigin.com>

The documentation for the collections.deque type states that "Deques
support thread-safe, memory efficient appends and pops from either
side of the deque". I'm not clear what the implication of saying that
these operations are "thread-safe" is.

To contrast, list.append() does not claim thread safety. Should I
therefore conclude that to append an item in a thread-safe manner to a
list requires a mutex, but not for a deque?

Overall, the documentation does not tend to state what operations
are "atomic" in thread-safety terms. Continuing that trend, it may
be better *not* to make any statement about deques. If we do, then
not saying anything about other types seems to imply that they are
not thread-safe. Which may well be true, but if so should probably
be explicit, once we start making thread safety claims of other data
structures.

This way to the slippery slope... :-)

Paul.


__________________________________________________________________________
This e-mail and the documents attached are confidential and intended 
solely for the addressee; it may also be privileged. If you receive this 
e-mail in error, please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos Origin group 
liability cannot be triggered for the message content. Although the 
sender endeavours to maintain a computer virus-free network, the sender 
does not warrant that this transmission is virus-free and will not be 
liable for any damages resulting from any virus transmitted.
__________________________________________________________________________
From colanderman at gmail.com  Mon Jul 19 18:27:58 2004
From: colanderman at gmail.com (Chris King)
Date: Mon Jul 19 18:28:02 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <875c7e0704071517101e10f27a@mail.gmail.com>
References: <1089894294.40f677961fc20@login.werra.lunarpages.com>
	<40F6D628.7010002@v.loewis.de>
	<e8bf7a5304071513046311217@mail.gmail.com>
	<1f7befae040715150164b1481b@mail.gmail.com>
	<875c7e0704071517101e10f27a@mail.gmail.com>
Message-ID: <875c7e07040719092747381a6d@mail.gmail.com>

Given the proposed changes:

1) make a seperate eval loop that supports tail calls so as not to
impact performance of normal programs,
2) force tail calls to be explicitly enabled using a run-time function call,
3) print a warning in stack traces when tail calls are enabled and used.

is Guido still -1 on this?
From neal at metaslash.com  Mon Jul 19 18:37:35 2004
From: neal at metaslash.com (Neal Norwitz)
Date: Mon Jul 19 18:37:42 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	bltinmodule.c, 2.312, 2.313
In-Reply-To: <E1Bmb0y-0007fl-3z@sc8-pr-cvs1.sourceforge.net>
References: <E1Bmb0y-0007fl-3z@sc8-pr-cvs1.sourceforge.net>
Message-ID: <20040719163735.GE25168@epoch.metaslash.com>

On Mon, Jul 19, 2004 at 09:29:20AM -0700, nascheme@users.sourceforge.net wrote:

> ! 	res = (*nb->nb_hex)(v);
> ! 	if (res && !PyString_Check(res)) {
> ! 		PyErr_Format(PyExc_TypeError,
> ! 			     "__hex__ returned non-string (type %.200s)",
> ! 			     res->ob_type->tp_name);
> ! 		Py_DECREF(res);
> ! 		return NULL;
> ! 	}
> ! 	return res;

Should hex/oct be allowed to return Unicode?

Neal
From mal at egenix.com  Mon Jul 19 18:41:32 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon Jul 19 18:41:36 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
 bltinmodule.c, 2.312, 2.313
In-Reply-To: <20040719163735.GE25168@epoch.metaslash.com>
References: <E1Bmb0y-0007fl-3z@sc8-pr-cvs1.sourceforge.net>
	<20040719163735.GE25168@epoch.metaslash.com>
Message-ID: <40FBF9BC.1000506@egenix.com>

Neal Norwitz wrote:
> On Mon, Jul 19, 2004 at 09:29:20AM -0700, nascheme@users.sourceforge.net wrote:
> 
> 
>>! 	res = (*nb->nb_hex)(v);
>>! 	if (res && !PyString_Check(res)) {
>>! 		PyErr_Format(PyExc_TypeError,
>>! 			     "__hex__ returned non-string (type %.200s)",
>>! 			     res->ob_type->tp_name);
>>! 		Py_DECREF(res);
>>! 		return NULL;
>>! 	}
>>! 	return res;
> 
> Should hex/oct be allowed to return Unicode?

-0

hex/oct values fit nicely into ASCII... unless you want to
use more fancy code points for the numbers in there :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 19 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From jhylton at gmail.com  Mon Jul 19 18:48:11 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Mon Jul 19 18:48:13 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <875c7e07040719092747381a6d@mail.gmail.com>
References: <1089894294.40f677961fc20@login.werra.lunarpages.com>
	<40F6D628.7010002@v.loewis.de>
	<e8bf7a5304071513046311217@mail.gmail.com>
	<1f7befae040715150164b1481b@mail.gmail.com>
	<875c7e0704071517101e10f27a@mail.gmail.com>
	<875c7e07040719092747381a6d@mail.gmail.com>
Message-ID: <e8bf7a5304071909487cf66052@mail.gmail.com>

On Mon, 19 Jul 2004 12:27:58 -0400, Chris King <colanderman@gmail.com> wrote:
> Given the proposed changes:
> 
> 1) make a seperate eval loop that supports tail calls so as not to
> impact performance of normal programs,
> 2) force tail calls to be explicitly enabled using a run-time function call,
> 3) print a warning in stack traces when tail calls are enabled and used.
> 
> is Guido still -1 on this?

Just a guess but "make a seperate eval loop" sounds like a sure way to
get a 10,000 pound weight dropped on you? <wink>  Seriously, how do
you accomplish this without code duplication?

Jeremy
From colanderman at gmail.com  Mon Jul 19 19:01:36 2004
From: colanderman at gmail.com (Chris King)
Date: Mon Jul 19 19:01:42 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <e8bf7a5304071909487cf66052@mail.gmail.com>
References: <1089894294.40f677961fc20@login.werra.lunarpages.com>
	<40F6D628.7010002@v.loewis.de>
	<e8bf7a5304071513046311217@mail.gmail.com>
	<1f7befae040715150164b1481b@mail.gmail.com>
	<875c7e0704071517101e10f27a@mail.gmail.com>
	<875c7e07040719092747381a6d@mail.gmail.com>
	<e8bf7a5304071909487cf66052@mail.gmail.com>
Message-ID: <875c7e07040719100129413009@mail.gmail.com>

On Mon, 19 Jul 2004 12:48:11 -0400, Jeremy Hylton <jhylton@gmail.com> wrote:

> Just a guess but "make a seperate eval loop" sounds like a sure way to
> get a 10,000 pound weight dropped on you? <wink>  Seriously, how do
> you accomplish this without code duplication?

Oh, you don't -- you just let the C preprocessor do the duplication for you ;)
From pje at telecommunity.com  Mon Jul 19 19:06:59 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Jul 19 19:03:31 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <875c7e07040719092747381a6d@mail.gmail.com>
References: <875c7e0704071517101e10f27a@mail.gmail.com>
	<1089894294.40f677961fc20@login.werra.lunarpages.com>
	<40F6D628.7010002@v.loewis.de>
	<e8bf7a5304071513046311217@mail.gmail.com>
	<1f7befae040715150164b1481b@mail.gmail.com>
	<875c7e0704071517101e10f27a@mail.gmail.com>
Message-ID: <5.1.1.6.0.20040719130354.03113040@mail.telecommunity.com>

At 12:27 PM 7/19/04 -0400, Chris King wrote:
>Given the proposed changes:
>
>1) make a seperate eval loop that supports tail calls so as not to
>impact performance of normal programs,

Um, no.  Please look at ceval.c before you even consider proposing 
something like that, and then be sure to have a proposal for how you'd 
*maintain* this second eval loop.



From python at rcn.com  Mon Jul 19 07:09:00 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Jul 19 19:10:31 2004
Subject: [Python-Dev] Thread safety of deques
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8803060F1A@UKDCX001.uk.int.atosorigin.com>
Message-ID: <000e01c46d4e$8137a200$e841fea9@oemcomputer>

[Paul Moore]
> The documentation for the collections.deque type states that "Deques
> support thread-safe, memory efficient appends and pops from either
> side of the deque". I'm not clear what the implication of saying that
> these operations are "thread-safe" is.

It is thread-safe in the same sense as the MersenneTwister's random()
method -- all state is updated in a single step.  This contrasts with
pure python implementations which can be interrupted in the middle of
updating state.

For random number generation, the implication is that two threads can
share the same generator (giving up ability to recreate a sequence)
without worrying about wrecking the generator's internal state.
Likewise, two threads can append or pop from the same deque without
trashing its internal state (of course, that's only a good idea if you
don't care about who appends or pops first).


> To contrast, list.append() does not claim thread safety. 

List operations like most C functions are also atomic.



> This way to the slippery slope... :-)

Yes.  It is clearly possible to twist words into meaningless knots while
talking about threads.  Feel free to submit a clarifying doc patch.  

Try to keep it in proportion.  It would be easy to introduce a rats nest
of discussion about the GIL and whatnot.  While possibly more precise,
it is likely to be a distraction from the surrounding documentation and
make things even less clear to those who have no interest in threads.  

As an alternative, it may be worthwhile to submit a candidate glossary
entry about thread-safety.



Raymond


From colanderman at gmail.com  Mon Jul 19 19:14:55 2004
From: colanderman at gmail.com (Chris King)
Date: Mon Jul 19 19:14:59 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <5.1.1.6.0.20040719130354.03113040@mail.telecommunity.com>
References: <875c7e0704071517101e10f27a@mail.gmail.com>
	<1089894294.40f677961fc20@login.werra.lunarpages.com>
	<40F6D628.7010002@v.loewis.de>
	<e8bf7a5304071513046311217@mail.gmail.com>
	<1f7befae040715150164b1481b@mail.gmail.com>
	<875c7e0704071517101e10f27a@mail.gmail.com>
	<5.1.1.6.0.20040719130354.03113040@mail.telecommunity.com>
Message-ID: <875c7e0704071910141fae69dc@mail.gmail.com>

On Mon, 19 Jul 2004 13:06:59 -0400, Phillip J. Eby
<pje@telecommunity.com> wrote:
> At 12:27 PM 7/19/04 -0400, Chris King wrote:
> >Given the proposed changes:
> >
> >1) make a seperate eval loop that supports tail calls so as not to
> >impact performance of normal programs,
> 
> Um, no.  Please look at ceval.c before you even consider proposing
> something like that, and then be sure to have a proposal for how you'd
> *maintain* this second eval loop.

I've spent a good deal of time in ceval.c implementing my previous
proposal, and an #include and some #defines should do the trick. 
Result: only one function to maintain, living in a seperate file. 
Yes, I realize that's a roundabout way of doing stuff, but even
projects as closely related to Python as numarray use much more
drastic measures to automate code duplication (hint: the C
preprocessor wasn't enough for numarray).
From walter at livinglogic.de  Mon Jul 19 19:18:17 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon Jul 19 19:18:21 2004
Subject: [Python-Dev] bdist_wininst
In-Reply-To: <u0w48gzh.fsf@python.net>
References: <8ydkuudh.fsf@python.net>
	<40F80403.9030904@livinglogic.de>	<r7rbsxjp.fsf@python.net>
	<40F80907.7040606@livinglogic.de>	<iscnstyg.fsf@python.net>
	<40F834D0.50903@livinglogic.de> <u0w48gzh.fsf@python.net>
Message-ID: <40FC0259.4090800@livinglogic.de>

Thomas Heller wrote:

> Walter D?rwald <walter@livinglogic.de> writes:
> 
>> [...]
>>Installing http://www.python.org/ftp/python/2.4/Python-2.4a1.msi,
>>applying your patch to the CVS version and copying the distutils
>>directory over C:\Programme\Python24\Lib\distutils gives me the
>>following:
> 
> [...]
> 
> I doubt this traceback has to do with my patch.  Can you try again,
> first without the patch, after doing 'cvs update -A', to make sure you
> have the most recent version?  (We should probably move this to the SF tracker)
> 
> Thomas

Done: http://www.python.org/sf/993943

Bye,
    Walter D?rwald


From guido at python.org  Mon Jul 19 19:18:14 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Jul 19 19:18:24 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: Your message of "Mon, 19 Jul 2004 12:27:58 EDT."
	<875c7e07040719092747381a6d@mail.gmail.com> 
References: <1089894294.40f677961fc20@login.werra.lunarpages.com>
	<40F6D628.7010002@v.loewis.de>
	<e8bf7a5304071513046311217@mail.gmail.com>
	<1f7befae040715150164b1481b@mail.gmail.com>
	<875c7e0704071517101e10f27a@mail.gmail.com> 
	<875c7e07040719092747381a6d@mail.gmail.com> 
Message-ID: <200407191718.i6JHIEo03220@guido.python.org>

> Given the proposed changes:
> 
> 1) make a seperate eval loop that supports tail calls so as not to
> impact performance of normal programs,
> 2) force tail calls to be explicitly enabled using a run-time function call,
> 3) print a warning in stack traces when tail calls are enabled and used.
> 
> is Guido still -1 on this?

Yes, very strongly against.

I don't like reading code that was written by someone trying to use
tail recursion.  It's the ultimate code obfuscation.

I also haven't seen the use case that requires this and couldn't
easily be fixed by changing the data structure or code slightly.
(Andrew Koenig's theoretical objections don't count as use cases.)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Mon Jul 19 19:19:24 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Jul 19 19:19:32 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	bltinmodule.c, 2.312, 2.313
In-Reply-To: Your message of "Mon, 19 Jul 2004 12:37:35 EDT."
	<20040719163735.GE25168@epoch.metaslash.com> 
References: <E1Bmb0y-0007fl-3z@sc8-pr-cvs1.sourceforge.net>  
	<20040719163735.GE25168@epoch.metaslash.com> 
Message-ID: <200407191719.i6JHJOk03240@guido.python.org>

> On Mon, Jul 19, 2004 at 09:29:20AM -0700, nascheme@users.sourceforge.net wrote:
> 
> > ! 	res = (*nb->nb_hex)(v);
> > ! 	if (res && !PyString_Check(res)) {
> > ! 		PyErr_Format(PyExc_TypeError,
> > ! 			     "__hex__ returned non-string (type %.200s)",
> > ! 			     res->ob_type->tp_name);
> > ! 		Py_DECREF(res);
> > ! 		return NULL;
> > ! 	}
> > ! 	return res;
> 
> Should hex/oct be allowed to return Unicode?

What would the point be?  I don't know of any hexadecimal or octal
digits that aren't ASCII.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From ark-mlist at att.net  Mon Jul 19 19:25:45 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Mon Jul 19 19:25:47 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <200407191718.i6JHIEo03220@guido.python.org>
Message-ID: <005701c46db5$764b48d0$6602a8c0@arkdesktop>

> I also haven't seen the use case that requires this and couldn't
> easily be fixed by changing the data structure or code slightly.
> (Andrew Koenig's theoretical objections don't count as use cases.)

Didn't we just hear that this problem affects pickling?

From neal at metaslash.com  Mon Jul 19 19:39:28 2004
From: neal at metaslash.com (Neal Norwitz)
Date: Mon Jul 19 19:39:37 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	bltinmodule.c, 2.312, 2.313
In-Reply-To: <200407191719.i6JHJOk03240@guido.python.org>
References: <E1Bmb0y-0007fl-3z@sc8-pr-cvs1.sourceforge.net>
	<20040719163735.GE25168@epoch.metaslash.com>
	<200407191719.i6JHJOk03240@guido.python.org>
Message-ID: <20040719173928.GG25168@epoch.metaslash.com>

On Mon, Jul 19, 2004 at 10:19:24AM -0700, Guido van Rossum wrote:
>
> > Should hex/oct be allowed to return Unicode?
> 
> What would the point be?  I don't know of any hexadecimal or octal
> digits that aren't ASCII.

I wasn't paying close enough attention.  Originally, I was thinking
about consistency, but on second thought I agree it's not worthwhile.

Neal

From eppstein at ics.uci.edu  Mon Jul 19 20:30:30 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Mon Jul 19 20:30:39 2004
Subject: [Python-Dev] Re: Proper tail recursion
References: <200407191718.i6JHIEo03220@guido.python.org>
	<005701c46db5$764b48d0$6602a8c0@arkdesktop>
Message-ID: <eppstein-9AD036.11303019072004@sea.gmane.org>

In article <005701c46db5$764b48d0$6602a8c0@arkdesktop>,
 "Andrew Koenig" <ark-mlist@att.net> wrote:

> > I also haven't seen the use case that requires this and couldn't
> > easily be fixed by changing the data structure or code slightly.
> > (Andrew Koenig's theoretical objections don't count as use cases.)
> 
> Didn't we just hear that this problem affects pickling?

We heard that the recursion limit prevents recursive DFS from being 
applied to large graphs, and that pickling uses DFS.  However, tail 
recursion wouldn't help in this case -- it's a deep recursion but not a 
tail recursion.

I did end up implementing a non-recursive DFS, btw.  It has two 
advantages over recursive DFS: (1) the amount of stuff stored per stack 
item is much smaller: a pair (vertex, iterator of neighbors), rather 
than all the overhead of a call stack frame, and (2) the flat control 
structure allows it to be used as a simple generator (e.g. for listing 
vertices in preorder or postorder).  The disadvantage is that the code 
is (I think) less readable than a recursive DFS would be.  You can view 
my attempt at http://www.ics.uci.edu/~eppstein/PADS/DFS.py (one hint 
that Python's control structures are missing something is the explicit 
use of try...iterator.next()...except StopIteration) but please send any 
feedback about my code to me off-list since it's not really a python-dev 
issue any more.

-- 
David Eppstein                      http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science

From kiko at async.com.br  Mon Jul 19 20:44:49 2004
From: kiko at async.com.br (Christian Robottom Reis)
Date: Mon Jul 19 20:44:54 2004
Subject: [Python-Dev] PEP 331: Locale-Independent Float/String Conversions
Message-ID: <20040719184449.GM623@async.com.br>


The LC_NUMERIC PEP has just been allocated a number, PEP 331. 

    http://www.python.org/peps/pep-0331.html

Though the code has been integrated, comments and corrections to the PEP
would be appreciated -- in particular, Tim might have some comments on
the thread-safety issues, Gustavo, about the implementation or Martin,
about the actual constraints on the change itself.

Take care,
--
Christian Robottom Reis | http://async.com.br/~kiko/ | [+55 16] 3361 2331
From mcherm at mcherm.com  Mon Jul 19 21:17:23 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon Jul 19 21:16:50 2004
Subject: [Python-Dev] Proper tail recursion
Message-ID: <1090264643.40fc1e4311be2@mcherm.com>

Chris King writes:
> Given the proposed changes:
       [...]
> is Guido still -1 on this?

Guido replies:
> Yes, very strongly against.
>
> I don't like reading code that was written by someone trying to use
> tail recursion.  It's the ultimate code obfuscation.

As one of the people who has been defending this proposal, let me
now suggest that we just drop the idea. Guido has made it quite
clear... it's not the way that it's implemented that he objects to,
it's the mere fact that it enables the user to write using a highly
recursive style. No change to the implementation (even if you could
avoid having the second event loop) will fix the problem. Your
original change can be kept around as a patch that people can apply
for themselves if they want to. I expect that anyone who *really*
cared would be using stackless anyway.

You and I may think that Guido is wrong on this one, but he's been
quite clear and it's not worth arguing over.

-- Michael Chermside

From colanderman at gmail.com  Mon Jul 19 21:44:49 2004
From: colanderman at gmail.com (Chris King)
Date: Mon Jul 19 21:44:51 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <1090264643.40fc1e4311be2@mcherm.com>
References: <1090264643.40fc1e4311be2@mcherm.com>
Message-ID: <875c7e07040719124460a2e7d5@mail.gmail.com>

On Mon, 19 Jul 2004 12:17:23 -0700, Michael Chermside <mcherm@mcherm.com> wrote:

> As one of the people who has been defending this proposal, let me
> now suggest that we just drop the idea.

Agreed.  Thanks to everyone who helped defend it though, and thanks to
others for providing obective feedback!

> Your original change can be kept around as a patch that people can apply
> for themselves if they want to. I expect that anyone who *really*
> cared would be using stackless anyway.

I think it would actually even be possible to turn it into an
extension module, but that's a project for another day ;)
From python at rcn.com  Mon Jul 19 09:44:02 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Jul 19 21:45:35 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <1090264643.40fc1e4311be2@mcherm.com>
Message-ID: <002501c46d64$295af300$e841fea9@oemcomputer>

[Michael Chermside]
> As one of the people who has been defending this proposal, let me
> now suggest that we just drop the idea.

I second that motion.  It is not worth complicating ceval.c and not
advisable to steer users down a path that is usually the wrong solution
to a problem.



Raymond

From kiko at async.com.br  Mon Jul 19 22:22:49 2004
From: kiko at async.com.br (Christian Robottom Reis)
Date: Mon Jul 19 22:23:04 2004
Subject: [Python-Dev] PEP 331: Locale-Independent Float/String Conversions
In-Reply-To: <20040719184449.GM623@async.com.br>
References: <20040719184449.GM623@async.com.br>
Message-ID: <20040719202249.GP623@async.com.br>

On Mon, Jul 19, 2004 at 03:44:49PM -0300, Christian Robottom Reis wrote:
> 
> The LC_NUMERIC PEP has just been allocated a number, PEP 331. 
> 
>     http://www.python.org/peps/pep-0331.html

There is a single XXX comment there: 

    [XXX: was the agreement actually received and accepted?]

-- referring to Alex Larsson's requested PSF form, which was submitted
on August of last year: 

    http://mail.python.org/pipermail/python-dev/2003-August/037755.html

Does anyone have a confirmation it was received and filed?

Take care,
--
Christian Robottom Reis | http://async.com.br/~kiko/ | [+55 16] 3361 2331
From neal at metaslash.com  Mon Jul 19 22:35:13 2004
From: neal at metaslash.com (Neal Norwitz)
Date: Mon Jul 19 22:35:19 2004
Subject: [Python-Dev] PEP 331: Locale-Independent Float/String Conversions
In-Reply-To: <20040719202249.GP623@async.com.br>
References: <20040719184449.GM623@async.com.br>
	<20040719202249.GP623@async.com.br>
Message-ID: <20040719203513.GI25168@epoch.metaslash.com>

On Mon, Jul 19, 2004 at 05:22:49PM -0300, Christian Robottom Reis wrote:
> 
> -- referring to Alex Larsson's requested PSF form, which was submitted
> on August of last year: 
> 
>     http://mail.python.org/pipermail/python-dev/2003-August/037755.html
> 
> Does anyone have a confirmation it was received and filed?

Yes, a completed PSF contribution licensing agreement was received.

Neal
From tismer at stackless.com  Mon Jul 19 23:09:29 2004
From: tismer at stackless.com (Christian Tismer)
Date: Mon Jul 19 23:07:16 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <1f7befae040715175725c86abf@mail.gmail.com>
References: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>	<40F4FF98.2000204@sweetapp.com>
	<40F656A2.70803@stackless.com>
	<1f7befae040715175725c86abf@mail.gmail.com>
Message-ID: <40FC3889.8010401@stackless.com>

Tim Peters wrote:

> [Christian Tismer]
> 
>>...
>>Boehm GC might become feasible.
>>If we then can write C extensions without having to think
>>of reference counts, code would be simplified quite
>>much, and I would save 80 percent of debugging time.
> 
> 
> There's no escape from gc debugging nightmares at the C level,
> regardless of gc strategy.  With Boehm gc (Bgc) you have to ensure
> that all live objects are reachable from what Bgc guesses is the
> current root set, and screwing that up is just as deadly as screwing
> up refcounts.  It's harder to debug, though, because refcounting is
> very simple to picture, while Bgc is a relatively huge pile of
> complicated code in its own right.  I'm sure Neil Schemenauer still
> has fond memories of the pleasant days he spent trying to account for
> Tk crashes last time he tried hooking Bgc to Python <heh>.  Bgc has a
> special entry point to call, to register roots that Bgc can't find on
> its own -- guess how *you* find out which roots those are?  One crash
> at a time, and in the presence of threads you're damned lucky to get a
> repeatable test case.

Well, I was about to stick with this answer.

Then I saw that Prothon is going a different path: They don't
use a Boehm gc, but they make every object indirect, over a
global object table, and use generational garbage collection
with arenas.
I guess this would not lead to problems like stack introspection,
finding roots etc., because all objects are in the object table.
Of course it is another level of indirection
everywhere, with the advantage that object bodies become moveable.
I think this is a simple enough gc to shorten the nightmares.
No idea how much the performance loss is, or if this is amortized
by the missing reference counting? At least an opportunity to try
would be nice to have.

If not now, we will try it with PyPy, anyway -- ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From tismer at stackless.com  Mon Jul 19 23:17:36 2004
From: tismer at stackless.com (Christian Tismer)
Date: Mon Jul 19 23:15:27 2004
Subject: [Python-Dev] Re: Proper tail recursion
In-Reply-To: <eppstein-9AD036.11303019072004@sea.gmane.org>
References: <200407191718.i6JHIEo03220@guido.python.org>	<005701c46db5$764b48d0$6602a8c0@arkdesktop>
	<eppstein-9AD036.11303019072004@sea.gmane.org>
Message-ID: <40FC3A70.1030809@stackless.com>

David Eppstein wrote:

> In article <005701c46db5$764b48d0$6602a8c0@arkdesktop>,
>  "Andrew Koenig" <ark-mlist@att.net> wrote:
> 
> 
>>>I also haven't seen the use case that requires this and couldn't
>>>easily be fixed by changing the data structure or code slightly.
>>>(Andrew Koenig's theoretical objections don't count as use cases.)
>>
>>Didn't we just hear that this problem affects pickling?
> 
> 
> We heard that the recursion limit prevents recursive DFS from being 
> applied to large graphs, and that pickling uses DFS.  However, tail 
> recursion wouldn't help in this case -- it's a deep recursion but not a 
> tail recursion.

[dropping interesting details for my brute-force solution]

I went a different and very simple path for Stackless:
When cPickle goes too deep, I simply allocate a new C stack
and let it continue. That was the simplest possible patch
for me, since I only wanted it to no longer crash.
Not that this is not needed for unpickling; this structure
is as flat as it can be.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From python at rcn.com  Mon Jul 19 11:41:04 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Jul 19 23:42:42 2004
Subject: [Python-Dev] Prototypes  [was: Proper tail recursion]
In-Reply-To: <1089987282.8409.157.camel@localhost>
Message-ID: <000401c46d74$830e4180$e841fea9@oemcomputer>

>The
> > > request is make it possible to write:
> > >
> > >     def Account.deposit(self, v):
> > >         self.balance += v

[Barry]
> I've wanted this for a long time ...

I'm building a list of use cases and am curious about your application
needs.



Raymond


From pf_moore at yahoo.co.uk  Tue Jul 20 00:01:39 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Tue Jul 20 00:01:25 2004
Subject: [Python-Dev] Re: Thread safety of deques
References: <16E1010E4581B049ABC51D4975CEDB8803060F1A@UKDCX001.uk.int.atosorigin.com>
	<000e01c46d4e$8137a200$e841fea9@oemcomputer>
Message-ID: <bribhd18.fsf@yahoo.co.uk>

"Raymond Hettinger" <python@rcn.com> writes:

> [Paul Moore]
>> The documentation for the collections.deque type states that "Deques
>> support thread-safe, memory efficient appends and pops from either
>> side of the deque". I'm not clear what the implication of saying that
>> these operations are "thread-safe" is.
>
> It is thread-safe in the same sense as the MersenneTwister's random()
> method -- all state is updated in a single step.  This contrasts with
> pure python implementations which can be interrupted in the middle of
> updating state.

[...]

>> To contrast, list.append() does not claim thread safety. 
>
> List operations like most C functions are also atomic.

Thanks for that clarification. A note *somewhere* in the python-level
documentation, stating that operations implemented in C are typically
atomic, and so thread-safe, might be useful.

>> This way to the slippery slope... :-)
>
> Yes.  It is clearly possible to twist words into meaningless knots while
> talking about threads.  Feel free to submit a clarifying doc patch.  
>
> Try to keep it in proportion.  It would be easy to introduce a rats nest
> of discussion about the GIL and whatnot.

Oh, definitely! Anything more than a simple paragraph would be
overkill. But some general guidance would be useful (a while ago, I
started a thread on c.l.p discussing whether d[id] = {} was
thread-safe, and at least one person stated that you should *always*
use a mutex to protect access to a shared object, which is a pretty
extreme position...)

Of course, Jython probably complicates the issue further...

> As an alternative, it may be worthwhile to submit a candidate glossary
> entry about thread-safety.

That may be a good idea. I'll try to put something together.

Paul.

PS The random quote below seems appropriate :-)
-- 
A little inaccuracy sometimes saves tons of explanation -- Saki

From pje at telecommunity.com  Tue Jul 20 00:53:16 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Jul 20 00:49:48 2004
Subject: [Python-Dev] Re: Thread safety of deques
In-Reply-To: <bribhd18.fsf@yahoo.co.uk>
References: <16E1010E4581B049ABC51D4975CEDB8803060F1A@UKDCX001.uk.int.atosorigin.com>
	<000e01c46d4e$8137a200$e841fea9@oemcomputer>
Message-ID: <5.1.1.6.0.20040719184449.032380f0@mail.telecommunity.com>

At 11:01 PM 7/19/04 +0100, Paul Moore wrote:

> >> To contrast, list.append() does not claim thread safety.
> >
> > List operations like most C functions are also atomic.
>
>Thanks for that clarification. A note *somewhere* in the python-level
>documentation, stating that operations implemented in C are typically
>atomic, and so thread-safe, might be useful.

Perhaps it would be useful to point out that "safe" here means, "doesn't 
corrupt the dictionary or crash Python".  It doesn't mean "actually safe".  :)

It would probably be "safest" not to describe anything as "thread safe", 
but rather point out that in general, Python won't crash because a data 
structure was accessed or altered by more than one thread, and that most 
operations on most built-in types are atomic.

However, anything that calls back into Python code isn't necessarily 
atomic, and in general, almost anything can call back into Python code.

For example, the seemingly harmless dictionary item assignment can call 
back into Python for a custom type's __hash__ and __eq__ methods, so could 
it be that a thread switch could actually occur during this statement:

     foo[bar] = baz

if 'bar' has such custom methods?  Maybe I'm wrong, and the GIL is still 
held in such a case, but I'm not sure *how*.

From guido at python.org  Tue Jul 20 01:34:06 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul 20 01:34:13 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: Your message of "Mon, 19 Jul 2004 13:25:45 EDT."
	<005701c46db5$764b48d0$6602a8c0@arkdesktop> 
References: <005701c46db5$764b48d0$6602a8c0@arkdesktop> 
Message-ID: <200407192334.i6JNY6G03670@guido.python.org>

> > I also haven't seen the use case that requires this and couldn't
> > easily be fixed by changing the data structure or code slightly.
> > (Andrew Koenig's theoretical objections don't count as use cases.)
> 
> Didn't we just hear that this problem affects pickling?

That's a rather special case, and unless you have a trick to eliminate
tail recursion C too, it won't help us much.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From jepler at unpythonic.net  Tue Jul 20 01:52:09 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Tue Jul 20 01:52:32 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
References: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
Message-ID: <20040719235208.GB5065@unpythonic.net>

One more random Tagged Types thought.

If we were introducing the test & branch for tagged types in incref/decref
anyway (unlikely, I know) wouldn't this let us add another oft-suggested
optimization at the same price: for certain immortal objects (None, (),
'', etc) there's no need to actually change the refcount.  Make these
objects Tagged, and those memory accesses go away in favor of a single
register test that already exists to provide tagged integers.

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040719/95012132/attachment.pgp
From tjreedy at udel.edu  Tue Jul 20 03:58:12 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue Jul 20 03:58:24 2004
Subject: [Python-Dev] Re: Proper tail recursion
References: <1089894294.40f677961fc20@login.werra.lunarpages.com><40F6D628.7010002@v.loewis.de><e8bf7a5304071513046311217@mail.gmail.com><1f7befae040715150164b1481b@mail.gmail.com>
	<875c7e0704071517101e10f27a@mail.gmail.com>
Message-ID: <cdhu7r$fiq$1@sea.gmane.org>


"Chris King" <colanderman@gmail.com> wrote in message
news:875c7e0704071517101e10f27a@mail.gmail.com...
> I'd then place this mechanism in the same category as such other
> implementation-altering functions as gc.disable(),

I believe gc.disable merely disables an execution path rather than
providing an alternative.

> sys.setrecursionlimt(1e9).

Changing a constant is rather trivial compared to what you are proposing.

>psyco.full(), and

This does seem analogous; it provides a large chunk of alternative
execution.  And it is not part of the core distribution (yet, that I know
of) but is instead a third party package/module, just a stackless is.  This
is probably what you should start with.

If a tail recursion group shows itself capable of handling patches, bugs,
docs, and tests for such an addon, and develops a large constituency, then
you might propose integration.  Or you might well decide you prefer to stay
independent and keep total control of your work.

Terry J. Reedy



From dave at boost-consulting.com  Tue Jul 20 04:59:39 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Tue Jul 20 05:10:14 2004
Subject: [Python-Dev] Re: Tagged integers
References: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>
	<40F4FF98.2000204@sweetapp.com> <40F656A2.70803@stackless.com>
	<1f7befae040715175725c86abf@mail.gmail.com>
	<40FC3889.8010401@stackless.com>
Message-ID: <uacxvbcys.fsf@boost-consulting.com>

Christian Tismer <tismer@stackless.com> writes:

> Then I saw that Prothon is going a different path: They don't
> use a Boehm gc, but they make every object indirect, over a
> global object table, and use generational garbage collection
> with arenas.
> I guess this would not lead to problems like stack introspection,
> finding roots etc., because all objects are in the object table.
> Of course it is another level of indirection
> everywhere, with the advantage that object bodies become moveable.

That would break countless extension modules, though.  Today,
extensions can count on stable object addresses.


-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From greg at cosc.canterbury.ac.nz  Tue Jul 20 05:55:29 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Jul 20 05:55:34 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <40FC3889.8010401@stackless.com>
Message-ID: <200407200355.i6K3tTmE018841@cosc353.cosc.canterbury.ac.nz>

Christian Tismer <tismer@stackless.com>:

> I guess this would not lead to problems like stack introspection,
> finding roots etc., because all objects are in the object table.

You still need roots to find reachable objects. The object
table contains all objects, whether reachable or not.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From aahz at pythoncraft.com  Tue Jul 20 05:55:54 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue Jul 20 05:55:57 2004
Subject: [Python-Dev] Re: Thread safety of deques
In-Reply-To: <5.1.1.6.0.20040719184449.032380f0@mail.telecommunity.com>
References: <16E1010E4581B049ABC51D4975CEDB8803060F1A@UKDCX001.uk.int.atosorigin.com>
	<000e01c46d4e$8137a200$e841fea9@oemcomputer>
	<5.1.1.6.0.20040719184449.032380f0@mail.telecommunity.com>
Message-ID: <20040720035553.GA26153@panix.com>

On Mon, Jul 19, 2004, Phillip J. Eby wrote:
>
> However, anything that calls back into Python code isn't necessarily 
> atomic, and in general, almost anything can call back into Python code.

Yup.

> For example, the seemingly harmless dictionary item assignment can call 
> back into Python for a custom type's __hash__ and __eq__ methods, so could 
> it be that a thread switch could actually occur during this statement:
> 
>     foo[bar] = baz
> 
> if 'bar' has such custom methods?  Maybe I'm wrong, and the GIL is still 
> held in such a case, but I'm not sure *how*.

No, it isn't.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Barbara Boxer speaks for me:
http://buffaloreport.com/2004/040713.boxer.marriage.html
From bac at OCF.Berkeley.EDU  Tue Jul 20 08:02:46 2004
From: bac at OCF.Berkeley.EDU (Brett Cannon)
Date: Tue Jul 20 08:02:54 2004
Subject: [Python-Dev] test_urllib failing
Message-ID: <40FCB586.9060107@ocf.berkeley.edu>

On OS X 10.3.4:


test_urllib
Exception exceptions.AttributeError: "FancyURLopener instance has no 
attribute 'tempcache'" in <bound method FancyURLopener.__del__ of 
<urllib.FancyURLopener instance at 0x3741578>> ignored
Exception exceptions.AttributeError: "FancyURLopener instance has no 
attribute 'tempcache'" in <bound method FancyURLopener.__del__ of 
<urllib.FancyURLopener instance at 0x3741eb8>> ignored
Exception exceptions.AttributeError: "FancyURLopener instance has no 
attribute 'tempcache'" in <bound method FancyURLopener.__del__ of 
<urllib.FancyURLopener instance at 0x3741df8>> ignored
Exception exceptions.AttributeError: "FancyURLopener instance has no 
attribute 'tempcache'" in <bound method FancyURLopener.__del__ of 
<urllib.FancyURLopener instance at 0x3741d78>> ignored
Exception exceptions.AttributeError: "FancyURLopener instance has no 
attribute 'tempcache'" in <bound method FancyURLopener.__del__ of 
<urllib.FancyURLopener instance at 0x3741e38>> ignored
Exception exceptions.AttributeError: "FancyURLopener instance has no 
attribute 'tempcache'" in <bound method FancyURLopener.__del__ of 
<urllib.FancyURLopener instance at 0x3741578>> ignored
Exception exceptions.AttributeError: "FancyURLopener instance has no 
attribute 'tempcache'" in <bound method FancyURLopener.__del__ of 
<urllib.FancyURLopener instance at 0x3741eb8>> ignored
Exception exceptions.AttributeError: "FancyURLopener instance has no 
attribute 'tempcache'" in <bound method FancyURLopener.__del__ of 
<urllib.FancyURLopener instance at 0x3741df8>> ignored
Exception exceptions.AttributeError: "FancyURLopener instance has no 
attribute 'tempcache'" in <bound method FancyURLopener.__del__ of 
<urllib.FancyURLopener instance at 0x3741d78>> ignored
Exception exceptions.AttributeError: "FancyURLopener instance has no 
attribute 'tempcache'" in <bound method FancyURLopener.__del__ of 
<urllib.FancyURLopener instance at 0x3741db8>> ignored
Exception exceptions.AttributeError: "FancyURLopener instance has no 
attribute 'tempcache'" in <bound method FancyURLopener.__del__ of 
<urllib.FancyURLopener instance at 0x373a4f8>> ignored
Exception exceptions.AttributeError: "FancyURLopener instance has no 
attribute 'tempcache'" in <bound method FancyURLopener.__del__ of 
<urllib.FancyURLopener instance at 0x373a4b8>> ignored
Exception exceptions.AttributeError: "FancyURLopener instance has no 
attribute 'tempcache'" in <bound method FancyURLopener.__del__ of 
<urllib.FancyURLopener instance at 0x373a438>> ignored
test test_urllib failed -- errors occurred; run in verbose mode for details
test_urllib2
test test_urllib2 failed -- errors occurred; run in verbose mode for details


This only happens when I run regrtest on the whole test suite; 
individually the tests pass.  I don't have time to debug this and keep 
running the test suite (getting late and I burned my Python hours on my 
own bug today that does not affect this), but there was Tim's whitespace 
cleanup yesterday and Jack checked in a fix for proxies (which I don't 
use, but it is OS X related so who knows; this is a --disable-framework 
--disable-toolbox-glue build) on the 16th.  Otherwise urllib has not 
been touched since March.

-Brett
From tismer at stackless.com  Tue Jul 20 09:48:09 2004
From: tismer at stackless.com (Christian Tismer)
Date: Tue Jul 20 09:47:37 2004
Subject: [Python-Dev] Re: Tagged integers
In-Reply-To: <uacxvbcys.fsf@boost-consulting.com>
References: <D0537558-D560-11D8-B5AB-000A95A50FB2@fuhm.net>	<40F4FF98.2000204@sweetapp.com>
	<40F656A2.70803@stackless.com>	<1f7befae040715175725c86abf@mail.gmail.com>	<40FC3889.8010401@stackless.com>
	<uacxvbcys.fsf@boost-consulting.com>
Message-ID: <40FCCE39.7070105@stackless.com>

David Abrahams wrote:
> Christian Tismer <tismer@stackless.com> writes:
> 
> 
>>Then I saw that Prothon is going a different path: They don't
>>use a Boehm gc, but they make every object indirect, over a
>>global object table, and use generational garbage collection
>>with arenas.
>>I guess this would not lead to problems like stack introspection,
>>finding roots etc., because all objects are in the object table.
>>Of course it is another level of indirection
>>everywhere, with the advantage that object bodies become moveable.
> 
> That would break countless extension modules, though.  Today,
> extensions can count on stable object addresses.

That's ok. I don't think this whole thread is about a real
change of Python in the near future. It is about testing
alternatives and to study how things change then.
Sure, this is all impossible to make compatible.

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From tismer at stackless.com  Tue Jul 20 09:50:41 2004
From: tismer at stackless.com (Christian Tismer)
Date: Tue Jul 20 09:50:00 2004
Subject: [Python-Dev] Tagged integers
In-Reply-To: <200407200355.i6K3tTmE018841@cosc353.cosc.canterbury.ac.nz>
References: <200407200355.i6K3tTmE018841@cosc353.cosc.canterbury.ac.nz>
Message-ID: <40FCCED1.6020100@stackless.com>

Greg Ewing wrote:

> Christian Tismer <tismer@stackless.com>:
> 
> 
>>I guess this would not lead to problems like stack introspection,
>>finding roots etc., because all objects are in the object table.
> 
> 
> You still need roots to find reachable objects. The object
> table contains all objects, whether reachable or not.

I was referring to the problems which Tim pointed out.
With an explicit object table, I see much less problems.

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From Paul.Moore at atosorigin.com  Tue Jul 20 10:24:03 2004
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Tue Jul 20 11:19:13 2004
Subject: [Python-Dev] Re: Thread safety of deques
Message-ID: <16E1010E4581B049ABC51D4975CEDB88052CB2B3@UKDCX001.uk.int.atosorigin.com>

From: Aahz
> On Mon, Jul 19, 2004, Phillip J. Eby wrote:
>>
>> However, anything that calls back into Python code isn't necessarily 
>> atomic, and in general, almost anything can call back into Python code.
>
> Yup.

>> For example, the seemingly harmless dictionary item assignment can call 
>> back into Python for a custom type's __hash__ and __eq__ methods, so could 
>> it be that a thread switch could actually occur during this statement:
>> 
>>     foo[bar] = baz
>> 
>> if 'bar' has such custom methods?  Maybe I'm wrong, and the GIL is still 
>> held in such a case, but I'm not sure *how*.

> No, it isn't.

Hmm. I think that makes the point fairly well - thread safety is subtle, and
not to be taken lightly. Whether that was Raymond's point or mine, I'm less
sure :-)

I'll take Raymond's suggestion and see if I can come up with a relatively
succinct description of thread safety which would be suitable as a glossary
entry.

Paul.


__________________________________________________________________________
This e-mail and the documents attached are confidential and intended 
solely for the addressee; it may also be privileged. If you receive this 
e-mail in error, please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos Origin group 
liability cannot be triggered for the message content. Although the 
sender endeavours to maintain a computer virus-free network, the sender 
does not warrant that this transmission is virus-free and will not be 
liable for any damages resulting from any virus transmitted.
__________________________________________________________________________
From P at draigBrady.com  Tue Jul 20 13:24:34 2004
From: P at draigBrady.com (P@draigBrady.com)
Date: Tue Jul 20 13:25:01 2004
Subject: [Python-Dev] Re: PEP 331: Locale-Independent Float/String
	Conversions
In-Reply-To: <mailman.588.1090262695.5135.python-list@python.org>
References: <mailman.588.1090262695.5135.python-list@python.org>
Message-ID: <40FD00F2.3050408@draigBrady.com>

Christian Robottom Reis wrote:
> The LC_NUMERIC PEP has just been allocated a number, PEP 331. 
> 
>     http://www.python.org/peps/pep-0331.html
> 
> Though the code has been integrated, comments and corrections to the PEP
> would be appreciated -- in particular, Tim might have some comments on
> the thread-safety issues, Gustavo, about the implementation or Martin,
> about the actual constraints on the change itself.

Oh cool. Since we're now setting LC_NUMERIC at the glibc level
can we fix up locale.format. It's is very simplistic at the moment
as it groups %s items as numbers:
     locale.format("%s",1234,1) -> '1,234'
and treats non numbers as numbers:
      locale.format("%s\n",1234,1) -> '12,34\n'

This could be fixed by only allowing numbers in?
Better though would be to pass things down to glibc's snprintf
I.E. use the glibc facility of the ' modifier. e.g "%'d"
This applies to any decimal conversion (i,d,u,f,F,g,G)
You wouldn't have to use it directly, instead in locale.format()
you could add in a ' in the appropriate places.
Note the ' formating mechanism is SUSV2 not just glibc

P?draig.
From mwh at python.net  Tue Jul 20 16:15:27 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Jul 20 16:15:29 2004
Subject: [Python-Dev] Proper tail recursion
In-Reply-To: <875c7e07040719124460a2e7d5@mail.gmail.com> (Chris King's
	message of "Mon, 19 Jul 2004 15:44:49 -0400")
References: <1090264643.40fc1e4311be2@mcherm.com>
	<875c7e07040719124460a2e7d5@mail.gmail.com>
Message-ID: <2mk6wysr28.fsf@starship.python.net>

Chris King <colanderman@gmail.com> writes:

> On Mon, 19 Jul 2004 12:17:23 -0700, Michael Chermside <mcherm@mcherm.com> wrote:
>
>> As one of the people who has been defending this proposal, let me
>> now suggest that we just drop the idea.
>
> Agreed.  Thanks to everyone who helped defend it though, and thanks to
> others for providing obective feedback!
>
>> Your original change can be kept around as a patch that people can apply
>> for themselves if they want to. I expect that anyone who *really*
>> cared would be using stackless anyway.
>
> I think it would actually even be possible to turn it into an
> extension module, but that's a project for another day ;)

I think Armin Rigo has already done that with 'greenlets':

http://www.stackless.com/pipermail/stackless-dev/2004-March/000057.html

I don't pretend to understand...

Cheers,
mwh

-- 
58. Fools ignore complexity. Pragmatists suffer it. Some can avoid
    it. Geniuses remove it.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html
From mwh at python.net  Tue Jul 20 16:20:01 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Jul 20 16:20:05 2004
Subject: [Python-Dev] Re: Thread safety of deques
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB88052CB2B3@UKDCX001.uk.int.atosorigin.com>
	(Paul Moore's message of "Tue, 20 Jul 2004 09:24:03 +0100")
References: <16E1010E4581B049ABC51D4975CEDB88052CB2B3@UKDCX001.uk.int.atosorigin.com>
Message-ID: <2mfz7msqum.fsf@starship.python.net>

"Moore, Paul" <Paul.Moore@atosorigin.com> writes:

> From: Aahz
>> On Mon, Jul 19, 2004, Phillip J. Eby wrote:
>>>
>>> However, anything that calls back into Python code isn't necessarily 
>>> atomic, and in general, almost anything can call back into Python code.
>>
>> Yup.
>
>>> For example, the seemingly harmless dictionary item assignment can call 
>>> back into Python for a custom type's __hash__ and __eq__ methods, so could 
>>> it be that a thread switch could actually occur during this statement:
>>> 
>>>     foo[bar] = baz
>>> 
>>> if 'bar' has such custom methods?  Maybe I'm wrong, and the GIL is still 
>>> held in such a case, but I'm not sure *how*.
>
>> No, it isn't.
>
> Hmm. I think that makes the point fairly well - thread safety is subtle, and
> not to be taken lightly. Whether that was Raymond's point or mine, I'm less
> sure :-)

To be fair, you don't need threads to tickle very odd behaviour here
-- check out vicious_descriptor_nonsense in test_descr or test_mutants
or some stuff in test_sort.  Threads just add that delicious air of
unpredictability to the mix :-)

Cheers,
mwh

-- 
  To summarise the summary of the summary:- people are a problem.
                   -- The Hitch-Hikers Guide to the Galaxy, Episode 12
From martin at v.loewis.de  Tue Jul 20 18:47:40 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Jul 20 18:47:46 2004
Subject: [Python-Dev] Re: PEP 331: Locale-Independent
	Float/String	Conversions
In-Reply-To: <40FD00F2.3050408@draigBrady.com>
References: <mailman.588.1090262695.5135.python-list@python.org>
	<40FD00F2.3050408@draigBrady.com>
Message-ID: <40FD4CAC.9030000@v.loewis.de>

P@draigBrady.com wrote:
> Oh cool. Since we're now setting LC_NUMERIC at the glibc level
> can we fix up locale.format. It's is very simplistic at the moment
> as it groups %s items as numbers:
>     locale.format("%s",1234,1) -> '1,234'
> and treats non numbers as numbers:
>      locale.format("%s\n",1234,1) -> '12,34\n'

Indeed. The second one is incorrect usage of the function.

Feel free to submit patches. However, make sure they work on all
relevant platforms.

Regards,
Martin
From colanderman at gmail.com  Tue Jul 20 19:13:49 2004
From: colanderman at gmail.com (Chris King)
Date: Tue Jul 20 19:13:51 2004
Subject: [Python-Dev] sre.split question
Message-ID: <875c7e07040720101360ee8b7e@mail.gmail.com>

I'm curious as to this bit of code in pattern_split() in Modules/_sre.c:

        if (state.start == state.ptr) {
            if (last == state.end)
                break;
            /* skip one character */
            state.start = (void*) ((char*) state.ptr + state.charsize);
            continue;
        }

This precludes use of patterns that can successfully match zero-length
strings (e.g. r'(?<=[A-Za-z])(?=[^A-Za-z])'.  Skipping one character
is of course the correct behaviour, but what purpose do the break and
continue serve?  The only one I can think of is to stop silly patterns
like r'\s*' from returning a list of characters, but there may be
other reasons I haven't thought of.

(Yes, I know I can get the effect I want by using finditer() ;))
From mkc+dated+1091668631.4741a6 at mathdogs.com  Wed Jul 21 03:24:24 2004
From: mkc+dated+1091668631.4741a6 at mathdogs.com (Mike Coleman)
Date: Wed Jul 21 03:30:37 2004
Subject: [Python-Dev] Re: sre.split question
References: <875c7e07040720101360ee8b7e@mail.gmail.com>
Message-ID: <lkd658ijgon.fsf@panix2.panix.com>

Chris King <colanderman@gmail.com> writes:
> This precludes use of patterns that can successfully match zero-length
> strings (e.g. r'(?<=[A-Za-z])(?=[^A-Za-z])'.

As you say, it's very useful to be able to match such zero-length patterns.  I
posted a patch that fixes this, but have not heard any comments on it.  Please
take a look and post a comment if you like.

   https://sourceforge.net/tracker/index.php?func=detail&aid=988761&group_id=5470&atid=305470

Mike

From kbk at shore.net  Wed Jul 21 05:19:12 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed Jul 21 05:19:15 2004
Subject: [Python-Dev] Fun with whitespace
References: <20040718061309.DCAA01E4002@bag.python.org>
Message-ID: <87bria6o9b.fsf@hydra.localdomain>

"Tim Peters" <tim.one@comcast.net> writes:

> Heads up!  Using reindent.py to normalize whitespace in the Tools/scripts
> directory uncovered a file with bad syntax problems.  So I repaired that
> file's errors, then ran reindent over the entire source tree.

I'm thinking that it might be a good idea to modify IDLE to use
four-space indenting only, and remove the Tab options from the
configuration dialog.

Right now, every time a program is Run/F5, IDLE runs a version of
tabnanny on it and then runs a syntax check.  We could add a
reindent.py check after tabnanny (if tabnanny fails, the user is notified
to fix things first).  Indents would be normalized at 4 spaces, and if
an error was detected, a dialog would warn the user that the file
would be changed.  The user could confirm (auto fix) or cancel and fix
things manually.

I've tried to use IDLE to convert tab-indented code to space-indented,
but it's never worked well for me.  If you set the wrong number of
spaces/tab the code gets scrambled.  reindent.py seems to work fine.
Are there situations in which it can fail, assuming tabnanny is happy?

The implementation would continue to create the .bak file.  Alternatively,
maybe the undo capability could be extended to cover the reindent.  Or
both.

Is it time for IDLE to enforce four-space indentation?  Any opinions?
-- 
KBK
From tim.peters at gmail.com  Wed Jul 21 05:52:11 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Wed Jul 21 05:52:14 2004
Subject: [Python-Dev] Fun with whitespace
In-Reply-To: <87bria6o9b.fsf@hydra.localdomain>
References: <20040718061309.DCAA01E4002@bag.python.org>
	<87bria6o9b.fsf@hydra.localdomain>
Message-ID: <1f7befae0407202052214ba6b@mail.gmail.com>

[Kurt B. Kaiser]
> I'm thinking that it might be a good idea to modify IDLE to use
> four-space indenting only, and remove the Tab options from the
> configuration dialog.

The conceptual disconnect here comes from the IDLE shell window, which
still defaults to using hard tabs.  This appears to be a consequence
of setting (the conceptual) sys.PS2 to an empty string but (the
conceptual) sys.PS1 to a 4-character string, so that continued lines
"don't look right" in the shell unless hard tabs are used.  I dislike
this so much that I avoid using the shell window most days -- I'd
rather it looked "just like" a command-line shell.

> Right now, every time a program is Run/F5, IDLE runs a version of
> tabnanny on it and then runs a syntax check.  We could add a
> reindent.py check after tabnanny (if tabnanny fails, the user is notified
> to fix things first).  Indents would be normalized at 4 spaces, and if
> an error was detected, a dialog would warn the user that the file
> would be changed.  The user could confirm (auto fix) or cancel and fix
> things manually.

I know Guido wants IDLE to whine whenever it's about to save a code
file, but that dialog is a PITA.  I sure don't want two of 'em!  I
vote for a silent revolution, where you add a secret "don't whine at
me about any changes you make when I hit F5 -- just do it and leave me
alone" option that only I know about <wink>.

> I've tried to use IDLE to convert tab-indented code to space-indented,
> but it's never worked well for me.  If you set the wrong number of
> spaces/tab the code gets scrambled.  reindent.py seems to work fine.
> Are there situations in which it can fail, assuming tabnanny is happy?

None are known.  All of Python has been run through it now <wink>, and
all of Zope3 was run through it many months ago w/o problems.  If you
*don't* check the syntax first, it can be baffling, because
tokenize.py can be baffling.  For example, if you have an extra left
paren on some line, tokenize is likely to produce an incomprehensible
exception at the end of the file, claiming that it hit EOF in an
unterminated string.

> The implementation would continue to create the .bak file.  Alternatively,
> maybe the undo capability could be extended to cover the reindent.  Or
> both.
>
> Is it time for IDLE to enforce four-space indentation?  Any opinions?

4-space is all I ever use -- except in the shell window.  So have to
leave objections to someone insane ...
From kbk at shore.net  Wed Jul 21 06:59:26 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed Jul 21 06:59:46 2004
Subject: [Python-Dev] Fun with whitespace
References: <20040718061309.DCAA01E4002@bag.python.org>
	<87bria6o9b.fsf@hydra.localdomain>
	<1f7befae0407202052214ba6b@mail.gmail.com>
Message-ID: <87y8le551t.fsf@hydra.localdomain>

Tim Peters <tim.peters@gmail.com> writes:

> [Kurt B. Kaiser]
>> I'm thinking that it might be a good idea to modify IDLE to use
>> four-space indenting only, and remove the Tab options from the
>> configuration dialog.
>
> The conceptual disconnect here comes from the IDLE shell window, which
> still defaults to using hard tabs.  This appears to be a consequence
> of setting (the conceptual) sys.PS2 to an empty string but (the
> conceptual) sys.PS1 to a 4-character string, so that continued lines
> "don't look right" in the shell unless hard tabs are used.  I dislike
> this so much that I avoid using the shell window most days -- I'd
> rather it looked "just like" a command-line shell.

The thing that bothers me about the command-line shell is the first
indent is four characters in from the "def", and subsequent indents
are eight.

IDLE lines up code the same way, except without the leading '...'.
First an apparent four character indent, then subsequent eight
character indents, using tabs.

I remember that you previously remarked that it would be useful to be
able to cut/paste from the shell to an Edit window.  I agree.  Right
now it's a disaster because of the '>>> ", the strange indentation,
and the tabs, which causes tabnanny nag.  Then the unfortunate user
gets to play with 'untabify region', and if he doesn't watch out
(i.e. convert the eight character tabs to four spaces) a real snarl
results.

What I'd like is for the shell to look the same as an Edit window.
This would imply replacing the '>>>' with a prompt column on the left
which was not part of the text region.  I haven't investigated that,
but it would seem to involve a Tkinter/Tk modification.  There are
other editors which take this approach, which also allows the
cut/paste and the use of spaces in the Shell.  But it may not be
possible without modifying Tk, which is probably not desirable.

Alternatively, PS2 could be four spaces (or do you all prefer the
dots?)  and indents could be four spaces.  It seems that cut/paste
would work if the reindent first magically replaced '^>>> ' (and
'^...') with ''.

>> Right now, every time a program is Run/F5, IDLE runs a version of
>> tabnanny on it and then runs a syntax check.  We could add a
>> reindent.py check after tabnanny (if tabnanny fails, the user is notified
>> to fix things first).  Indents would be normalized at 4 spaces, and if
>> an error was detected, a dialog would warn the user that the file
>> would be changed.  The user could confirm (auto fix) or cancel and fix
>> things manually.
>
> I know Guido wants IDLE to whine whenever it's about to save a code
> file, but that dialog is a PITA.  

I'm happy to report that there is now an autosave option.  It's set
off by default, but I think it works great.  With the undo feature,
there really isn't any reason not to use it.  Code is saved and run
in a fresh environment with a single F5.

> I sure don't want two of 'em!  I vote for a silent revolution, where
> you add a secret "don't whine at me about any changes you make when
> I hit F5 -- just do it and leave me alone" option that only I know
> about <wink>.

If reindent can't damage user code, that would be OK by me.

I'd say also get rid of 'untabify' and let the reindent handle it.

>> I've tried to use IDLE to convert tab-indented code to space-indented,
>> but it's never worked well for me.  If you set the wrong number of
>> spaces/tab the code gets scrambled.  reindent.py seems to work fine.
>> Are there situations in which it can fail, assuming tabnanny is happy?
>
> None are known.  All of Python has been run through it now <wink>, and
> all of Zope3 was run through it many months ago w/o problems.  If you
> *don't* check the syntax first, it can be baffling, because
> tokenize.py can be baffling.  For example, if you have an extra left
> paren on some line, tokenize is likely to produce an incomprehensible
> exception at the end of the file, claiming that it hit EOF in an
> unterminated string.

OK, so the sequence would be: tabnanny, syntax check, reindent.

>> The implementation would continue to create the .bak file.  Alternatively,
>> maybe the undo capability could be extended to cover the reindent.  Or
>> both.
>>
>> Is it time for IDLE to enforce four-space indentation?  Any opinions?
>
> 4-space is all I ever use -- except in the shell window.  So have to
> leave objections to someone insane ...

:-)   Do people have issues with a 4-space indented Shell?

-- 
KBK
From kbk at shore.net  Wed Jul 21 07:15:19 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed Jul 21 07:15:22 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200407210515.i6L5FJQ9011868@hydra.localdomain>


Patch / Bug Summary
___________________

Patches :  275 open ( -3) /  2470 closed (+12) /  2745 total ( +9)
Bugs    :  750 open (-11) /  4281 closed (+51) /  5031 total (+40)
RFE     :  145 open ( +0) /   130 closed ( +1) /   275 total ( +1)

New / Reopened Patches
______________________

socketmodule on OpenBSD/sparc64 (64bit machine)  (2004-07-16)
CLOSED http://python.org/sf/992055  opened by  Aleksander Piotrowski 

zipfile and big zipped file  (2004-07-17)
       http://python.org/sf/992750  opened by  Keyphrene 

cgi.parse_qs: make an empty string give {}, like in 2.3.  (2004-07-17)
CLOSED http://python.org/sf/993062  opened by  Richie Hindle 

audioop extension should be enabled on 64-bit archs  (2004-07-18)
CLOSED http://python.org/sf/993173  opened by  Matthias Klose 

audioop extension should be enabled on 64-bit archs  (2004-07-18)
CLOSED http://python.org/sf/993189  opened by  Matthias Klose 

Fix docs for string/rstrip  (2004-07-18)
CLOSED http://python.org/sf/993187  opened by  Matthias Klose 

Unicode in distutils.core.setup()  (2004-07-19)
       http://python.org/sf/993943  opened by  Walter Dörwald 

faster os.walk  (2004-07-19)
       http://python.org/sf/994057  opened by  Felix Lee 

HTTP basic authentication problem  (2004-07-20)
       http://python.org/sf/994595  opened by  Catalin Marinas 

Patches Closed
______________

AF_PACKET Hardware address support in socket module  (2004-05-03)
       http://python.org/sf/947352  closed by  akuchling

socketmodule on OpenBSD/sparc64 (64bit machine)  (2004-07-16)
       http://python.org/sf/992055  closed by  loewis

Add maketrans to string object  (2004-04-29)
       http://python.org/sf/943953  closed by  perky

pimp.py has old URL for default database  (2003-05-02)
       http://python.org/sf/731151  closed by  perky

socket module does not support AF_NETLINK  (2004-07-03)
       http://python.org/sf/984654  closed by  loewis

Py_HUGE_VAL HUGE_VAL HUGE undefined  (2004-06-08)
       http://python.org/sf/968728  closed by  perky

cgi.parse_qs: make an empty string give {}, like in 2.3.  (2004-07-17)
       http://python.org/sf/993062  closed by  bcannon

audioop extension should be enabled on 64-bit archs  (2004-07-18)
       http://python.org/sf/993173  closed by  loewis

audioop extension should be enabled on 64-bit archs  (2004-07-18)
       http://python.org/sf/993189  closed by  nnorwitz

Fix docs for string/rstrip  (2004-07-18)
       http://python.org/sf/993187  closed by  loewis

fix for bug #812325 (tarfile violates bufsize)  (2003-11-21)
       http://python.org/sf/846659  closed by  nnorwitz

fix for bug #857297 (tarfile and hardlinks)  (2004-03-15)
       http://python.org/sf/916874  closed by  nnorwitz

New / Reopened Bugs
___________________

os.path.realpath() does not handle symlinks correctly  (2004-07-14)
       http://python.org/sf/990669  opened by  Anton Koinov 

Crash when importing encoded file  (2004-07-14)
       http://python.org/sf/990743  opened by  Jurjen N.E. Bos 

os constants missing  (2004-07-14)
       http://python.org/sf/990749  opened by  Connelly 

sre.sub documentation is incomplete  (2004-07-14)
       http://python.org/sf/990792  opened by  Laurent Pinchart 

Crash from Rapid Clicks  (2004-07-14)
       http://python.org/sf/990911  opened by  Karl 

Uninstall Python shortcut does not work  (2004-07-14)
CLOSED http://python.org/sf/990941  opened by  Thomas Heller 

python24.dll installation problems  (2004-07-14)
       http://python.org/sf/990945  opened by  Thomas Heller 

2.4 msi installer installs too many files  (2004-07-14)
CLOSED http://python.org/sf/990953  opened by  Thomas Heller 

IDLE displays wrong help pages on Windows  (2004-07-14)
CLOSED http://python.org/sf/990954  opened by  Thomas Heller 

staticmethod not in index  (2004-07-14)
CLOSED http://python.org/sf/991123  opened by  George Young 

Make Problem on HPUX  (2004-07-14)
       http://python.org/sf/991125  opened by  Paul D. Lusk 

An inconsistency with nested scopes  (2004-07-14)
       http://python.org/sf/991196  opened by  Nick Jacobson 

Cookie.py loses 'secure' attribute  (2004-07-15)
CLOSED http://python.org/sf/991258  opened by  Stuart Bishop 

Cookie.py does not correctly quote Morsels  (2004-07-15)
       http://python.org/sf/991266  opened by  Stuart Bishop 

test_imp failure  (2004-07-15)
       http://python.org/sf/991708  opened by  Jim Fulton 

os.access reports true for read-only directories  (2004-07-15)
       http://python.org/sf/991735  opened by  Mark Moales 

_bsddb segfault  (2004-07-15)
       http://python.org/sf/991754  opened by  Jim Fulton 

Documentation of DOMImplmentation lacking  (2004-07-15)
       http://python.org/sf/991805  opened by  Robert M. Zigweid 

PyArg_ParseTuple can miss errors with warnings as exceptions  (2004-07-15)
       http://python.org/sf/991812  opened by  mssf 

Warning in cjkcodecs/iso2022common.h  (2004-07-15)
       http://python.org/sf/991834  opened by  Jack Jansen 

PyObject_CallMethod docs unclear (with dangerous results)  (2004-07-15)
       http://python.org/sf/991883  opened by  Itamar Shtull-Trauring 

Building with --disable-toolbox-glue fails  (2004-07-16)
       http://python.org/sf/991962  opened by  Jack Jansen 

code that generates a segfault on Python 2.1-2.3  (2004-07-15)
       http://python.org/sf/992017  opened by  Ted Bach 

test_format fails 2.4a1  (2004-07-16)
       http://python.org/sf/992078  opened by  roadkill 

test_unicode fails 2.4a1  (2004-07-16)
CLOSED http://python.org/sf/992081  opened by  roadkill 

test_macostools fails when running from source  (2004-07-16)
       http://python.org/sf/992185  opened by  Jack Jansen 

exec statement balks at CR/LF  (2004-07-16)
       http://python.org/sf/992207  opened by  Konrad Hinsen 

attribute error after non-from import  (2004-07-16)
       http://python.org/sf/992389  opened by  Jim Fulton 

variable reuse in the logging module  (2004-07-16)
       http://python.org/sf/992397  opened by  Gintautas Miliauskas 

test_dummy_threading vs -O  (2004-07-18)
CLOSED http://python.org/sf/993394  opened by  Tim Peters 

inspect.findsource does not call linecache.checkcache  (2004-07-18)
       http://python.org/sf/993580  opened by  Jason Mobarak 

optparse libref documentation missing "ref"  (2004-07-19)
       http://python.org/sf/993601  opened by  Richard Jones 

bdist_dumb and --relative on Windows fails  (2004-07-19)
       http://python.org/sf/993766  opened by  Mark Hammond 

threads duplicated on fork() prevent child from terminating  (2004-07-19)
       http://python.org/sf/994023  opened by  L.C. 

seg fault when calling bsddb.hashopen()  (2004-07-19)
       http://python.org/sf/994100  opened by  Skip Montanaro 

urllib2: improper capitalization of headers  (2004-07-19)
       http://python.org/sf/994101  opened by  Robert Sayre 

Py_RETURN_NONE causes too much warnings  (2004-07-20)
       http://python.org/sf/994255  opened by  Correa 

logging module: default levels do not work as documented.  (2004-07-20)
       http://python.org/sf/994416  opened by  Roy Smith 

logging module prints unexpected message when no handler set  (2004-07-20)
       http://python.org/sf/994421  opened by  Roy Smith 

typo in time.tzset's docstring  (2004-07-20)
CLOSED http://python.org/sf/994580  opened by  Marius Gedminas 

fcntl example is broken  (2004-07-20)
CLOSED http://python.org/sf/994605  opened by  Chris Green 

Bugs Closed
___________

Python-2.4.0a1 test_strptime.py fails on HP-UX11i  (2004-07-12)
       http://python.org/sf/989273  closed by  bcannon

bsddb/weakref error  (2004-07-07)
       http://python.org/sf/986528  closed by  montanaro

Py_BuildValue loses reference counts on error  (2004-07-03)
       http://python.org/sf/984722  closed by  mwh

Uninstall Python shortcut does not work  (2004-07-14)
       http://python.org/sf/990941  closed by  loewis

Python Logging filename & file number   (2004-06-07)
       http://python.org/sf/968245  closed by  vsajip

2.4 msi installer installs too many files  (2004-07-14)
       http://python.org/sf/990953  closed by  theller

IDLE displays wrong help pages on Windows  (2004-07-14)
       http://python.org/sf/990954  closed by  kbk

gethostbyname is broken on hpux 11.11  (2004-06-25)
       http://python.org/sf/979967  closed by  ehab_teima

staticmethod not in index  (2004-07-14)
       http://python.org/sf/991123  closed by  rhettinger

Cookie.py loses 'secure' attribute  (2004-07-15)
       http://python.org/sf/991258  closed by  zenzen

IDLE docs broken on OSX  (2004-02-19)
       http://python.org/sf/900580  closed by  kbk

Different behaviour using super() and explicit base class.  (2004-04-11)
       http://python.org/sf/933068  closed by  loewis

OSATerminology is broken  (2004-01-25)
       http://python.org/sf/884085  closed by  jackjansen

can't compile _localemodule.c w/o --enable-toolbox-glue  (2004-06-23)
       http://python.org/sf/978662  closed by  bcannon

CFStringGetUnicode() returns null-terminated unicode string  (2004-03-14)
       http://python.org/sf/915942  closed by  jackjansen

MacOS module doesn't compile on 10.1  (2003-08-01)
       http://python.org/sf/781433  closed by  jackjansen

bgenlocationscustomize: per-user configuration of bgen  (2004-01-05)
       http://python.org/sf/870997  closed by  jackjansen

Standardize on systemwide bgenlocations INCLUDEDIR  (2004-01-05)
       http://python.org/sf/870972  closed by  jackjansen

bgenlocations.py only works for OS9 users and jack ;)  (2003-07-28)
       http://python.org/sf/779151  closed by  jackjansen

MacPython-OS9 distutils breaks on OSX  (2003-05-20)
       http://python.org/sf/740424  closed by  jackjansen

MacPython-OS9 eats CPU while waiting for I/O  (2003-04-29)
       http://python.org/sf/729871  closed by  jackjansen

Mac/OSX/README outdated  (2003-06-20)
       http://python.org/sf/757567  closed by  jackjansen

PackMan needs to be able to install outside site-packages  (2003-06-28)
       http://python.org/sf/762147  closed by  jackjansen

bundlebuilder --lib mishandles frameworks  (2004-01-23)
       http://python.org/sf/883283  closed by  jackjansen

Windows installer missing test files  (2004-07-11)
       http://python.org/sf/988784  closed by  loewis

Framework Info.plist is out of date  (2003-12-08)
       http://python.org/sf/856407  closed by  jackjansen

IDE textwindow scrollbar is over-enthusiastic  (2003-03-30)
       http://python.org/sf/711989  closed by  jackjansen

Definitive way to link to the correct Python.framework  (2003-12-08)
       http://python.org/sf/856401  closed by  jackjansen

printing date causes crash  (2004-07-14)
       http://python.org/sf/990569  closed by  mhammond

Unloading extension modules not always safe  (2003-11-07)
       http://python.org/sf/838140  closed by  jackjansen

pydoc crash on MacOS X  (2003-11-25)
       http://python.org/sf/848907  closed by  jackjansen

Recommend TeachText in OSX documentation  (2003-08-27)
       http://python.org/sf/795649  closed by  jackjansen

New mandatory 2nd argument to site.addsitedir breaks compat  (2004-07-07)
       http://python.org/sf/986795  closed by  bcannon

test_unicode fails 2.4a1  (2004-07-16)
       http://python.org/sf/992081  closed by  loewis

IDE has dirty sys.argv  (2003-07-03)
       http://python.org/sf/765603  closed by  jackjansen

Carbon.CF module needs new style classes  (2003-03-03)
       http://python.org/sf/696527  closed by  jackjansen

Investigate weak linking  (2002-12-14)
       http://python.org/sf/653871  closed by  jackjansen

Mac/OSX/Makefile assumes case-insensitive build  (2003-07-05)
       http://python.org/sf/766210  closed by  jackjansen

MacPython installer: disk image does not mount from NFS  (2003-11-11)
       http://python.org/sf/839865  closed by  jackjansen

urllib proxy handling incomplete for MacOSX  (2003-07-28)
       http://python.org/sf/779167  closed by  jackjansen

PythonLauncher has empty popup for interpreters  (2003-07-22)
       http://python.org/sf/775878  closed by  jackjansen

bundlebuilder Info.plist files.  (2003-07-04)
       http://python.org/sf/765903  closed by  jackjansen

hardcoded python paths  (2003-07-09)
       http://python.org/sf/768391  closed by  jackjansen

mistyped example  (2004-07-14)
       http://python.org/sf/990497  closed by  perky

CJK codecs list incomplete  (2004-06-09)
       http://python.org/sf/969415  closed by  perky

test_dummy_threading vs -O  (2004-07-18)
       http://python.org/sf/993394  closed by  bcannon

cgi.parse_qsl broken  (2004-07-13)
       http://python.org/sf/990307  closed by  nascheme

float_subtype_new() bug  (2004-06-04)
       http://python.org/sf/966618  closed by  nascheme

typo in time.tzset's docstring  (2004-07-20)
       http://python.org/sf/994580  closed by  nnorwitz

fcntl example is broken  (2004-07-20)
       http://python.org/sf/994605  closed by  nnorwitz

Tarfile and hard-links  (2003-12-09)
       http://python.org/sf/857297  closed by  nnorwitz

FutureWarning in Carbon/Controls.py  (2003-07-28)
       http://python.org/sf/779147  closed by  kasplat

New / Reopened RFE
__________________

array.array objects should support sequences  (2004-07-17)
       http://python.org/sf/992967  opened by  Bob Ippolito 

RFE Closed
__________

add POST support to webbrowser module  (2002-01-02)
       http://python.org/sf/498855  closed by  kasplat

From mwh at python.net  Wed Jul 21 12:38:12 2004
From: mwh at python.net (Michael Hudson)
Date: Wed Jul 21 12:38:13 2004
Subject: [Python-Dev] Fun with whitespace
In-Reply-To: <87y8le551t.fsf@hydra.localdomain> (Kurt B. Kaiser's message of
	"Wed, 21 Jul 2004 00:59:26 -0400")
References: <20040718061309.DCAA01E4002@bag.python.org>
	<87bria6o9b.fsf@hydra.localdomain>
	<1f7befae0407202052214ba6b@mail.gmail.com>
	<87y8le551t.fsf@hydra.localdomain>
Message-ID: <2msmblodbf.fsf@starship.python.net>

kbk@shore.net (Kurt B. Kaiser) writes:

> Alternatively, PS2 could be four spaces (or do you all prefer the
> dots?)  and indents could be four spaces.  It seems that cut/paste
> would work if the reindent first magically replaced '^>>> ' (and
> '^...') with ''.

Is it possible to have copy/paste see the text w/o prompts?

Cheers,
mwh

-- 
  I think perhaps we should have electoral collages and construct
  our representatives entirely of little bits of cloth and papier 
  mache.          -- Owen Dunn, ucam.chat, from his review of the year
From mcherm at mcherm.com  Wed Jul 21 13:56:57 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed Jul 21 13:56:14 2004
Subject: [Python-Dev] Fun with whitespace
Message-ID: <1090411017.40fe5a09111ad@mcherm.com>

Kurt writes:
> Alternatively, PS2 could be four spaces (or do you all prefer the
> dots?)  and indents could be four spaces.  It seems that cut/paste
> would work if the reindent first magically replaced '^>>> ' (and
> '^...') with ''.

Michael Hudson replies:
> Is it possible to have copy/paste see the text w/o prompts?

I agree. After all, I often paste my code into someplace OTHER than
IDLE.

-- Michael Chermside

From jepler at unpythonic.net  Wed Jul 21 13:59:29 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Wed Jul 21 13:59:51 2004
Subject: [Python-Dev] Fun with whitespace
In-Reply-To: <2msmblodbf.fsf@starship.python.net>
References: <20040718061309.DCAA01E4002@bag.python.org>
	<87bria6o9b.fsf@hydra.localdomain>
	<1f7befae0407202052214ba6b@mail.gmail.com>
	<87y8le551t.fsf@hydra.localdomain>
	<2msmblodbf.fsf@starship.python.net>
Message-ID: <20040721115929.GA8232@unpythonic.net>

On Wed, Jul 21, 2004 at 11:38:12AM +0100, Michael Hudson wrote:
> kbk@shore.net (Kurt B. Kaiser) writes:
> 
> > Alternatively, PS2 could be four spaces (or do you all prefer the
> > dots?)  and indents could be four spaces.  It seems that cut/paste
> > would work if the reindent first magically replaced '^>>> ' (and
> > '^...') with ''.
> 
> Is it possible to have copy/paste see the text w/o prompts?
> 
> Cheers,
> mwh

This should be possible, through use of Tk's "selection handle" command,
available as Tkinter.Misc.selection_handle() in Python.  I've never used
this, though, under plain tk or under Python.

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040721/d20d153d/attachment.pgp
From gmccaughan at synaptics-uk.com  Wed Jul 21 16:05:37 2004
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Wed Jul 21 16:06:09 2004
Subject: [Python-Dev] Fun with whitespace
In-Reply-To: <87bria6o9b.fsf@hydra.localdomain>
References: <20040718061309.DCAA01E4002@bag.python.org>
	<87bria6o9b.fsf@hydra.localdomain>
Message-ID: <200407211505.37587.gmccaughan@synaptics-uk.com>

On Wednesday 2004-07-21 04:19, Kurt B. Kaiser wrote:

> I'm thinking that it might be a good idea to modify IDLE to use
> four-space indenting only, and remove the Tab options from the
> configuration dialog.
...
> Is it time for IDLE to enforce four-space indentation?  Any opinions?

Great idea. While you're at it, it should also enforce
StudlyCaps class names, refuse to save files containing
recursive functions, and do automatic grammar checking
on your comments. Heaven forbid that any individual or
organization should have a coding style different from
the officially sanctioned norm.

More concisely: Please, please, *no*.

-- 
Gareth McCaughan

From Jack.Jansen at cwi.nl  Wed Jul 21 16:18:34 2004
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Wed Jul 21 16:17:32 2004
Subject: [Python-Dev] Fun with whitespace
In-Reply-To: <1090411017.40fe5a09111ad@mcherm.com>
References: <1090411017.40fe5a09111ad@mcherm.com>
Message-ID: <DA01879C-DB20-11D8-B1B5-000A958D1666@cwi.nl>


On 21-jul-04, at 13:56, Michael Chermside wrote:

> Kurt writes:
>> Alternatively, PS2 could be four spaces (or do you all prefer the
>> dots?)  and indents could be four spaces.  It seems that cut/paste
>> would work if the reindent first magically replaced '^>>> ' (and
>> '^...') with ''.
>
> Michael Hudson replies:
>> Is it possible to have copy/paste see the text w/o prompts?
>
> I agree. After all, I often paste my code into someplace OTHER than
> IDLE.

But it makes a difference where you're pasting it: in an editor window 
(where you don't want the >>>) or in an email message (where you 
probably do want the >>>).
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2086 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040721/442cdd1d/smime.bin
From aahz at pythoncraft.com  Wed Jul 21 19:58:09 2004
From: aahz at pythoncraft.com (Aahz)
Date: Wed Jul 21 19:58:12 2004
Subject: [Python-Dev] More joy with test_strptime
In-Reply-To: <40F2DE5B.7040806@ocf.berkeley.edu>
References: <20040712002612.39A4D1E4002@bag.python.org>
	<40F1E5CC.1000001@ocf.berkeley.edu>
	<1f7befae0407111852780f4fc2@mail.gmail.com>
	<1f7befae04071120025e3f3233@mail.gmail.com>
	<1f7befae04071120245dedbd21@mail.gmail.com>
	<40F2D8B0.2020004@ocf.berkeley.edu>
	<1f7befae04071211501fbab08f@mail.gmail.com>
	<40F2DE5B.7040806@ocf.berkeley.edu>
Message-ID: <20040721175809.GA29855@panix.com>

On Mon, Jul 12, 2004, Brett C. wrote:
>
> =)  Didn't realize I managed to fit that all into a single sentence. 
> How you guys can stand to allow me to continue to write the Summaries I 
> sometimes question when instances like this come about.

Nobody else wants to do the work, and the work is too important to let
not happen.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Barbara Boxer speaks for me:
http://buffaloreport.com/2004/040713.boxer.marriage.html
From tim at zope.com  Wed Jul 21 23:16:07 2004
From: tim at zope.com (Tim Peters)
Date: Wed Jul 21 23:16:44 2004
Subject: [Python-Dev] Would like to add Edward Loper as a Python developer
Message-ID: <20040721211609.8FA333B8038@smtp.zope.com>

Edward Loper runs the Natural Language Toolkit project on SF:

    http://nltk.sourceforge.net/

and the Epydoc project:

    http://epydoc.sourceforge.net/

More importantly <wink>, he's a heavy doctest user and has lots of ideas for
improving it.  doctest's implementation has grown unwieldy, and is in need
of refactoring.  Jim (Fulton), Edward and I intend to do a lot there over
the coming two weeks, and giving Edward commit privileges would simplify and
speed the process.

This is intended to be limited to doctest changes.  Are there objections?

If anyone else is interested in contributing to doctest surgery, let me
know.  A Wiki area might be a decent way to coordinate.  Ideas already on
the table include restructuring for plugability (e.g., there's no sane way
for you to take over doctest error reporting now, but restructuring to do
much more via (overridable) class methods is a sane alternative); new
gimmicks like notation for "there's a blank line here in this expected
output"; a sane way to get into the debugger upon a doctest failure (right
now, if you try, doctest is still swallowing stdout, so you can't see what
you're doing! Jeremy did some quick hacks around that for Zope3 already);
better support for writing "whole file" doctests (tutorial narratives
liberally illustrated with Python examples -- Zope3 is finding these very
helpful); and documenting all the additions that snuck into Python 2.3.

From vinay_sajip at red-dove.com  Wed Jul 21 23:40:56 2004
From: vinay_sajip at red-dove.com (Vinay Sajip)
Date: Wed Jul 21 23:42:55 2004
Subject: [Python-Dev] Logging with no handlers configured
Message-ID: <000701c46f6b$68e8fd60$652b6992@alpha>

Currently, if the logging module is used with no handlers configured for a
logger or its parents and you try to log events with that logger, a single
message is printed to sys.stderr:

No handlers could be found for logger <logger name>

It has been suggested that this puts out a spurious message when, for
whatever reason, a developer intentionally does not configure any
handlers. A good use case for this is when writing a library component - the
library may happily log events, and if the application using it doesn't use
logging or configure any handlers, the spurious message should not be
printed.

However, the message is there for a reason - to help spot logging
configuration problems, which can be a bit hard to debug if the logging
system
doesn't output anything at all.

To meet both requirements, I propose that a change be made as follows: the
missing handlers situation will only be notified if raiseExceptions is set.
The current default value will cause the missing handlers situation to be
notified (once) - i.e. the default behaviour will remain as is. If
raiseExceptions is unset, the logging system will not give any notification.

Another question - exactly how to make the notification - also needs
resolving; is printing a message to sys.stderr better or worse than raising
an exception? I could create a new  LoggingConfigurationException, an
instance of which would be thrown instead of printing the message. However,
this could break existing applications, so I'm canvassing opinions on this.

All feedback gratefully received,

Vinay Sajip


From bac at OCF.Berkeley.EDU  Thu Jul 22 01:19:27 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Jul 22 01:19:41 2004
Subject: [Python-Dev] More joy with test_strptime
In-Reply-To: <20040721175809.GA29855@panix.com>
References: <20040712002612.39A4D1E4002@bag.python.org>	<40F1E5CC.1000001@ocf.berkeley.edu>	<1f7befae0407111852780f4fc2@mail.gmail.com>	<1f7befae04071120025e3f3233@mail.gmail.com>	<1f7befae04071120245dedbd21@mail.gmail.com>	<40F2D8B0.2020004@ocf.berkeley.edu>	<1f7befae04071211501fbab08f@mail.gmail.com>	<40F2DE5B.7040806@ocf.berkeley.edu>
	<20040721175809.GA29855@panix.com>
Message-ID: <40FEF9FF.6020909@ocf.berkeley.edu>

Aahz wrote:
> On Mon, Jul 12, 2004, Brett C. wrote:
> 
>>=)  Didn't realize I managed to fit that all into a single sentence. 
>>How you guys can stand to allow me to continue to write the Summaries I 
>>sometimes question when instances like this come about.
> 
> 
> Nobody else wants to do the work, and the work is too important to let
> not happen.

Acceptability thanks to complacency!  =)  I feel so bureaucratic!

-Brett
From python at rcn.com  Wed Jul 21 13:33:34 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Jul 22 01:34:56 2004
Subject: [Python-Dev] More joy with test_strptime
In-Reply-To: <40FEF9FF.6020909@ocf.berkeley.edu>
Message-ID: <002101c46f16$8ef1ee40$261bc797@oemcomputer>

> >>=)  Didn't realize I managed to fit that all into a single sentence.
> >>How you guys can stand to allow me to continue to write the
Summaries I
> >>sometimes question when instances like this come about.
> >
> >
> > Nobody else wants to do the work, and the work is too important to
let
> > not happen.
> 
> Acceptability thanks to complacency!  =)  I feel so bureaucratic!

Take solace, with only one historian, there are no errors, everything
written becomes fact.


Raymond

From jhylton at gmail.com  Thu Jul 22 01:42:20 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Thu Jul 22 01:42:23 2004
Subject: [Python-Dev] Would like to add Edward Loper as a Python developer
In-Reply-To: <20040721211609.8FA333B8038@smtp.zope.com>
References: <20040721211609.8FA333B8038@smtp.zope.com>
Message-ID: <e8bf7a5304072116424d166035@mail.gmail.com>

On Wed, 21 Jul 2004 17:16:07 -0400, Tim Peters <tim@zope.com> wrote:
> Edward Loper runs the Natural Language Toolkit project on SF:
> 
>     http://nltk.sourceforge.net/
> 
> and the Epydoc project:
> 
>     http://epydoc.sourceforge.net/
> 
> More importantly <wink>, he's a heavy doctest user and has lots of ideas for
> improving it.  doctest's implementation has grown unwieldy, and is in need
> of refactoring.  Jim (Fulton), Edward and I intend to do a lot there over
> the coming two weeks, and giving Edward commit privileges would simplify and
> speed the process.

+1

I'd be happy to have Edward involved and I'm eager to see the results
of your work.
 
Jeremy
From tim.peters at gmail.com  Thu Jul 22 05:21:14 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Jul 22 05:21:18 2004
Subject: [Python-Dev] Would like to add Edward Loper as a Python developer
In-Reply-To: <20040721211609.8FA333B8038@smtp.zope.com>
References: <20040721211609.8FA333B8038@smtp.zope.com>
Message-ID: <1f7befae04072120212deeecde@mail.gmail.com>

Edward must not be trying hard enough:  nobody hates him <wink>.

I got 3 +1's in private, 1 in public, not counting Jim or me, and no
objections, so I just added Edward as a Python developer.

There's also a new, too-telegraphic Wiki page here:

    http://zope.org/Members/tim_one/DoctestIdeas

Anyone with a zope.org account(*) should be able to add comments
there, and/or edit it.  You're encouraged.


(*) If you don't have one, click the Join link in the upper right
corner.  It's free, and there are no bad consequences (e.g., you won't
even get Zope marketing email as a result -- although I'm not sure
whether that's intentional or just because Barry left before finishing
ZpamBlaster <heh>).
From hyeshik at gmail.com  Thu Jul 22 05:53:33 2004
From: hyeshik at gmail.com (Hye-Shik Chang)
Date: Thu Jul 22 05:53:59 2004
Subject: [Python-Dev] Would like to add Edward Loper as a Python developer
In-Reply-To: <1f7befae04072120212deeecde@mail.gmail.com>
References: <20040721211609.8FA333B8038@smtp.zope.com>
	<1f7befae04072120212deeecde@mail.gmail.com>
Message-ID: <4f0b69dc0407212053f041460@mail.gmail.com>

On Wed, 21 Jul 2004 23:21:14 -0400, Tim Peters <tim.peters@gmail.com> wrote:
> Edward must not be trying hard enough:  nobody hates him <wink>.
> 
> I got 3 +1's in private, 1 in public, not counting Jim or me, and no
> objections, so I just added Edward as a Python developer.
> 

Yay!  I'm a big fan of epydoc. :)
Welcome, Edward!


Hye-Shik
From walter at livinglogic.de  Thu Jul 22 12:26:37 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu Jul 22 12:26:56 2004
Subject: [Python-Dev] Logging with no handlers configured
In-Reply-To: <000701c46f6b$68e8fd60$652b6992@alpha>
References: <000701c46f6b$68e8fd60$652b6992@alpha>
Message-ID: <40FF965D.3040001@livinglogic.de>

Vinay Sajip wrote:

> Currently, if the logging module is used with no handlers configured for a
> logger or its parents and you try to log events with that logger, a single
> message is printed to sys.stderr:
> 
> No handlers could be found for logger <logger name>
> 
> It has been suggested that this puts out a spurious message when, for
> whatever reason, a developer intentionally does not configure any
> handlers.
 > [...]
>
> All feedback gratefully received,

How about a LoggingConfigurationWarning and using warnings.warn()?
Then what should happen with the warning (ignore, print, raise
exception) could be configured via the warning framework.

Bye,
    Walter D?rwald

From claireguilbaud at yahoo.fr  Thu Jul 22 13:15:23 2004
From: claireguilbaud at yahoo.fr (=?iso-8859-1?q?Claire=20Guilbaud?=)
Date: Thu Jul 22 13:15:25 2004
Subject: [Python-Dev] SWIG comments
Message-ID: <20040722111523.36366.qmail@web52801.mail.yahoo.com>

Hi, I use SWIG to create a module from several C++ classes. In my C++ classes, I have a lot of comments to generate doxygen documentation.
I would like to have the same comments for the module (with help(mymodule)) but in doc strings for pydoc. I don't want to have doc strings for pydoc in my C++ part.
 
Does anybody know a solution ?
 
Thanks very much
 
Margebulle

		
---------------------------------
Cr?ez gratuitement votre Yahoo! Mail avec 100 Mo de stockage !
Cr?ez votre Yahoo! Mail

Yahoo! Messenger : dialoguez instantan?ment avec vos amis - c'est  gratuit !
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040722/f909b0cc/attachment.htm
From aahz at pythoncraft.com  Thu Jul 22 16:03:20 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Jul 22 16:03:23 2004
Subject: [Python-Dev] SWIG comments
In-Reply-To: <20040722111523.36366.qmail@web52801.mail.yahoo.com>
References: <20040722111523.36366.qmail@web52801.mail.yahoo.com>
Message-ID: <20040722140320.GA28768@panix.com>

On Thu, Jul 22, 2004, Claire Guilbaud wrote:
>
> Hi, I use SWIG to create a module from several C++ classes. In
> my C++ classes, I have a lot of comments to generate doxygen
> documentation.  I would like to have the same comments for the module
> (with help(mymodule)) but in doc strings for pydoc. I don't want to
> have doc strings for pydoc in my C++ part.

python-dev is for discussion of the Python project, not for development
using Python.  Please post your question to comp.lang.python (or look for
a mailing list about Python extensions -- the Boost people would probably
be willing to help even if you're not using Boost).
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Barbara Boxer speaks for me:
http://buffaloreport.com/2004/040713.boxer.marriage.html
From dave at boost-consulting.com  Thu Jul 22 22:03:19 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Jul 22 22:03:27 2004
Subject: [Python-Dev] Re: SWIG comments
References: <20040722111523.36366.qmail@web52801.mail.yahoo.com>
	<20040722140320.GA28768@panix.com>
Message-ID: <ud62n4xo8.fsf@boost-consulting.com>

Aahz <aahz@pythoncraft.com> writes:

> On Thu, Jul 22, 2004, Claire Guilbaud wrote:
>>
>> Hi, I use SWIG to create a module from several C++ classes. In
>> my C++ classes, I have a lot of comments to generate doxygen
>> documentation.  I would like to have the same comments for the module
>> (with help(mymodule)) but in doc strings for pydoc. I don't want to
>> have doc strings for pydoc in my C++ part.
>
> python-dev is for discussion of the Python project, not for development
> using Python.  Please post your question to comp.lang.python (or look for
> a mailing list about Python extensions -- the Boost people would probably
> be willing to help even if you're not using Boost).

Actually most of us Boost people know almost nothing about SWIG.  I'm
pretty sure there's a SWIG-specific list where you could get answers.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From skip at pobox.com  Thu Jul 22 22:29:06 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Jul 22 22:29:23 2004
Subject: [Python-Dev] Re: SWIG comments
In-Reply-To: <ud62n4xo8.fsf@boost-consulting.com>
References: <20040722111523.36366.qmail@web52801.mail.yahoo.com>
	<20040722140320.GA28768@panix.com>
	<ud62n4xo8.fsf@boost-consulting.com>
Message-ID: <16640.9106.553133.106253@montanaro.dyndns.org>


    David> Actually most of us Boost people know almost nothing about SWIG.
    David> I'm pretty sure there's a SWIG-specific list where you could get
    David> answers.

Which most folks would guess is located at (drum roll please) ...

    http://www.swig.org/

I guess the OP isn't "most folks"... :-/

Skip

From bac at OCF.Berkeley.EDU  Fri Jul 23 05:25:20 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Jul 23 05:25:30 2004
Subject: [Python-Dev] PEP 309?
In-Reply-To: <40F2FEAD.5030209@blueyonder.co.uk>
References: <20040712164648.66DB41E4004@bag.python.org>
	<40F2FEAD.5030209@blueyonder.co.uk>
Message-ID: <41008520.8030103@ocf.berkeley.edu>

Peter Harris wrote:

> I know it's a busy time, but is there any chance that PEP 309 could get 
> squeezed in some time between 2.4a2 and the final release?  It's only a 
> tiny "waffer-theen" Python module, and as far as I know the code, docs 
> and tests are ready to go.
> Or if there's still some work to do, I'm ready to do it.
> 

So where does PEP 309 stand?  It says it is accepted and in the 
standards track for 2.4 .  Does it just need someone to check it in?

-Brett
From bac at OCF.Berkeley.EDU  Fri Jul 23 06:35:45 2004
From: bac at OCF.Berkeley.EDU (Brett Cannon)
Date: Fri Jul 23 06:35:52 2004
Subject: [Python-Dev] python-dev Summary for 2004-07-01 through 2004-07-15
	[draft]
Message-ID: <410095A1.2040002@ocf.berkeley.edu>

I was hoping to get this out sooner but moving threw a wrench into that 
plan.


I won't be getting to this until after the weekend so earliest deadline 
for edits will be Monday night PT.

Oh, and thanks to David Goodger all of you people with names requiring 
Unicode should now have their names spelled correctly.  =)

-------------------------

Encoding: utf-8

Summary Announcements
=====================
XXX

Summaries
=========
------------------
2.4a1 out the door
------------------
Thanks to Anthony Baxter, `Python 2.4a1`_ went out the door in early 
July.  If you have not downloaded it and run the regression test suite, 
please do (see the stdlib docs on the 'test' pacakge on how to run the 
tests and please check if you can that any failures you got were not 
already posted and possibly fixed in CVS).

.. _Python 2.4a1: http://www.python.org/2.4/

Contributing threads:
   - `2.4a1 
<http://mail.python.org/pipermail/python-dev/2004-July/045944.html>`__
   - `RELEASED Python 2.4, alpha 1 
<http://mail.python.org/pipermail/python-dev/2004-July/046013.html>`__

--------------------------------
MSI installer on Windows for 2.4
--------------------------------
Some discussion on Martin v. L?wis' MSI installer code took place on the 
list.  It seemed like Martin's installer was good and just needed some docs.

Contributing threads:
   - `VC 7.1 maintenance? 
<http://mail.python.org/pipermail/python-dev/2004-July/045815.html>`__
   - `Windows installer for 2.4? 
<http://mail.python.org/pipermail/python-dev/2004-July/045954.html>`__

---------------------------------------------------------------------------------
Decimal is in the stdlib with defaults for people who don't have a "bot" 
nickname
---------------------------------------------------------------------------------
Since the Decimal package went in at the beginning of July, most of the 
discussions about Decimal revolved around making it easy to use out of 
the box for normal folk; "practicality beats purity".

First discussion was over invalid constructor arguments.  Originally if 
you tried something like ``Decimal("Raymond")``, it would return NaN. 
Obviously that ain't cool.  Turned out to be a misinterpretation of the 
IBM spec that Decimal is based off of.  In the doc an exception being 
"signaled" is the same as it being raised in Python.  So that got 
clarified and now raises an exception.

 From there a discussion of what signals should be on by default ensued. 
  Michael Chermside sparked it with his list of what he thought would 
work.  Some other people posted theirs and all of them coagulated into a 
final list.

Contributing threads:
   - `decimal API 
<http://mail.python.org/pipermail/python-dev/2004-July/045859.html>`__
   - `decimal.py signals & traps 
<http://mail.python.org/pipermail/python-dev/2004-July/045963.html>`__

---------------------------------------------------------
The second Python Bug Day stomped out some of the buggers
---------------------------------------------------------
AM Kuchling organized another Python Bug Day on July 10.  A couple of 
people, including yours truly, showed up on #python-dev and managed to 
close a bunch of patches and bugs; 18 and 15 of them to be exact. 
Thanks to all that helped out, it was fun.

The next one is tentatively scheduled for August 7th.

Contributing threads:
   - `Python Bug day this Saturday 
<http://mail.python.org/pipermail/python-dev/2004-July/045901.html>`__
   - `Bug day outcome 
<http://mail.python.org/pipermail/python-dev/2004-July/046065.html>`__

----------------------------------
file()?  open()?  Paper?  Plastic?
----------------------------------
Guido had noticed someone had committed a patch that changed open() to 
file().  While some of us thought that using the file type's factory 
constructor was the proper way to open files, Guido set us straight. 
Turns out he wants open() to be used for opening files.  Part of the 
reason is that open() might at some point develop the abilities to open 
other things such as URLs (think ``open`` on OS X).

Contributing threads:
   - `file() or open()? 
<http://mail.python.org/pipermail/python-dev/2004-July/045921.html>`__

---------------------------------------------------------------------------------------------
What happens when you try to enter a new key in a dict that has another 
key considered equal?
---------------------------------------------------------------------------------------------
Michael Chermside wondered the same thing.  Tim Peters answered.  Turns 
out to be undefined, but currently implemented as to keep the original 
key and not the new key.

Contributing threads:
   - `Equal but different keys in dicts 
<http://mail.python.org/pipermail/python-dev/2004-July/046040.html>`__

-------------------------------------------------------------------------------------
New macro that decrefs and sets to NULL a value, all while spinning 
plates on sticks!
-------------------------------------------------------------------------------------
Jim Fulton added a new macro named Py_CLEAR() that takes in a variable, 
Py_DECREFs it, and then sets it to NULL.  The common use case is a field 
in a struct that can hold a PyObject but not always which is set to NULL 
when it doesn't.  It is in no way meant to signify the PyObject should 
be garbage collected immediately, just that the current variable no 
longer holds a reference to anything.

Contributing threads:
   - `Proposal: C API Macro to decref and set to NULL 
<http://mail.python.org/pipermail/python-dev/2004-July/046086.html>`__

--------------------------------------
Trying out tagged integers in the core
--------------------------------------
Result: can be faster, but people don't want the added complexity (and 
Guido had a bad experience while developing the ABC language).

For those of you who don't know what tagged integers are, imagine all 
PyObjects being either an object or integer based on whether certain 
bits were flipped.  This has the perk of saving space and not having to 
do a memory fetch to get to the int value in an int object.  Problem is 
you now have to check constantly whether a PyObject is really a pointer 
or a tagged integer.

Contributing threads:
   - `Tagged integers 
<http://mail.python.org/pipermail/python-dev/2004-July/046139.html>`__

----------------------------------
You are not getting tail recursion
----------------------------------
Guido has spoken and is not about to change his mind.  If you want to 
read the **long** thread on why then go ahead.  =)

Contributing threads:
   - `Proper tail recursion 
<http://mail.python.org/pipermail/python-dev/2004-July/046150.html>`__
From heikowu at ceosg.de  Fri Jul 23 08:26:21 2004
From: heikowu at ceosg.de (Heiko Wundram)
Date: Fri Jul 23 08:26:24 2004
Subject: [Python-Dev] PEP 262 - active?
Message-ID: <200407230826.21976.heikowu@ceosg.de>

Hey all!

Is anybody actively working on PEP 262? I'm currently in the process of 
implementing something like it for a package I'm writing, and I'd be glad to 
sponsor the code once it's complete.

I've tried to check out what's in the Python CVS sandbox, but can't get it at 
the moment because of network connectivity issues, and the tracker entries on 
the bugtracker aren't very helpful either (besides stating that the PEP was 
abandoned, and later reopened).

If there's interest, I'll write up a short text describing what my package can 
do in addition to what PEP 262 describes, and how it's integrated with 
distutils.

Heiko.
From tjreedy at udel.edu  Fri Jul 23 09:45:17 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri Jul 23 09:45:28 2004
Subject: [Python-Dev] Re: python-dev Summary for 2004-07-01 through
	2004-07-15[draft]
References: <410095A1.2040002@ocf.berkeley.edu>
Message-ID: <cdqfmk$t53$1@sea.gmane.org>


"Brett Cannon" <bac@OCF.Berkeley.EDU> wrote in message
news:410095A1.2040002@ocf.berkeley.edu...
----------------------------------
You are not getting tail recursion
----------------------------------

This strikes me as both misleading and potentially inflamatory.  People
remain free to use any form of recursion, including tail recursion.  As I
understand it, what Guido rejected was the automagic special case
single-framing of tail recursive calls (for time and space saving).  So
what people are not getting is automatic tail recursion optimization (or
elimination, though I consider that term less accurate).

It seems to me that this decision is consistent with the general policy of
not doing code-rewriting optimatization.  We also are 'not getting', for
instance, automatic common-expression elimination or automatic strength
reduction or automatic movement of constant expressions outside of loops.
So I suggest something like

---------------------------
Automatic tail recursion optimization
---------------------------
In keeping with the general policy of not doing code-rewrite optimation in
the compiler, Guido rejected this proposal.  For more, read...


Terry J. Reedy



From mcherm at mcherm.com  Fri Jul 23 14:20:52 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Fri Jul 23 14:19:43 2004
Subject: [Python-Dev] Re: python-dev Summary for 2004-07-01
	through2004-07-15[draft]
Message-ID: <1090585252.410102a457ec2@mcherm.com>

The summary stated:
> You are not getting tail recursion

Terry Reedy writes:
> This strikes me as both misleading and potentially inflamatory.

I agree with Terry that this one should be phrased (and titled?)
differently. Here's how I might word it:

----------------------------------
Tail recursion proposal rejected
----------------------------------
Christopher King proposed a patch that performs tail call elimination
-- if a recursive call were the last action in a function then it
would be performed without creating a new stack frame. During a
**long** discussion, Guido rejected it, not because of implementation
problems (although there were some), but because he felt that it
might encourage a programming style that he considered non-Pythonic.

Contributing threads:
   - `Proper tail recursion
<http://mail.python.org/pipermail/python-dev/2004-July/046150.html>`__


-- Michael Chermside

From aahz at pythoncraft.com  Fri Jul 23 15:22:04 2004
From: aahz at pythoncraft.com (Aahz)
Date: Fri Jul 23 15:22:07 2004
Subject: [Python-Dev] PEPping along
In-Reply-To: <200407230826.21976.heikowu@ceosg.de>
References: <200407230826.21976.heikowu@ceosg.de>
Message-ID: <20040723132204.GA3549@panix.com>

On Fri, Jul 23, 2004, Heiko Wundram wrote:
> 
> Is anybody actively working on PEP 262? I'm currently in the process
> of implementing something like it for a package I'm writing, and I'd
> be glad to sponsor the code once it's complete.

Just a suggestion: I'm not able to keep track of all the PEPs, and I
doubt many other people can, either.  Rather than forcing everyone else
to look up the PEP, it'd be a big help to provide a sentence or two of
description about the PEP -- please provide the title, at least!

Thanks in advance.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From fdrake at acm.org  Fri Jul 23 15:41:36 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri Jul 23 15:41:44 2004
Subject: [Python-Dev] PEP 262 - active?
In-Reply-To: <200407230826.21976.heikowu@ceosg.de>
References: <200407230826.21976.heikowu@ceosg.de>
Message-ID: <200407230941.36679.fdrake@acm.org>

On Friday 23 July 2004 02:26 am, Heiko Wundram wrote:
 > Is anybody actively working on PEP 262? I'm currently in the process of
 > implementing something like it for a package I'm writing, and I'd be glad
 > to sponsor the code once it's complete.

I was going to look into this as well, but haven't had time due to other 
priorities.

 > If there's interest, I'll write up a short text describing what my package
 > can do in addition to what PEP 262 describes, and how it's integrated with
 > distutils.

I'm interested!  I presume this should be discussed on the Distutils-SIG 
mailing list rather than here, though, since this is generally a distutils / 
package management issue rather than a broader topic.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>

From colanderman at gmail.com  Fri Jul 23 22:26:54 2004
From: colanderman at gmail.com (Chris King)
Date: Fri Jul 23 22:26:56 2004
Subject: [Python-Dev] Fun with whitespace
In-Reply-To: <DA01879C-DB20-11D8-B1B5-000A958D1666@cwi.nl>
References: <1090411017.40fe5a09111ad@mcherm.com>
	<DA01879C-DB20-11D8-B1B5-000A958D1666@cwi.nl>
Message-ID: <875c7e0704072313265f959001@mail.gmail.com>

On Wed, 21 Jul 2004 16:18:34 +0200, Jack Jansen <jack.jansen@cwi.nl> wrote:

> But it makes a difference where you're pasting it: in an editor window
> (where you don't want the >>>) or in an email message (where you
> probably do want the >>>).

I like the way Gnucap (http://www.gnu.org/software/gnucap/) handles
prompt cut and pastes.  It simply ignores any prompt noise at the
beginning of a line.  Would it be difficult to have Python do the same
thing (i.e. ignore '>>>' and '...' at the beginning of a line)?

Of course you wouldn't want production code to look like this, so it
should be something tabnanny checks for and complains about. 
Otherwise this would be very useful for cut & paste testing of code
gleaned from newsgroup postings and the like into any editor (or even
the command prompt itself).
From gward at python.net  Sat Jul 24 03:33:49 2004
From: gward at python.net (Greg Ward)
Date: Sat Jul 24 03:33:52 2004
Subject: [Python-Dev] "What's New": stable sorting?
Message-ID: <20040724013349.GA13781@cthulhu.gerg.ca>

Just reading through "What's New in Python 2.4" and I spotted this:

"""
The results of sorting are now guaranteed to be stable. This means that
two entries with equal keys will be returned in the same order as they
were input. For example, you can sort a list of people by name, and then
sort the list by age, resulting in a list sorted by age where people
with the same age are in name-sorted order.
"""

I thought the Tim-bot fixed Python's list.sort() to be stable *aaaages*
ago -- 1.6 or 2.0 rings a bell.  Not true?

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
Police arrested two kids yesterday -- one was drinking battery acid,
the other was eating fireworks.  They charged one and let the other one off.
From tim.peters at gmail.com  Sat Jul 24 05:17:51 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Jul 24 05:17:54 2004
Subject: [Python-Dev] "What's New": stable sorting?
In-Reply-To: <20040724013349.GA13781@cthulhu.gerg.ca>
References: <20040724013349.GA13781@cthulhu.gerg.ca>
Message-ID: <1f7befae04072320177a7112d2@mail.gmail.com>

[Greg Ward]
> Just reading through "What's New in Python 2.4" and I spotted this:
>
> """
> The results of sorting are now guaranteed to be stable. This means that
> two entries with equal keys will be returned in the same order as they
> were input. For example, you can sort a list of people by name, and then
> sort the list by age, resulting in a list sorted by age where people
> with the same age are in name-sorted order.
> """
>
> I thought the Tim-bot fixed Python's list.sort() to be stable *aaaages*
> ago -- 1.6 or 2.0 rings a bell.  Not true?

CPython's list.sort() first *became* stable in all cases in 2.3.  But
the 2.3 docs didn't guarantee stability, CPython just happened to
provide stability.  What's new here in 2.4 is that the docs now
guarantee stability.  This constrains future implementations,
something we weren't comfortable doing until lots of experience with
the new sort strongly suggested that only a fool would even consider
the possibility that a different implementation of sorting may ever
exist <wink>.
From bac at OCF.Berkeley.EDU  Sat Jul 24 06:04:28 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sat Jul 24 06:04:33 2004
Subject: [Python-Dev] Re: python-dev Summary for 2004-07-01
	through	2004-07-15[draft]
In-Reply-To: <cdqfmk$t53$1@sea.gmane.org>
References: <410095A1.2040002@ocf.berkeley.edu> <cdqfmk$t53$1@sea.gmane.org>
Message-ID: <4101DFCC.3090003@ocf.berkeley.edu>

Terry Reedy wrote:

> "Brett Cannon" <bac@OCF.Berkeley.EDU> wrote in message
> news:410095A1.2040002@ocf.berkeley.edu...
> ----------------------------------
> You are not getting tail recursion
> ----------------------------------
> 
> This strikes me as both misleading and potentially inflamatory.

Possibly, but I am not PC in any way.  =)

I think I have just spent too much of my life coding in Scheme.  I 
should have clarified it with "proper".  I will change it somehow for 
the final version.

-Brett
From exarkun at divmod.com  Sat Jul 24 07:44:09 2004
From: exarkun at divmod.com (Jp Calderone)
Date: Sat Jul 24 07:44:16 2004
Subject: [Python-Dev] timemodule.c, 2.140
Message-ID: <4101F729.2070008@divmod.com>


   time.strftime() in 2.4a0 places more restrictions on its inputs than 
does the version in 2.3.x.  Revision 2.140 seems to be where this was 
introduced.

   I believe it is a common use of strftime() to fill out only some 
fields of the time tuple.  Requiring the day of year and day of week is 
particularly burdensome.

   I have real code that will not work at all on Python 2.4 because of 
this.  Is there any possiblity of the restriction being removed?

   Jp

From exarkun at divmod.com  Sat Jul 24 07:58:14 2004
From: exarkun at divmod.com (Jp Calderone)
Date: Sat Jul 24 07:58:19 2004
Subject: [Python-Dev] timemodule.c, 2.140
In-Reply-To: <4101F729.2070008@divmod.com>
References: <4101F729.2070008@divmod.com>
Message-ID: <4101FA76.7040306@divmod.com>

Jp Calderone wrote:
> 
>   time.strftime() in 2.4a0 places more restrictions on its inputs than 
> does the version in 2.3.x.  Revision 2.140 seems to be where this was 
> introduced.
> 

   I meant to include this example:

     import time
     now = time.gmtime()
     now = now[:-3] + (0, 0, 0)
     print time.strftime("", now)

   Jp
From tim.peters at gmail.com  Sat Jul 24 08:20:18 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Jul 24 08:20:23 2004
Subject: [Python-Dev] timemodule.c, 2.140
In-Reply-To: <4101F729.2070008@divmod.com>
References: <4101F729.2070008@divmod.com>
Message-ID: <1f7befae04072323204bc7f8da@mail.gmail.com>

[Jp Calderone]
>   time.strftime() in 2.4a0 places more restrictions on its inputs than
> does the version in 2.3.x.  Revision 2.140 seems to be where this was
> introduced.
>
>   I believe it is a common use of strftime() to fill out only some
> fields of the time tuple.  Requiring the day of year and day of week is
> particularly burdensome.
>
>   I have real code that will not work at all on Python 2.4 because of
> this.  Is there any possiblity of the restriction being removed?

Yes, but the likelihood would zoom if you opened a bug report about
it.  This is what alpha releases are for.  IIRC, the change was to
prevent core dumps on some platforms in some cases.  The sooner you
file that bug report <wink>, the better the chance somone can
volunteer enough time to make you happy without reintroducing crashes.
From tjreedy at udel.edu  Sat Jul 24 08:33:09 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat Jul 24 08:33:20 2004
Subject: [Python-Dev] Re: Re: python-dev Summary for
	2004-07-01through	2004-07-15[draft]
References: <410095A1.2040002@ocf.berkeley.edu> <cdqfmk$t53$1@sea.gmane.org>
	<4101DFCC.3090003@ocf.berkeley.edu>
Message-ID: <cdsvrd$1cs$1@sea.gmane.org>


"Brett C." <bac@OCF.Berkeley.EDU> wrote in message
news:4101DFCC.3090003@ocf.berkeley.edu...
> > This strikes me as both misleading and potentially inflamatory.
>
> Possibly, but I am not PC in any way.  =)

Neither am I, but I was thinking of the possible effect on readers of
comp.lang.python, most of whom have not programmed in scheme, etc, and a
few of whom sometimes look for things to flame python about.  I don't want
to deal with an unnecessary false rumor, and I suspect that you don't
either.

Terry J. Reedy



From bac at OCF.Berkeley.EDU  Sat Jul 24 20:48:35 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sat Jul 24 20:49:42 2004
Subject: [Python-Dev] timemodule.c, 2.140
In-Reply-To: <1f7befae04072323204bc7f8da@mail.gmail.com>
References: <4101F729.2070008@divmod.com>
	<1f7befae04072323204bc7f8da@mail.gmail.com>
Message-ID: <4102AF03.5050808@ocf.berkeley.edu>

Tim Peters wrote:

> [Jp Calderone]
> 
>>  time.strftime() in 2.4a0 places more restrictions on its inputs than
>>does the version in 2.3.x.  Revision 2.140 seems to be where this was
>>introduced.
>>
>>  I believe it is a common use of strftime() to fill out only some
>>fields of the time tuple.  Requiring the day of year and day of week is
>>particularly burdensome.
>>
>>  I have real code that will not work at all on Python 2.4 because of
>>this.  Is there any possiblity of the restriction being removed?
> 
> 
> Yes, but the likelihood would zoom if you opened a bug report about
> it.  This is what alpha releases are for.  IIRC, the change was to
> prevent core dumps on some platforms in some cases.  The sooner you
> file that bug report <wink>, the better the chance somone can
> volunteer enough time to make you happy without reintroducing crashes.

Yeah, that fix was introduced to prevent core dumps when bad info was 
used (usually from using a value that was outside the bounds of 
something, such as a negative, and that leading to indexing outside the 
array's actual size).  See 
http://mail.python.org/pipermail/python-dev/2004-February/042675.html 
for the thread that came up with this solution (which Guido okayed).

-Brett
From martin at v.loewis.de  Sun Jul 25 00:21:12 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Jul 25 00:21:26 2004
Subject: [Python-Dev] MSI Documentation
Message-ID: <4102E0D8.5020105@v.loewis.de>

I've added a bit of documentation on using the 2.4 MSI package, at

http://www.python.org/2.4/msi.html

If you think something is incorrect, or should be added, please let me
know (or change it yourself if you can).

Regards,
Martin
From jim at zope.com  Tue Jul 27 14:52:22 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 27 14:52:29 2004
Subject: [Python-Dev] Fix import errors to have data
Message-ID: <41065006.6020607@zope.com>


A common idiom to optionally support some module if it is
present is to use ImportError handlers:

   try:
       import foo
   except ImportError:
       # Configure for absense of foo
       ...
   else:
       # Configure for presense of foo
       ...

Unfortunately, this is a bug trap.  The module foo
might be present and yet it's import could fail with an import error.
This can happen if one of *its* imports fails.  Code like that
above will hide such bugs.

Unfortunately, it's hard to do this correctly, because ImportErrors
don't have the module name as data.  When the import error is raised,
it is raised with an error message rather than data.  This is because
most standard exception classes share a common __str__ that
simply prints their initialization arguments.

At present, to conditionally support a module, you have to
use code like:

   try:
       import foo
   except ImportError, v:
       if not str(v).endsswith(' foo'):
           raise
       # Configure for absense of foo
       ...
   else:
       # Configure for presense of foo
       ...

which is ugly and brittle.

I'd like to get this fixed.

I propose to:

   - Provide ImportError with an __init__ that takes a module name
     and sets a module_name attribute

   - Provide ImportError with an __str__ that produces the message
     we have now

   - Change standard code that raises import errors to provide just the
     module name.

With this change, one could write careful conditional
import code like this:

   try:
       import foo
   except ImportError, v:
       if v.module_name != 'foo':
           raise
       # Configure for absense of foo
       ...
   else:
       # Configure for presense of foo
       ...

which is much cleaner IMO.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From guido at python.org  Tue Jul 27 16:45:14 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul 27 17:21:46 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Tue, 27 Jul 2004 08:52:22 EDT."
	<41065006.6020607@zope.com> 
References: <41065006.6020607@zope.com> 
Message-ID: <200407271445.i6REjEK24454@guido.python.org>

> A common idiom to optionally support some module if it is
> present is to use ImportError handlers:
> 
>    try:
>        import foo
>    except ImportError:
>        # Configure for absense of foo
>        ...
>    else:
>        # Configure for presense of foo
>        ...
> 
> Unfortunately, this is a bug trap.  The module foo
> might be present and yet it's import could fail with an import error.
> This can happen if one of *its* imports fails.  Code like that
> above will hide such bugs.

Why is it important to catch the bug at this point?  If the module
imported by foo has a bug, importing it directly will reveal it.  The
code importing foo will work without foo.  In some very real sense,
foo is unusable, and its importer is making the right choice.

> Unfortunately, it's hard to do this correctly, because ImportErrors
> don't have the module name as data.  When the import error is raised,
> it is raised with an error message rather than data.  This is because
> most standard exception classes share a common __str__ that
> simply prints their initialization arguments.
> 
> At present, to conditionally support a module, you have to
> use code like:
> 
>    try:
>        import foo
>    except ImportError, v:
>        if not str(v).endsswith(' foo'):
>            raise
>        # Configure for absense of foo
>        ...
>    else:
>        # Configure for presense of foo
>        ...
> 
> which is ugly and brittle.
> 
> I'd like to get this fixed.
> 
> I propose to:
> 
>    - Provide ImportError with an __init__ that takes a module name
>      and sets a module_name attribute
> 
>    - Provide ImportError with an __str__ that produces the message
>      we have now
> 
>    - Change standard code that raises import errors to provide just the
>      module name.
> 
> With this change, one could write careful conditional
> import code like this:
> 
>    try:
>        import foo
>    except ImportError, v:
>        if v.module_name != 'foo':
>            raise
>        # Configure for absense of foo
>        ...
>    else:
>        # Configure for presense of foo
>        ...
> 
> which is much cleaner IMO.

If you provide a working patch, I have no objection against its
introduction.  I'm just not going out of my way for it.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From jim at zope.com  Tue Jul 27 17:36:00 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 27 17:36:04 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200407271445.i6REjEK24454@guido.python.org>
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
Message-ID: <41067660.2070109@zope.com>

Guido van Rossum wrote:
>>A common idiom to optionally support some module if it is
>>present is to use ImportError handlers:
>>
>>   try:
>>       import foo
>>   except ImportError:
>>       # Configure for absense of foo
>>       ...
>>   else:
>>       # Configure for presense of foo
>>       ...
>>
>>Unfortunately, this is a bug trap.  The module foo
>>might be present and yet it's import could fail with an import error.
>>This can happen if one of *its* imports fails.  Code like that
>>above will hide such bugs.
> 
> 
> Why is it important to catch the bug at this point? 

Because the masked import failure could leave modules in a
a broken state that causes very hard to diagnose problems
later.

 > If the module
> imported by foo has a bug, importing it directly will reveal it. 

No, it won't.  For example, suppose foo imports B. B tries to import
C, but fails.  B is now broken, but it is still importable.  Actually,
both foo and B can be imported without errors, even though they are
broken.

 > The
> code importing foo will work without foo.  In some very real sense,
> foo is unusable, and its importer is making the right choice.

The intent is to ignore the absense of foo, not the brokenness
of foo.  If foo is around, but broken, I want to know about it.

As a bonus, if I ignore the error in importing foo, I may leave a
broken, but importable foo around.  If any other code tries
to conditionally import foo, it will think that foo is imported
and usable.

...

> If you provide a working patch, I have no objection against its
> introduction.  I'm just not going out of my way for it.

Cool.  I expected to do this myself anyway.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From tim.peters at gmail.com  Tue Jul 27 17:57:01 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Jul 27 17:57:03 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <41067660.2070109@zope.com>
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
Message-ID: <1f7befae040727085715ce2536@mail.gmail.com>

[Jim Fulton]
...
> No, it won't.  For example, suppose foo imports B. B tries to import
> C, but fails.  B is now broken, but it is still importable.  Actually,
> both foo and B can be imported without errors, even though they are
> broken.

Then you're proposing a way for a highly knowledgable user to
anticipate, and partially worm around, that Python leaves behind
insane module objects in sys.modules.  Wouldn't it be better to change
Python to stop leaving insane module objects in sys.modules to begin
with?  That's harder, but seems to me it would do a lot more good for
a lot more people.

Indeed, Python's own test suite fails on Windows often when run with
-r (randomize test order) because of broken modules left behind by
trying to run Unix tests.  Typically one test imports pty, which fails
because it can't import termios, but then a later test importing pty
succeeds (getting an insane pty) -- and despite this insanity in
pty.py already:

# Absurd:  import termios and then delete it.  This is to force an attempt
# to import pty to raise an ImportError on platforms that lack termios.
# Without this explicit import of termios here, some other module may
# import tty first, which in turn imports termios and dies with an
# ImportError then.  But since tty *does* exist across platforms, that
# leaves a damaged module object for tty in sys.modules, and the import
# of tty here then appears to work despite that the tty imported is junk.
import termios
del termios
import tty

That's a good example of the crazy brittleness we endure now.  I
really don't want to see more brittle code worming around a problem
that "shouldn't" be there at all.

That said, I have no objection to giving ImportError exceptions a
module_name attribute.  My objection is to the idea that it solves the
real problem.
From jim at zope.com  Tue Jul 27 18:39:22 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 27 18:39:57 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1f7befae040727085715ce2536@mail.gmail.com>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
Message-ID: <4106853A.8080109@zope.com>

Tim Peters wrote:
> [Jim Fulton]
> ...
> 
>>No, it won't.  For example, suppose foo imports B. B tries to import
>>C, but fails.  B is now broken, but it is still importable.  Actually,
>>both foo and B can be imported without errors, even though they are
>>broken.
> 
> 
> Then you're proposing a way for a highly knowledgable user to
> anticipate, and partially worm around,  that Python leaves behind
> insane module objects in sys.modules. 

No.  I'm proposing a way for a Python developer to detect the
presence or absence of a module.

Hm, perhaps it would be better to provide an API (if there isn't one
already) to test whether a module is present.

AFAIK, the only way to do it is by trying the import.  Unfortunately,
the error type alone isn't enough to tell whether the error you
caught is due to the error condition you are trying to test.

Perhaps providing an API is a better way to go.

In general, however, I hate the approach of formatting errors
as they are raised, rather than as they are converted to strings.
Formatting them earlier is a bit more convenient for the exception-class
author, but:

- It makes the exception data inaccessible without ugly brittle
   error parsing hacks, and

- It makes error raising more costly, which sometimes matters
   (e.g. for attribute errors).

- It makes I18n harder, as it's hard (impossible?) to translate
   error messages with embedded data.

 > Wouldn't it be better to change
> Python to stop leaving insane module objects in sys.modules to begin
> with? 

It would make the current situation less icky, but it wouldn't solve
the problem I posed.  You still wouldn't be able to tell the difference
between an absent module and a broken one without a error-parsing hack.
I think that not ignoring errors is a good thing even if the errors
ignored didn't have side effects.

...

> That said, I have no objection to giving ImportError exceptions a
> module_name attribute. 

Cool.  I could also live with an API that tests whether a
module exists.

 > My objection is to the idea that it solves the
> real problem.

It isn't intended to solve broken-module problem.
It solves (or, at least addresses) a different problem, but it is
certainly exacerbated by the broken-module problem.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From neal at metaslash.com  Tue Jul 27 18:49:14 2004
From: neal at metaslash.com (Neal Norwitz)
Date: Tue Jul 27 18:49:28 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <4106853A.8080109@zope.com>
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
Message-ID: <20040727164913.GS7708@epoch.metaslash.com>

On Tue, Jul 27, 2004 at 12:39:22PM -0400, Jim Fulton wrote:
> Tim Peters wrote:
> >[Jim Fulton]
> >
> >>No, it won't.  For example, suppose foo imports B. B tries to import
> >>C, but fails.  B is now broken, but it is still importable.  Actually,
> >>both foo and B can be imported without errors, even though they are
> >>broken.
>
> >Then you're proposing a way for a highly knowledgable user to
> >anticipate, and partially worm around,  that Python leaves behind
> >insane module objects in sys.modules. 
> 
> No.  I'm proposing a way for a Python developer to detect the
> presence or absence of a module.
> 
> Hm, perhaps it would be better to provide an API (if there isn't one
> already) to test whether a module is present.

Another possibility could be to add a ChainedImportError (or
SubImportError?) which would derive from ImportError.  The module
which imports a broken module would get the normal ImportError.  If
the module doesn't catch the exception, it would get converted to this
new type of ImportError.

I don't know how easy this would be to implement, but I think it
would solve Jim's problem and perhaps be easier to deal with?

Neal

From tim.peters at gmail.com  Tue Jul 27 18:59:35 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Jul 27 18:59:57 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <4106853A.8080109@zope.com>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
Message-ID: <1f7befae04072709592dc58bd9@mail.gmail.com>

[Jim Fulton]
> ...
> No.  I'm proposing a way for a Python developer to detect the
> presence or absence of a module.

Seems to me a module is present if and only if you try to import it,
and the import succeeds, or the import fails and the module name is in
sys.modules after.

>>> import cupcake  # no such thing
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ImportError: No module named cupcake
>>> 'cupcake' in sys.modules
False
>>> 

>>> import pty  # exists, but is broken on Windows
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\Program Files\Zope-2.7.2-0\bin\Lib\pty.py", line 19, in ?
    import termios
ImportError: No module named termios
>>> 'pty' in sys.modules
True
>>>

Not that I like this -- it turns broken modules into "a feature".  A
function that answered the question directly would be mounds better.
From jim at zope.com  Tue Jul 27 19:07:01 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 27 19:07:05 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1f7befae04072709592dc58bd9@mail.gmail.com>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<1f7befae04072709592dc58bd9@mail.gmail.com>
Message-ID: <41068BB5.8060209@zope.com>

Tim Peters wrote:
> [Jim Fulton]
> 
>>...
>>No.  I'm proposing a way for a Python developer to detect the
>>presence or absence of a module.
> 
> 
> Seems to me a module is present if and only if you try to import it,
> and the import succeeds, or the import fails and the module name is in
> sys.modules after.

That's a definition I don't really care for at all.
It's like saying that an object has an attribute if
trying to get the attribute doesn't raise an error. :)

Seems to me a module isn't present if it's implementation
isn't in an import path.  To me a module is broken if it
has an implementation that fails.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From guido at python.org  Tue Jul 27 19:07:10 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul 27 19:07:16 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Tue, 27 Jul 2004 12:49:14 EDT."
	<20040727164913.GS7708@epoch.metaslash.com> 
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com> 
	<20040727164913.GS7708@epoch.metaslash.com> 
Message-ID: <200407271707.i6RH7A524790@guido.python.org>

> Another possibility could be to add a ChainedImportError (or
> SubImportError?) which would derive from ImportError.  The module
> which imports a broken module would get the normal ImportError.  If
> the module doesn't catch the exception, it would get converted to this
> new type of ImportError.
> 
> I don't know how easy this would be to implement, but I think it
> would solve Jim's problem and perhaps be easier to deal with?

Having had the recent opportunity to root around in Java code that
chains exceptions like this a lot, I think I'm against that approach.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From jim at zope.com  Tue Jul 27 19:09:03 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 27 19:09:06 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <20040727164913.GS7708@epoch.metaslash.com>
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
Message-ID: <41068C2F.40102@zope.com>

Neal Norwitz wrote:
> On Tue, Jul 27, 2004 at 12:39:22PM -0400, Jim Fulton wrote:
> 
>>Tim Peters wrote:
>>
>>>[Jim Fulton]
>>>
>>>
>>>>No, it won't.  For example, suppose foo imports B. B tries to import
>>>>C, but fails.  B is now broken, but it is still importable.  Actually,
>>>>both foo and B can be imported without errors, even though they are
>>>>broken.
>>
>>>Then you're proposing a way for a highly knowledgable user to
>>>anticipate, and partially worm around,  that Python leaves behind
>>>insane module objects in sys.modules. 
>>
>>No.  I'm proposing a way for a Python developer to detect the
>>presence or absence of a module.
>>
>>Hm, perhaps it would be better to provide an API (if there isn't one
>>already) to test whether a module is present.
> 
> 
> Another possibility could be to add a ChainedImportError (or
> SubImportError?) which would derive from ImportError.  The module
> which imports a broken module would get the normal ImportError.  If
> the module doesn't catch the exception, it would get converted to this
> new type of ImportError.
> 
> I don't know how easy this would be to implement, but I think it
> would solve Jim's problem and perhaps be easier to deal with?

An even simpler aproach would be to have a ModuleNotFound error
that extends ImportError and gets raised if the module can't
be found.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From python at discworld.dyndns.org  Tue Jul 27 19:20:31 2004
From: python at discworld.dyndns.org (Charles Cazabon)
Date: Tue Jul 27 19:12:58 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1f7befae04072709592dc58bd9@mail.gmail.com>
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<1f7befae04072709592dc58bd9@mail.gmail.com>
Message-ID: <20040727172030.GA6919@discworld.dyndns.org>

Tim Peters <tim.peters@gmail.com> wrote:
> 
> Seems to me a module is present if and only if you try to import it,
> and the import succeeds, or the import fails and the module name is in
> sys.modules after.

This is a bit I've never been quite sure of.  If a module knows it imports
something not available everywhere, and can handle that bit missing, shouldn't
it already wrap its import of that module in a try: ... except ImportError:
block?

> >>> import pty  # exists, but is broken on Windows
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "C:\Program Files\Zope-2.7.2-0\bin\Lib\pty.py", line 19, in ?
>     import termios
> ImportError: No module named termios
> >>> 'pty' in sys.modules
> True

To me, it would seem more sensible (and would address the Windows brittleness
Tim mentions) if the above did /not/ leave pty in sys.modules -- i.e. the
import failed either completely, or not at all.  The above is a "partial
failure" which seems (to me) to be the root cause of the confusion.  Instead,
it should fail completely, and then the pty module can (if it wants) try to
import termios and handle it gracefully at that level if necessary.

Charles
-- 
-----------------------------------------------------------------------
Charles Cazabon                           <python@discworld.dyndns.org>
GPL'ed software available at:     http://www.qcc.ca/~charlesc/software/
-----------------------------------------------------------------------
From casey at zope.com  Tue Jul 27 19:17:54 2004
From: casey at zope.com (Casey Duncan)
Date: Tue Jul 27 19:17:14 2004
Subject: [Python-Dev] Re: Fix import errors to have data
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
Message-ID: <20040727131754.5998927d.casey@zope.com>

On Tue, 27 Jul 2004 12:39:22 -0400
Jim Fulton <jim@zope.com> wrote:

> Tim Peters wrote:
> > [Jim Fulton]
> > ...
> > 
> >>No, it won't.  For example, suppose foo imports B. B tries to import
> >>C, but fails.  B is now broken, but it is still importable. 
> >Actually,>both foo and B can be imported without errors, even though
> >they are>broken.
> > 
> > 
> > Then you're proposing a way for a highly knowledgable user to
> > anticipate, and partially worm around,  that Python leaves behind
> > insane module objects in sys.modules. 
> 
> No.  I'm proposing a way for a Python developer to detect the
> presence or absence of a module.
> 
> Hm, perhaps it would be better to provide an API (if there isn't one
> already) to test whether a module is present.

If such an api were to exist, what question exactly would it answer?

1. That there is a module by a particular name that could possibly be
imported, but the import may not succeed when actually tried.

2. There is a module by a particular name which can be imported and the
import will not
fail.

I would vote for the former, but it might be surprising to run into
behavior like this:

>>> sys.hasmodule('foo')
True
>>> import foo
ImportError: foo blew

What if there was a new exception ModuleNotFoundError which subclassed
ImportError. This would be raised by the import machinery when the
module could not be found. Errors during import would continue to raise
a standard ImportError.

I think this, coupled with Jim's original suggestion and Tim's to
prevent broken modules being retained in os.modules would help a lot.

-Casey

From jim at zope.com  Tue Jul 27 19:21:15 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 27 19:21:19 2004
Subject: [Python-Dev] Re: Fix import errors to have data
In-Reply-To: <20040727131754.5998927d.casey@zope.com>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>	<1f7befae040727085715ce2536@mail.gmail.com>	<4106853A.8080109@zope.com>
	<20040727131754.5998927d.casey@zope.com>
Message-ID: <41068F0B.3050608@zope.com>

Casey Duncan wrote:
> On Tue, 27 Jul 2004 12:39:22 -0400
> Jim Fulton <jim@zope.com> wrote:
> 
> 
>>Tim Peters wrote:
>>
>>>[Jim Fulton]
>>>...
>>>
>>>
>>>>No, it won't.  For example, suppose foo imports B. B tries to import
>>>>C, but fails.  B is now broken, but it is still importable. 
>>>
>>>Actually,>both foo and B can be imported without errors, even though
>>>they are>broken.
>>>
>>>
>>>Then you're proposing a way for a highly knowledgable user to
>>>anticipate, and partially worm around,  that Python leaves behind
>>>insane module objects in sys.modules. 
>>
>>No.  I'm proposing a way for a Python developer to detect the
>>presence or absence of a module.
>>
>>Hm, perhaps it would be better to provide an API (if there isn't one
>>already) to test whether a module is present.
> 
> 
> If such an api were to exist, what question exactly would it answer?
> 
> 1. That there is a module by a particular name that could possibly be
> imported, but the import may not succeed when actually tried.

Yup

> 2. There is a module by a particular name which can be imported and the
> import will not
> fail.
> 
> I would vote for the former, but it might be surprising to run into
> behavior like this:
> 
> 
>>>>sys.hasmodule('foo')
> 
> True
> 
>>>>import foo
> 
> ImportError: foo blew

Of course it would be surprising, because foo would be broken.
That's why they call them exceptions. :)

> What if there was a new exception ModuleNotFoundError which subclassed
> ImportError. This would be raised by the import machinery when the
> module could not be found. Errors during import would continue to raise
> a standard ImportError.

Yup.

> I think this, coupled with Jim's original suggestion and Tim's to
> prevent broken modules being retained in os.modules would help a lot.

Yup. It would certainly address the imediate problem.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From tim.peters at gmail.com  Tue Jul 27 19:25:02 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Jul 27 19:25:05 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <20040727172030.GA6919@discworld.dyndns.org>
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<1f7befae04072709592dc58bd9@mail.gmail.com>
	<20040727172030.GA6919@discworld.dyndns.org>
Message-ID: <1f7befae04072710256595510@mail.gmail.com>

[Charles Cazabon]
> This is a bit I've never been quite sure of.  If a module knows it imports
> something not available everywhere, and can handle that bit missing, shouldn't
> it already wrap its import of that module in a try: ... except ImportError:
> block?

Sure!  Jim wants to distinguish "absenceness" from "brokenness" for
some reason.  It's not clear to my why (in response to Guido, he
mentioned only the "broken module in sys.modules" problem, but when I
suggested we address that directly he just repeated "you still
wouldn't be able to tell the difference
between an absent module and a broken one").

> To me, it would seem more sensible (and would address the Windows brittleness
> Tim mentions) if the above did /not/ leave pty in sys.modules -- i.e. the
> import failed either completely, or not at all.  The above is a "partial
> failure" which seems (to me) to be the root cause of the confusion.  Instead,
> it should fail completely, and then the pty module can (if it wants) try to
> import termios and handle it gracefully at that level if necessary.

Leaving insane module objects behind is harmful, and I'd like to
repair that.  But you still wouldn't be able to tell the difference
between an absent module and a broken one <wink>.
From jim at zope.com  Tue Jul 27 19:29:21 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 27 19:29:24 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1f7befae04072710256595510@mail.gmail.com>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>	<1f7befae040727085715ce2536@mail.gmail.com>	<4106853A.8080109@zope.com>	<1f7befae04072709592dc58bd9@mail.gmail.com>	<20040727172030.GA6919@discworld.dyndns.org>
	<1f7befae04072710256595510@mail.gmail.com>
Message-ID: <410690F1.2090308@zope.com>

Tim Peters wrote:
> [Charles Cazabon]
> 
>>This is a bit I've never been quite sure of.  If a module knows it imports
>>something not available everywhere, and can handle that bit missing, shouldn't
>>it already wrap its import of that module in a try: ... except ImportError:
>>block?
> 
> 
> Sure!  Jim wants to distinguish "absenceness" from "brokenness" for
> some reason.  It's not clear to my why

Because a broken module is something that should get fixed.

Hiding errors is a bad thing.  It's like using hasattr. :)

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From barry at python.org  Tue Jul 27 19:37:59 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Jul 27 19:37:42 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1f7befae04072710256595510@mail.gmail.com>
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com> <4106853A.8080109@zope.com>
	<1f7befae04072709592dc58bd9@mail.gmail.com>
	<20040727172030.GA6919@discworld.dyndns.org>
	<1f7befae04072710256595510@mail.gmail.com>
Message-ID: <1090949879.8672.92.camel@localhost>

On Tue, 2004-07-27 at 13:25, Tim Peters wrote:

> Sure!  Jim wants to distinguish "absenceness" from "brokenness" for
> some reason.

So, why doesn't imp.find_module() fit the bill?

> Leaving insane module objects behind is harmful, and I'd like to
> repair that.

+1 (wishing I could vote with a higher score).  Leaving broken modules
in sys.modules is a recurring headache.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040727/bdc2fa35/attachment.pgp
From jim at zope.com  Tue Jul 27 19:45:07 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 27 19:45:22 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1xixz737.fsf@python.net>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>	<1f7befae040727085715ce2536@mail.gmail.com>	<4106853A.8080109@zope.com>
	<1xixz737.fsf@python.net>
Message-ID: <410694A3.7010008@zope.com>

Thomas Heller wrote:
> Jim Fulton <jim@zope.com> writes:
> 

...

>>Hm, perhaps it would be better to provide an API (if there isn't one
>>already) to test whether a module is present.
> 
> 
> Doesn't imp.find_module do this? On Windows:

Nope. It doesn't handle packages.

 >>> import compiler.ast
 >>> import imp
 >>> imp.find_module('compiler.ast')
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
ImportError: No module named compiler.ast


Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From mcherm at mcherm.com  Tue Jul 27 19:47:49 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue Jul 27 19:45:50 2004
Subject: [Python-Dev] Fix import errors to have data
Message-ID: <1090950469.410695453a9e4@mcherm.com>

Guido writes:
> Having had the recent opportunity to root around in Java code that
> chains exceptions [...] a lot, I think I'm against that approach.

I understand where you're coming from, but don't forget that the
annoyances about chained exceptions are MUCH worse in Java than
they would be in Python because Java's checked exceptions (plus a bit
of poor code design) practically require every single layer of code
to wrap the exceptions from the layer below. In Python you could just
ignore them unless you had something new to add. (Of course, that
could be done perfectly well *without* chained exceptions also...)

-- Michael Chermside

From jim at zope.com  Tue Jul 27 19:45:55 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 27 19:45:59 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1090949879.8672.92.camel@localhost>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>	<1f7befae04072709592dc58bd9@mail.gmail.com>	<20040727172030.GA6919@discworld.dyndns.org>	<1f7befae04072710256595510@mail.gmail.com>
	<1090949879.8672.92.camel@localhost>
Message-ID: <410694D3.20804@zope.com>

Barry Warsaw wrote:
> On Tue, 2004-07-27 at 13:25, Tim Peters wrote:
> 
> 
>>Sure!  Jim wants to distinguish "absenceness" from "brokenness" for
>>some reason.
> 
> 
> So, why doesn't imp.find_module() fit the bill?

It doesn't handle packages.

 >>> import compiler.ast
 >>> import imp
 >>> imp.find_module('compiler.ast')
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
ImportError: No module named compiler.ast

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From pje at telecommunity.com  Tue Jul 27 19:54:34 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Jul 27 19:51:25 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <410694D3.20804@zope.com>
References: <1090949879.8672.92.camel@localhost> <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<1f7befae04072709592dc58bd9@mail.gmail.com>
	<20040727172030.GA6919@discworld.dyndns.org>
	<1f7befae04072710256595510@mail.gmail.com>
	<1090949879.8672.92.camel@localhost>
Message-ID: <5.1.1.6.0.20040727135305.03d64ab0@mail.telecommunity.com>

At 01:45 PM 7/27/04 -0400, Jim Fulton wrote:
>Barry Warsaw wrote:
>>On Tue, 2004-07-27 at 13:25, Tim Peters wrote:
>>
>>>Sure!  Jim wants to distinguish "absenceness" from "brokenness" for
>>>some reason.
>>
>>So, why doesn't imp.find_module() fit the bill?
>
>It doesn't handle packages.
>
> >>> import compiler.ast
> >>> import imp
> >>> imp.find_module('compiler.ast')
>Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>ImportError: No module named compiler.ast

Sure it does; you just have to ask it nicely:  :)

Python 2.2.2 (#37, Oct 14 2002, 17:02:34) [MSC 32 bit (Intel)] on win32
Type "copyright", "credits" or "license" for more information.
IDLE 0.8 -- press F1 for help
 >>> import imp
 >>> imp.find_module('compiler')
(None, 'C:\\PYTHON22\\lib\\compiler', ('', '', 5))
 >>> import compiler
 >>> imp.find_module('ast',compiler.__path__)
(<open file 'C:\PYTHON22\lib\compiler\ast.py', mode 'r' at 0x00A99660>, 
'C:\\PYTHON22\\lib\\compiler\\ast.py', ('.py', 'r', 1))
 >>>


From jim at zope.com  Tue Jul 27 19:57:50 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 27 19:57:54 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <5.1.1.6.0.20040727135305.03d64ab0@mail.telecommunity.com>
References: <1090949879.8672.92.camel@localhost> <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<1f7befae04072709592dc58bd9@mail.gmail.com>
	<20040727172030.GA6919@discworld.dyndns.org>
	<1f7befae04072710256595510@mail.gmail.com>
	<1090949879.8672.92.camel@localhost>
	<5.1.1.6.0.20040727135305.03d64ab0@mail.telecommunity.com>
Message-ID: <4106979E.5080304@zope.com>

Phillip J. Eby wrote:
> At 01:45 PM 7/27/04 -0400, Jim Fulton wrote:
> 
>> Barry Warsaw wrote:
>>
>>> On Tue, 2004-07-27 at 13:25, Tim Peters wrote:
>>>
>>>> Sure!  Jim wants to distinguish "absenceness" from "brokenness" for
>>>> some reason.
>>>
>>>
>>> So, why doesn't imp.find_module() fit the bill?
>>
>>
>> It doesn't handle packages.
>>
>> >>> import compiler.ast
>> >>> import imp
>> >>> imp.find_module('compiler.ast')
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in ?
>> ImportError: No module named compiler.ast
> 
> 
> Sure it does; you just have to ask it nicely:  :)
> 
> Python 2.2.2 (#37, Oct 14 2002, 17:02:34) [MSC 32 bit (Intel)] on win32
> Type "copyright", "credits" or "license" for more information.
> IDLE 0.8 -- press F1 for help
>  >>> import imp
>  >>> imp.find_module('compiler')
> (None, 'C:\\PYTHON22\\lib\\compiler', ('', '', 5))
>  >>> import compiler
>  >>> imp.find_module('ast',compiler.__path__)
> (<open file 'C:\PYTHON22\lib\compiler\ast.py', mode 'r' at 0x00A99660>, 
> 'C:\\PYTHON22\\lib\\compiler\\ast.py', ('.py', 'r', 1))
>  >>>

Right. That's not exactly the API I had in mind though.

You could certainly create a usable API on top of find_module.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From barry at python.org  Tue Jul 27 19:58:58 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Jul 27 19:58:39 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <410694D3.20804@zope.com>
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com> <4106853A.8080109@zope.com>
	<1f7befae04072709592dc58bd9@mail.gmail.com>
	<20040727172030.GA6919@discworld.dyndns.org>
	<1f7befae04072710256595510@mail.gmail.com>
	<1090949879.8672.92.camel@localhost>  <410694D3.20804@zope.com>
Message-ID: <1090951138.8673.134.camel@localhost>

On Tue, 2004-07-27 at 13:45, Jim Fulton wrote:

> > So, why doesn't imp.find_module() fit the bill?
> 
> It doesn't handle packages.

Sounds like a bug to me. ;)

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040727/1a7822b5/attachment.pgp
From jim at zope.com  Tue Jul 27 20:02:20 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 27 20:02:24 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1090951138.8673.134.camel@localhost>
References: <41065006.6020607@zope.com>	
	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>	
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>	
	<1f7befae04072709592dc58bd9@mail.gmail.com>	
	<20040727172030.GA6919@discworld.dyndns.org>	
	<1f7befae04072710256595510@mail.gmail.com>	
	<1090949879.8672.92.camel@localhost> <410694D3.20804@zope.com>
	<1090951138.8673.134.camel@localhost>
Message-ID: <410698AC.5030806@zope.com>

Barry Warsaw wrote:
> On Tue, 2004-07-27 at 13:45, Jim Fulton wrote:
> 
> 
>>>So, why doesn't imp.find_module() fit the bill?
>>
>>It doesn't handle packages.
> 
> 
> Sounds like a bug to me. ;)

It's documented:

   http://docs.python.org/lib/module-imp.html#l2h-688

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From tim.peters at gmail.com  Tue Jul 27 20:03:36 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Jul 27 20:03:38 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <410690F1.2090308@zope.com>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>	<1f7befae040727085715ce2536@mail.gmail.com>	<4106853A.8080109@zope.com>	<1f7befae04072709592dc58bd9@mail.gmail.com>	<20040727172030.GA6919@discworld.dyndns.org>
	<1f7befae04072710256595510@mail.gmail.com> <410690F1.2090308@zope.com>
Message-ID: <1f7befae04072711034fddbb27@mail.gmail.com>

[Tim]
>> Sure!  Jim wants to distinguish "absenceness" from "brokenness" for
>> some reason.  It's not clear to my why

[Jim]
> Because a broken module is something that should get fixed.

Maybe I don't know what "broken" means to you then.  To me it means
things like syntax errors, and module initialization code that raises
runtime exceptions.  If I try to import a module with problems like
those, I don't get ImportError, I get the exception (like SyntaxError
or ZereDivisionError) raised by the broken code in the module, and
catching ImportError in the importer doesn't hide those.  Is the only
case you're concerned about of the "A imports B imports C, and B's
attempt to import C raises an ImportError that's passed on to A"
flavor?  I have one nasty example of that on hand:  Python's site.py
tries to import sitecustomize.py, but if the latter contains a bogus
import then site.py suppresses the error, because it treats the
ImportError as meaning "I guess sitecustomize.py doesn't exist --
that's fine".  That one cost me an hour yesterday!  So I'm becoming a
believer <wink>.

> Hiding errors is a bad thing.  It's like using hasattr. :)

No argument there.
From jim at zope.com  Tue Jul 27 20:07:56 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 27 20:08:01 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1f7befae04072711034fddbb27@mail.gmail.com>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>	<1f7befae040727085715ce2536@mail.gmail.com>	<4106853A.8080109@zope.com>	<1f7befae04072709592dc58bd9@mail.gmail.com>	<20040727172030.GA6919@discworld.dyndns.org>	<1f7befae04072710256595510@mail.gmail.com>
	<410690F1.2090308@zope.com>
	<1f7befae04072711034fddbb27@mail.gmail.com>
Message-ID: <410699FC.5060002@zope.com>

Tim Peters wrote:
> [Tim]
> 
>>>Sure!  Jim wants to distinguish "absenceness" from "brokenness" for
>>>some reason.  It's not clear to my why
> 
> 
> [Jim]
> 
>>Because a broken module is something that should get fixed.
> 
> 
> Maybe I don't know what "broken" means to you then.  To me it means
> things like syntax errors, and module initialization code that raises
> runtime exceptions.  If I try to import a module with problems like
> those, I don't get ImportError,

Yup

...

> Is the only
> case you're concerned about of the "A imports B imports C, and B's
> attempt to import C raises an ImportError that's passed on to A"
> flavor? 

Yes

 > I have one nasty example of that on hand:  Python's site.py
> tries to import sitecustomize.py, but if the latter contains a bogus
> import then site.py suppresses the error, because it treats the
> ImportError as meaning "I guess sitecustomize.py doesn't exist --
> that's fine".  That one cost me an hour yesterday!  So I'm becoming a
> believer <wink>.

:)

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From theller at python.net  Tue Jul 27 19:35:24 2004
From: theller at python.net (Thomas Heller)
Date: Tue Jul 27 20:09:42 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <4106853A.8080109@zope.com> (Jim Fulton's message of "Tue, 27
	Jul 2004 12:39:22 -0400")
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
Message-ID: <1xixz737.fsf@python.net>

Jim Fulton <jim@zope.com> writes:

> Tim Peters wrote:
>> [Jim Fulton]
>> ...
>>
>>>No, it won't.  For example, suppose foo imports B. B tries to import
>>>C, but fails.  B is now broken, but it is still importable.  Actually,
>>>both foo and B can be imported without errors, even though they are
>>>broken.
>> Then you're proposing a way for a highly knowledgable user to
>> anticipate, and partially worm around,  that Python leaves behind
>> insane module objects in sys.modules.
>
> No.  I'm proposing a way for a Python developer to detect the
> presence or absence of a module.
>
> Hm, perhaps it would be better to provide an API (if there isn't one
> already) to test whether a module is present.

Doesn't imp.find_module do this? On Windows:

>>> import imp
>>> imp.find_module("termios")
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ImportError: No module named termios
>>> imp.find_module("pty")
(<open file 'c:\python23\lib\pty.py', mode 'U' at 0x00981760>, 'c:\\python23\\lib\\pty.py', ('.py',
>>> ^Z

Thomas

From walter at livinglogic.de  Tue Jul 27 22:39:45 2004
From: walter at livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=)
Date: Tue Jul 27 22:39:48 2004
Subject: [Python-Dev] Decoding incomplete unicode
Message-ID: <4106BD91.4060009@livinglogic.de>

Pythons unicode machinery currently has problems when decoding
incomplete input.

When codecs.StreamReader.read() encounters a decoding error it
reads more bytes from the input stream and retries decoding.
This is broken for two reasons:
1) The error might be due to a malformed byte sequence in the input,
    a problem that can't be fixed by reading more bytes.
2) There may be no more bytes available at this time. Once more
    data is available decoding can't continue because bytes from
    the input stream have already been read and thrown away.
(sio.DecodingInputFilter has the same problems)

I've uploaded a patch that fixes these problems to SF:
http://www.python.org/sf/998993

The patch implements a few additional features:
- read() has an additional argument chars that can be used to
   specify the number of characters that should be returned.
- readline() is supported on all readers derived from
   codecs.StreamReader().
- readline() and readlines() have an additional option
   for dropping the u"\n".

The patch is still missing changes to the escape codecs
("unicode_escape" and "raw_unicode_escape") and I haven't
touched the CJK codecs, but it has test cases that check
the new functionality for all affected codecs
(UTF-7, UTF-8, UTF-16, UTF-16-LE, UTF-16-BE).

Could someone take a look at the patch?

Bye,
    Walter D?rwald


From guido at python.org  Tue Jul 27 22:48:22 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul 27 22:48:29 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Tue, 27 Jul 2004 13:45:07 EDT."
	<410694A3.7010008@zope.com> 
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com> <1xixz737.fsf@python.net> 
	<410694A3.7010008@zope.com> 
Message-ID: <200407272048.i6RKmN425120@guido.python.org>

> Nope. It doesn't handle packages.
> 
>  >>> import compiler.ast
>  >>> import imp
>  >>> imp.find_module('compiler.ast')
> Traceback (most recent call last):
>    File "<stdin>", line 1, in ?
> ImportError: No module named compiler.ast

The docs explain how to handle that though.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Jul 27 22:50:14 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul 27 22:52:50 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Tue, 27 Jul 2004 13:09:03 EDT."
	<41068C2F.40102@zope.com> 
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com> 
	<41068C2F.40102@zope.com> 
Message-ID: <200407272050.i6RKoEY25164@guido.python.org>

> An even simpler aproach would be to have a ModuleNotFound error
> that extends ImportError and gets raised if the module can't
> be found.

That wouldn't solve anything, because if foo brokenly tries to import
bar and that raises ModuleNotFound, so will "import foo".

--Guido van Rossum (home page: http://www.python.org/~guido/)
From mal at egenix.com  Tue Jul 27 22:59:59 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue Jul 27 23:00:25 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4106BD91.4060009@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>
Message-ID: <4106C24F.3070203@egenix.com>

Walter D?rwald wrote:
> Pythons unicode machinery currently has problems when decoding
> incomplete input.
> 
> When codecs.StreamReader.read() encounters a decoding error it
> reads more bytes from the input stream and retries decoding.
> This is broken for two reasons:
> 1) The error might be due to a malformed byte sequence in the input,
>    a problem that can't be fixed by reading more bytes.
> 2) There may be no more bytes available at this time. Once more
>    data is available decoding can't continue because bytes from
>    the input stream have already been read and thrown away.
> (sio.DecodingInputFilter has the same problems)
> 
> I've uploaded a patch that fixes these problems to SF:
> http://www.python.org/sf/998993
> 
> The patch implements a few additional features:
> - read() has an additional argument chars that can be used to
>   specify the number of characters that should be returned.
> - readline() is supported on all readers derived from
>   codecs.StreamReader().
> - readline() and readlines() have an additional option
>   for dropping the u"\n".
> 
> The patch is still missing changes to the escape codecs
> ("unicode_escape" and "raw_unicode_escape") and I haven't
> touched the CJK codecs, but it has test cases that check
> the new functionality for all affected codecs
> (UTF-7, UTF-8, UTF-16, UTF-16-LE, UTF-16-BE).
> 
> Could someone take a look at the patch?

Just did... please see the comments in the SF tracker.

I like the idea, but don't think the implementation is
the right way to do it. Instead, I'd suggest using a new
error handling strategy "break" ( = break processing as
soon as errors are found).

The advantage of this approach is twofold:

* no new APIs or API changes are required

* other codecs (including third-party ones) can easily
   implement the same strategy

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 27 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From guido at python.org  Tue Jul 27 22:43:44 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul 27 23:04:14 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Tue, 27 Jul 2004 11:57:01 EDT."
	<1f7befae040727085715ce2536@mail.gmail.com> 
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com> 
	<1f7befae040727085715ce2536@mail.gmail.com> 
Message-ID: <200407272043.i6RKhiD24954@guido.python.org>

> Then you're proposing a way for a highly knowledgable user to
> anticipate, and partially worm around, that Python leaves behind
> insane module objects in sys.modules.  Wouldn't it be better to change
> Python to stop leaving insane module objects in sys.modules to begin
> with?  That's harder, but seems to me it would do a lot more good for
> a lot more people.

+1

--Guido van Rossum (home page: http://www.python.org/~guido/)
From martin at v.loewis.de  Tue Jul 27 23:15:00 2004
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Jul 27 23:15:25 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4106C24F.3070203@egenix.com>
References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com>
Message-ID: <4106C5D4.1020200@v.loewis.de>

M.-A. Lemburg wrote:
> I like the idea, but don't think the implementation is
> the right way to do it. Instead, I'd suggest using a new
> error handling strategy "break" ( = break processing as
> soon as errors are found).

Can you demonstrate this approach in a patch? I think it
is unimplementable: the codec cannot communicate to the
error callback that it ran out of data.

Regards,
Martin
From guido at python.org  Tue Jul 27 22:47:32 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul 27 23:23:16 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Tue, 27 Jul 2004 13:29:21 EDT."
	<410690F1.2090308@zope.com> 
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<1f7befae04072709592dc58bd9@mail.gmail.com>
	<20040727172030.GA6919@discworld.dyndns.org>
	<1f7befae04072710256595510@mail.gmail.com> 
	<410690F1.2090308@zope.com> 
Message-ID: <200407272047.i6RKlW925105@guido.python.org>

> Because a broken module is something that should get fixed.

But nine out of ten times, a module's absence is caused by breakage
somewhere (e.g. a bad path).

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Jul 27 22:44:15 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Jul 27 23:28:06 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Tue, 27 Jul 2004 12:39:22 EDT."
	<4106853A.8080109@zope.com> 
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com> 
	<4106853A.8080109@zope.com> 
Message-ID: <200407272044.i6RKiFq24964@guido.python.org>

> No.  I'm proposing a way for a Python developer to detect the
> presence or absence of a module.
> 
> Hm, perhaps it would be better to provide an API (if there isn't one
> already) to test whether a module is present.
> 
> AFAIK, the only way to do it is by trying the import.  Unfortunately,
> the error type alone isn't enough to tell whether the error you
> caught is due to the error condition you are trying to test.
> 
> Perhaps providing an API is a better way to go.

This exists as imp.find_module.

> In general, however, I hate the approach of formatting errors
> as they are raised, rather than as they are converted to strings.

Agreed.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From mal at egenix.com  Tue Jul 27 23:43:28 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue Jul 27 23:43:31 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4106C5D4.1020200@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com>
	<4106C5D4.1020200@v.loewis.de>
Message-ID: <4106CC80.4070401@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
> 
>> I like the idea, but don't think the implementation is
>> the right way to do it. Instead, I'd suggest using a new
>> error handling strategy "break" ( = break processing as
>> soon as errors are found).
> 
> 
> Can you demonstrate this approach in a patch? I think it
> is unimplementable: the codec cannot communicate to the
> error callback that it ran out of data.

The codec can: the callback gets all the necessary information
and can even manipulate the objects being worked on.

But you have a point: the current implementations of the
various encode/decode functions don't provide interfaces to
report back the number of bytes read at C level - the codecs
module wrappers add these numbers assuming that all bytes
were read.

The error callbacks could, however, raise an exception which
includes all the needed information, including any state that
may be needed in order to continue with coding operation.

We may then need to allow additional keyword arguments on the
encode/decode functions in order to preset a start state.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 27 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From tdelaney at avaya.com  Tue Jul 27 23:49:08 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Tue Jul 27 23:49:48 2004
Subject: [Python-Dev] Fix import errors to have data
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com>

Guido van Rossum wrote:

>> Then you're proposing a way for a highly knowledgable user to
>> anticipate, and partially worm around, that Python leaves behind
>> insane module objects in sys.modules.  Wouldn't it be better to
>> change Python to stop leaving insane module objects in sys.modules
>> to begin with?  That's harder, but seems to me it would do a lot
>> more good for a lot more people.
> 
> +1

OK - the problem as I see it is that a given module that exists, but
raises ImportError, only raises ImportError *once*, whereas it really
should raise ImportError every time i.e. currently doing the following:

    # a.py
    raise ImportError

    # b.py

    try:
        import a
    except ImportError:
        import a

    print 'OK'

prints when it really should raise ImportError.

Additionally, a module could have messed with sys.modules directly, so
just putting None in for the broken module wouldn't be sufficient (you
could possibly get the broken module via another name).

Perhaps the import machinery could keep a (weak?) mapping from module
*object* to the ImportError it raised. If the module that would be
returned by the import is in the mapping, raise the corresponding
ImportError instead. The appropriate line, etc in the module would thus
be shown each time the module was imported.

I believe this would work properly for reload() as well.

Tim Delaney
From jim at zope.com  Tue Jul 27 23:55:26 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Jul 27 23:55:31 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200407272048.i6RKmN425120@guido.python.org>
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com> <1xixz737.fsf@python.net>
	<410694A3.7010008@zope.com>
	<200407272048.i6RKmN425120@guido.python.org>
Message-ID: <4106CF4E.5070307@zope.com>

Guido van Rossum wrote:
>>Nope. It doesn't handle packages.
>>
>> >>> import compiler.ast
>> >>> import imp
>> >>> imp.find_module('compiler.ast')
>>Traceback (most recent call last):
>>   File "<stdin>", line 1, in ?
>>ImportError: No module named compiler.ast
> 
> 
> The docs explain how to handle that though.

Sure.  But it would be better to have an API
that you could just call to find out if a module is present.
This could be implemented on top of find_module.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From barry at python.org  Wed Jul 28 00:00:26 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Jul 28 00:00:36 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <4106CF4E.5070307@zope.com>
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com> <4106853A.8080109@zope.com>
	<1xixz737.fsf@python.net> <410694A3.7010008@zope.com>
	<200407272048.i6RKmN425120@guido.python.org>
	<4106CF4E.5070307@zope.com>
Message-ID: <1090965626.32016.4.camel@localhost>

On Tue, 2004-07-27 at 17:55, Jim Fulton wrote:

> Sure.  But it would be better to have an API
> that you could just call to find out if a module is present.
> This could be implemented on top of find_module.

If a convenience function is desired, I'd recommend adding it to the imp
module.  Call it something like imp.package_or_module_exists() <wink>.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040727/0fac6b0c/attachment-0001.pgp
From bac at OCF.Berkeley.EDU  Wed Jul 28 01:58:02 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Jul 28 01:58:17 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <20040727164913.GS7708@epoch.metaslash.com>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>	<1f7befae040727085715ce2536@mail.gmail.com>	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
Message-ID: <4106EC0A.2000401@ocf.berkeley.edu>

Neal Norwitz wrote:
> On Tue, Jul 27, 2004 at 12:39:22PM -0400, Jim Fulton wrote:
> 
>>Tim Peters wrote:
>>
>>>[Jim Fulton]
>>>
>>>
>>>>No, it won't.  For example, suppose foo imports B. B tries to import
>>>>C, but fails.  B is now broken, but it is still importable.  Actually,
>>>>both foo and B can be imported without errors, even though they are
>>>>broken.
>>
>>>Then you're proposing a way for a highly knowledgable user to
>>>anticipate, and partially worm around,  that Python leaves behind
>>>insane module objects in sys.modules. 
>>
>>No.  I'm proposing a way for a Python developer to detect the
>>presence or absence of a module.
>>
>>Hm, perhaps it would be better to provide an API (if there isn't one
>>already) to test whether a module is present.
> 
> 
> Another possibility could be to add a ChainedImportError (or
> SubImportError?) which would derive from ImportError.  The module
> which imports a broken module would get the normal ImportError.  If
> the module doesn't catch the exception, it would get converted to this
> new type of ImportError.
> 
> I don't know how easy this would be to implement, but I think it
> would solve Jim's problem and perhaps be easier to deal with?
> 

Having exceptions keeping track of previous exceptions might also work. 
  I know that idea was brought up before last year (see 
http://www.python.org/dev/summary/2003-01-16_2003-01-31.html#idea-for-avoiding-exception-masking 
for the historically accurate summary of the thread).  Idea died when no 
one could come up with a good way of storing previous exception info.

-Brett
From tdelaney at avaya.com  Wed Jul 28 02:27:01 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Wed Jul 28 02:27:08 2004
Subject: [Python-Dev] Fix import errors to have data
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A305@au3010avexu1.global.avaya.com>

Delaney, Timothy C (Timothy) wrote:

> Perhaps the import machinery could keep a (weak?) mapping from module
> *object* to the ImportError it raised. If the module that would be
> returned by the import is in the mapping, raise the corresponding
> ImportError instead. The appropriate line, etc in the module would
> thus 
> be shown each time the module was imported.
> 
> I believe this would work properly for reload() as well.

Of course it won't ... (no name to reload). This would effectively
prevent ever loading that module in that session, even if it got fixed.

Instead, perhaps the following would be the right semantics:

    # import replacement

    import __builtin__
    import sys

    _builtin_import = __import__

    def __import__(name, *p):
        try:
            return _builtin_import(name, *p)
        except ImportError, e:
            module = sys.modules[name]

            for n, m in sys.modules.items():
                if m is module:
                    del sys.modules[n]

            raise e

    __builtin__.__import__ = __import__

    # a.py

    raise ImportError(__name__)

    # b.py

    try:
        import a
    except ImportError:
        import a

    print 'OK'

    >>> try:
    ...     import a
    ... except ImportError:
    ...     import a
    ...
    Traceback (most recent call last):
      File "<stdin>", line 4, in ?
      File "<stdin>", line 9, in __import__
    ImportError: a

Tim Delaney
From ncoghlan at iinet.net.au  Wed Jul 28 02:52:32 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Jul 28 02:53:09 2004
Subject: [Python-Dev] Non-Visual Studio builds on Windows XP?
In-Reply-To: <40FAEF8D.6020202@garthy.com>
References: <40F914A7.5070804@iinet.net.au> <40F9B0AD.8010109@v.loewis.de>
	<40FA45B2.8050804@iinet.net.au> <40FAEF8D.6020202@garthy.com>
Message-ID: <4106F8D0.2060301@iinet.net.au>

Garth wrote:
> If these scripts don't work I'm happy to help out.

Well, I spent more time with it without luck - however, it appeared to 
be due to the fact that neither the free VC 2003 toolkit nor the .NET 
toolkit have a version of windows.h

Rather than muck around further, I decided it would be more productive 
to just spend the time required to get Linux working on my laptop and 
dual boot with XP (seeing that the main purpose of this exercise was to 
get a Python devel environment on the laptop).

Thanks for the offer, though.

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From fperez528 at yahoo.com  Wed Jul 28 03:27:35 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Wed Jul 28 03:27:39 2004
Subject: [Python-Dev] Rationale behind removing kwdict from keyword.py?
Message-ID: <ce6ve7$g4m$1@sea.gmane.org>

Hi all,

I recently got a report from a user of a project of mine breaking under 2.4a1. 
The reason is that keyword stopped having a kwdict variable in it.  Is there
any rationale behind this change?  I know I can always work around it, but
this honestly feels, to me, like gratuitous breakage.

It would be nice if things like public members (it wasn't called _kwdict, so I
had no reason to assume it was off-limits) of modules in the standard library
were kept stably across releases unless there is a good reason to break
compatibility.  While the html docs for keyword don't mention kwdict, I often
simply use help(module) to look for what I need, and I saw kwdict happily
listed there, without any leading _ to indicate any sense of it being private.

The changes were made, according to viewCVS, in revision 1.13.  I know I can
use iskeyword() for my purposes, and patch accordingly.  But I'd like to
remind the core developers that when things go into the stdlib, people start
using them.  Removing stuff will _always_ break code.

I'll gladly file a SF bug report against this, but I'd like not to waste
anyone's time if it won't be considered.

Thanks to all.

Best,

Fernando Perez.

From jcarlson at uci.edu  Wed Jul 28 03:26:04 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed Jul 28 03:32:20 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4106BD91.4060009@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>
Message-ID: <20040727181432.576B.JCARLSON@uci.edu>


> - readline() and readlines() have an additional option
>    for dropping the u"\n".

I'll point out (since no one else has so far) that the reasons for
keeping the linefeed at the end of lines returned by readline() and
readlines() are documented here:
http://docs.python.org/lib/bltin-file-objects.html#foot3271

Specifically it allows one to use the following and have it "do the
right thing".

while 1:
    line = obj.readline()
    if not line:
        break
    process(line)

Having the option of readline() and readlines() being ambiguous anywhere
sounds like a misfeature.  Furthermore, since all other readline and
readlines calls that do not inherit from StreamReader use the
unambiguous "always include the line ending", changing StreamReader to
be different is obviously the wrong thing to do.

 - Josiah

From jepler at unpythonic.net  Wed Jul 28 04:01:08 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Wed Jul 28 04:01:18 2004
Subject: [Python-Dev] Rationale behind removing kwdict from keyword.py?
In-Reply-To: <ce6ve7$g4m$1@sea.gmane.org>
References: <ce6ve7$g4m$1@sea.gmane.org>
Message-ID: <20040728020108.GA4249@unpythonic.net>

In 2.3, __all__ = ["iskeyword", "kwlist"]

In the absence of separate documentation, I think __all__ may be the
best hint you have about what the "external API" of a module is.

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040727/1619888e/attachment.pgp
From tim.peters at gmail.com  Wed Jul 28 04:13:24 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Wed Jul 28 04:13:27 2004
Subject: [Python-Dev] Rationale behind removing kwdict from keyword.py?
In-Reply-To: <ce6ve7$g4m$1@sea.gmane.org>
References: <ce6ve7$g4m$1@sea.gmane.org>
Message-ID: <1f7befae04072719134056c647@mail.gmail.com>

[Fernando Perez]
> I recently got a report from a user of a project of mine breaking under 2.4a1.
> The reason is that keyword stopped having a kwdict variable in it.  Is there
> any rationale behind this change?

Looking at the code, because keyword.py no longer uses a dictionary
(Raymond changed it to use a set), and kwdict wasn't part of keyword's
public API so there was no reason to avoid changing this.

>  I know I can always work around it, but this honestly feels, to me, like
> gratuitous breakage.
>
> It would be nice if things like public members (it wasn't called _kwdict, so I
> had no reason to assume it was off-limits) ...

You actually had two reasons, and strong ones.  As you noted later, it
wasn't part of keyword's documented interface, and you *always* act at
your own risk when you go beyond the docs.

The second reason is that 'kwdict' didn't appear in the
keyword.__all__ list.  As the docs for Python's import statement say:

    The public names defined by a module are determined by checking the
    module's namespace for a variable named __all__; if defined, it must be a
    sequence of strings which are names defined or imported by that module.
    The names given in __all__ are all considered public and are
required to exist.
    If __all__ is not defined, the set of public names includes all
names found in
    the module's namespace which do not begin with an underscore
character ("_").
    __all__ should contain the entire public API. It is intended to
avoid accidentally
    exporting items that are not part of the API (such as library modules which
    were imported and used within the module).

Note that it's very common in older modules (which keyword.py is) to
define the newer __all__ rather than risk breaking code by massive
renaming of variables to stick an underscore in front of them.

> ...
> While the html docs for keyword don't mention kwdict, I often simply use
> help(module) to look for what I need, and I saw kwdict happily
> listed there, without any leading _ to indicate any sense of it being private.

You should stop doing that, then -- this is the kind of trouble it
leads to, and you'll only get more of it.  help(keyword) did advertise
the keyword.__all__ value in 2.3.4:

...
DATA
    __all__ = ['iskeyword', 'kwlist']
    keyword = 'yield'
    kwdict = {'and': 1, 'assert': 1, 'break': 1, 'class': 1, 'continue': 1...
    kwlist = ['and', 'assert', 'break', 'class', 'continue', 'def', 'del',...

Since 'keyword' and 'kwdict' weren't in __all__, you couldn't rely on
either of them; and, indeed, they're both gone in 2.4, while
keyword.__all__ hasn't changed.

> The changes were made, according to viewCVS, in revision 1.13.  I know I can
> use iskeyword() for my purposes, and patch accordingly.  But I'd like to
> remind the core developers that when things go into the stdlib, people start
> using them.  Removing stuff will _always_ break code.

The developers were careful to keep the public API in this case
exactly the same.

> I'll gladly file a SF bug report against this, but I'd like not to waste
> anyone's time if it won't be considered.

Sorry, no, I would not consider this one.
From bob at redivi.com  Wed Jul 28 04:20:24 2004
From: bob at redivi.com (Bob Ippolito)
Date: Wed Jul 28 04:20:32 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <20040727181432.576B.JCARLSON@uci.edu>
References: <4106BD91.4060009@livinglogic.de>
	<20040727181432.576B.JCARLSON@uci.edu>
Message-ID: <AF2FDF21-E03C-11D8-838F-000A95686CD8@redivi.com>

On Jul 27, 2004, at 9:26 PM, Josiah Carlson wrote:

>
>> - readline() and readlines() have an additional option
>>    for dropping the u"\n".
>
> I'll point out (since no one else has so far) that the reasons for
> keeping the linefeed at the end of lines returned by readline() and
> readlines() are documented here:
> http://docs.python.org/lib/bltin-file-objects.html#foot3271
>
> Specifically it allows one to use the following and have it "do the
> right thing".
>
> while 1:
>     line = obj.readline()
>     if not line:
>         break
>     process(line)
>
> Having the option of readline() and readlines() being ambiguous 
> anywhere
> sounds like a misfeature.  Furthermore, since all other readline and
> readlines calls that do not inherit from StreamReader use the
> unambiguous "always include the line ending", changing StreamReader to
> be different is obviously the wrong thing to do.

While this reasoning makes sense for readline(), it most definitely 
does not for readlines() or __iter__().  unicode objects have a 
splitlines() function with this feature, which is probably what he's 
using in his implementation (I used it in mine), so it's pretty natural 
to expose that option to the external interface.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3589 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040727/70120db2/smime-0001.bin
From tim.peters at gmail.com  Wed Jul 28 04:40:45 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Wed Jul 28 04:40:47 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com>
Message-ID: <1f7befae04072719401df511c3@mail.gmail.com>

[Delaney, Timothy C (Timothy)]
> OK - the problem as I see it is that a given module that exists, but
> raises ImportError, only raises ImportError *once*, whereas it really
> should raise ImportError every time

Jim's after something different, while the problem you're thinking
about is more general than just ImportError.  When an import of an
existing module fails for *any* reason, subsequent attempts to import
the same module succeed.  For example,

C:\Code>type a.py
1/0

C:\Code>\python23\python.exe
Python 2.3.4 (#53, May 25 2004, 21:17:02) [MSC v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import a
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "a.py", line 1, in ?
    1/0
ZeroDivisionError: integer division or modulo by zero
>>> import a
>>>

This is Bad, because an uncaught exception in module initialization
means the module probably can't fulfill its contract, yet subsequent
importers get no clue that the module is in a damaged state (until the
module fails to do its job, which may or may not be obvious when it
occurs).  A module failing to import because it suffers an ImportError
itself is once instance of this, but the same applies to a module
failing to import for any other reason:  in all these cases,
subsequent imports don't get the exception the initial importer saw,
they get a module object in an arbitrarily screwed-up state.
From tdelaney at avaya.com  Wed Jul 28 04:57:30 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Wed Jul 28 04:57:37 2004
Subject: [Python-Dev] Fix import errors to have data
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A3CA@au3010avexu1.global.avaya.com>

Tim Peters wrote:

> [Delaney, Timothy C (Timothy)]
>> OK - the problem as I see it is that a given module that exists, but
>> raises ImportError, only raises ImportError *once*, whereas it really
>> should raise ImportError every time
> 
> Jim's after something different, while the problem you're thinking
> about is more general than just ImportError.  When an import of an
> existing module fails for *any* reason, subsequent attempts to import
> the same module succeed.  For example,

Good point. This is the same as not wanting modules in an insane state
in sys.modules.

I've just gone back over Jim's original email, and it is a different
case being talked about there - specifically, determining if an
ImportError was thrown by the module that was specifically being
imported, rather than something it imports.

However, I don't see the utility of that. I definitely can't think of a
use case for it.

OTOH, ensuring that if *any* exception is thrown when importing a
module, it will not appear in sys.modules (and the exception is raised
each time you try to import) is IMO a very useful property, and I think
would solve what I think is Jim's issue at the root.

Tim Delaney
From hyeshik at gmail.com  Wed Jul 28 05:51:30 2004
From: hyeshik at gmail.com (Hye-Shik Chang)
Date: Wed Jul 28 05:51:33 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4106BD91.4060009@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>
Message-ID: <4f0b69dc04072720514dd3ab92@mail.gmail.com>

On Tue, 27 Jul 2004 22:39:45 +0200, Walter D?rwald
<walter@livinglogic.de> wrote:
> Pythons unicode machinery currently has problems when decoding
> incomplete input.
> 
> When codecs.StreamReader.read() encounters a decoding error it
> reads more bytes from the input stream and retries decoding.
> This is broken for two reasons:
> 1) The error might be due to a malformed byte sequence in the input,
>     a problem that can't be fixed by reading more bytes.
> 2) There may be no more bytes available at this time. Once more
>     data is available decoding can't continue because bytes from
>     the input stream have already been read and thrown away.
> (sio.DecodingInputFilter has the same problems)

StreamReaders and -Writers from CJK codecs are not suffering from
this problems because they have internal buffer for keeping states
and incomplete bytes of a sequence. In fact, CJK codecs has its
own implementation for UTF-8 and UTF-16 on base of its multibytecodec
system.  It provides a "working" StreamReader/Writer already. :)

> I've uploaded a patch that fixes these problems to SF:
> http://www.python.org/sf/998993
> 
> The patch implements a few additional features:
> - read() has an additional argument chars that can be used to
>    specify the number of characters that should be returned.
> - readline() is supported on all readers derived from
>    codecs.StreamReader().

I have no comment for these, yet.

> - readline() and readlines() have an additional option
>    for dropping the u"\n".

+1

I wonder whether we need to add optional argument for writelines()
to add newline characters for each lines, then.


Hye-Shik
From fperez528 at yahoo.com  Wed Jul 28 05:56:25 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Wed Jul 28 05:56:28 2004
Subject: [Python-Dev] Re: Rationale behind removing kwdict from keyword.py?
References: <ce6ve7$g4m$1@sea.gmane.org>
	<1f7befae04072719134056c647@mail.gmail.com>
Message-ID: <ce7857$tn0$1@sea.gmane.org>

Tim Peters wrote:

> Note that it's very common in older modules (which keyword.py is) to
> define the newer __all__ rather than risk breaking code by massive
> renaming of variables to stick an underscore in front of them.

Fair enough.  For a long time I'd gotten used to assuming that the lack of a
leading _ was a safe sign of 'publicness' (that's probably not a word :), and
didn't really rely too much on __all__.  Lesson learned... (I wouldn't be
surprised to find I'm not the only one with this bad habit)

Thanks,  I already committed a patch for my stuff anyway, since there's no point
in making my users unhappy.

Best regards,

f

From fperez528 at yahoo.com  Wed Jul 28 06:09:35 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Wed Jul 28 06:09:37 2004
Subject: [Python-Dev] Re: Rationale behind removing kwdict from keyword.py?
References: <ce6ve7$g4m$1@sea.gmane.org> <20040728020108.GA4249@unpythonic.net>
Message-ID: <ce78tt$ukh$1@sea.gmane.org>

Jeff Epler wrote:

> In 2.3, __all__ = ["iskeyword", "kwlist"]
> 
> In the absence of separate documentation, I think __all__ may be the
> best hint you have about what the "external API" of a module is.

Indeed, lesson learned.  In my slim defense, seeing in help(keyword):

DATA
    __all__ = ['iskeyword', 'kwlist']
    keyword = 'yield'
    kwdict = {'and': 1, 'assert': 1, 'break': 1, 'class': 1, 'continue': 1...
    kwlist = ['and', 'assert', 'break', 'class', 'continue', 'def', 'del',...

I (mistakenly) assumed kwdict was safe for usage.  Relying on help(), while I
acknowledge may have been mistaken, is not IMHO a completely farfetched error. 
Oh well, live and learn :)

Thanks to all for the clarifications.  I'll be far more careful in the future
with my usage of stuff from the stdlib.

Best,

f

From bob at redivi.com  Wed Jul 28 06:20:42 2004
From: bob at redivi.com (Bob Ippolito)
Date: Wed Jul 28 06:20:48 2004
Subject: [Python-Dev] Re: Rationale behind removing kwdict from keyword.py?
In-Reply-To: <ce78tt$ukh$1@sea.gmane.org>
References: <ce6ve7$g4m$1@sea.gmane.org> <20040728020108.GA4249@unpythonic.net>
	<ce78tt$ukh$1@sea.gmane.org>
Message-ID: <7D4E6283-E04D-11D8-8016-000A95686CD8@redivi.com>


On Jul 28, 2004, at 12:09 AM, Fernando Perez wrote:

> Jeff Epler wrote:
>
>> In 2.3, __all__ = ["iskeyword", "kwlist"]
>>
>> In the absence of separate documentation, I think __all__ may be the
>> best hint you have about what the "external API" of a module is.
>
> Indeed, lesson learned.  In my slim defense, seeing in help(keyword):
>
> DATA
>     __all__ = ['iskeyword', 'kwlist']
>     keyword = 'yield'
>     kwdict = {'and': 1, 'assert': 1, 'break': 1, 'class': 1, 
> 'continue': 1...
>     kwlist = ['and', 'assert', 'break', 'class', 'continue', 'def', 
> 'del',...
>
> I (mistakenly) assumed kwdict was safe for usage.  Relying on help(), 
> while I
> acknowledge may have been mistaken, is not IMHO a completely 
> farfetched error.
> Oh well, live and learn :)
>
> Thanks to all for the clarifications.  I'll be far more careful in the 
> future
> with my usage of stuff from the stdlib.

FWIW, I think I've made the same mistake before for the same reasons.  
This sounds like a bug in pydoc.  If __all__ is defined, help should 
probably only be given for items in it (public API).

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3589 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040728/53ebb933/smime.bin
From skip at pobox.com  Wed Jul 28 06:46:19 2004
From: skip at pobox.com (Skip Montanaro)
Date: Wed Jul 28 06:56:40 2004
Subject: [Python-Dev] Re: Rationale behind removing kwdict from keyword.py?
In-Reply-To: <7D4E6283-E04D-11D8-8016-000A95686CD8@redivi.com>
References: <ce6ve7$g4m$1@sea.gmane.org> <20040728020108.GA4249@unpythonic.net>
	<ce78tt$ukh$1@sea.gmane.org>
	<7D4E6283-E04D-11D8-8016-000A95686CD8@redivi.com>
Message-ID: <16647.12187.864822.158184@montanaro.dyndns.org>


    Bob> This sounds like a bug in pydoc.  If __all__ is defined, help
    Bob> should probably only be given for items in it (public API).

Funny you should mention that:

    revision 1.92
    date: 2004/06/11 04:46:12;  author: montanaro;  state: Exp;  lines: +22 -9
    Respect a module's __all__ attribute.  Closes #969938.

and the corresponding output since then for the keyword module looks like:

    NAME
        keyword - Keywords (from "graminit.c")

    FILE
        /Users/skip/local/lib/python2.4/keyword.py

    MODULE DOCS
        http://www.python.org/doc/current/lib/module-keyword.html

    DESCRIPTION
        This file is automatically generated; please don't muck it up!

        To update the symbols in this file, 'cd' to the top directory of
        the python source tree after building the interpreter and run:

            python Lib/keyword.py

    FUNCTIONS
        iskeyword = __contains__(...)
            x.__contains__(y) <==> y in x.

    DATA
        __all__ = ['iskeyword', 'kwlist']
        kwlist = ['and', 'assert', 'break', 'class', 'continue', 'def', 'del',...

while the output using Python 2.3 is

    NAME
        keyword - Keywords (from "graminit.c")

    FILE
        /Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/keyword.py

    DESCRIPTION
        This file is automatically generated; please don't muck it up!

        To update the symbols in this file, 'cd' to the top directory of
        the python source tree after building the interpreter and run:

            python Lib/keyword.py

    FUNCTIONS
        iskeyword = has_key(...)
            D.has_key(k) -> 1 if D has a key k, else 0

        main()

    DATA
        __all__ = ['iskeyword', 'kwlist']
        keyword = 'yield'
        kwdict = {'and': 1, 'assert': 1, 'break': 1, 'class': 1, 'continue': 1...
        kwlist = ['and', 'assert', 'break', 'class', 'continue', 'def', 'del',...

Skip
From fperez528 at yahoo.com  Wed Jul 28 07:10:40 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Wed Jul 28 07:10:47 2004
Subject: [Python-Dev] Re: Re: Rationale behind removing kwdict from
	keyword.py?
References: <ce6ve7$g4m$1@sea.gmane.org> <20040728020108.GA4249@unpythonic.net>
	<ce78tt$ukh$1@sea.gmane.org>
	<7D4E6283-E04D-11D8-8016-000A95686CD8@redivi.com>
	<16647.12187.864822.158184@montanaro.dyndns.org>
Message-ID: <ce7cgg$4br$1@sea.gmane.org>

Skip Montanaro wrote:

> 
>     Bob> This sounds like a bug in pydoc.  If __all__ is defined, help
>     Bob> should probably only be given for items in it (public API).
> 
> Funny you should mention that:
> 
>     revision 1.92
>     date: 2004/06/11 04:46:12;  author: montanaro;  state: Exp;  lines: +22 -9
>     Respect a module's __all__ attribute.  Closes #969938.
> 
> and the corresponding output since then for the keyword module looks like:
[...]

Aha!  Maybe I'm not a complete idiot after all :)

Glad to see this was fixed.  It's always a joy to see the time machine in
action, even when operated by someone other than Guido.

Best,

f

From martin at v.loewis.de  Wed Jul 28 08:53:55 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Jul 28 08:53:53 2004
Subject: [Python-Dev] Non-Visual Studio builds on Windows XP?
In-Reply-To: <4106F8D0.2060301@iinet.net.au>
References: <40F914A7.5070804@iinet.net.au>
	<40F9B0AD.8010109@v.loewis.de>	<40FA45B2.8050804@iinet.net.au>
	<40FAEF8D.6020202@garthy.com> <4106F8D0.2060301@iinet.net.au>
Message-ID: <41074D83.7070403@v.loewis.de>

Nick Coghlan wrote:
> Well, I spent more time with it without luck - however, it appeared to 
> be due to the fact that neither the free VC 2003 toolkit nor the .NET 
> toolkit have a version of windows.h

Ah. You need, of course, the platform SDK as well (downloadable only
with internet exploder).

http://www.microsoft.com/msdownload/platformsdk/sdkupdate/

Regards,
Martin
From ncoghlan at iinet.net.au  Wed Jul 28 09:37:37 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Jul 28 09:38:16 2004
Subject: [Python-Dev] Non-Visual Studio builds on Windows XP?
In-Reply-To: <41074D83.7070403@v.loewis.de>
References: <40F914A7.5070804@iinet.net.au>
	<40F9B0AD.8010109@v.loewis.de>	<40FA45B2.8050804@iinet.net.au>
	<40FAEF8D.6020202@garthy.com> <4106F8D0.2060301@iinet.net.au>
	<41074D83.7070403@v.loewis.de>
Message-ID: <410757C1.6040206@iinet.net.au>

Martin v. L?wis wrote:
> Nick Coghlan wrote:
> 
>> Well, I spent more time with it without luck - however, it appeared to 
>> be due to the fact that neither the free VC 2003 toolkit nor the .NET 
>> toolkit have a version of windows.h
> 
> 
> Ah. You need, of course, the platform SDK as well (downloadable only
> with internet exploder).

Ah, that's probably why I was having so much trouble finding that one (I 
was using Firefox).

(I was sure I'd seen the platform SDK available for download at some 
point, but had no luck finding it again. So thanks for the link!)

Well, maybe I'll give it another try at some point when I'm taking a 
break from trying to persuade Fedora to talk to all the builtin devices 
in my notebook ;)

(Besides, it would give me the ability to test changes on Windows, even 
if most of my development ends up being on Fedora)

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From walter at livinglogic.de  Wed Jul 28 10:29:00 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Jul 28 10:29:03 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A3CA@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A3CA@au3010avexu1.global.avaya.com>
Message-ID: <410763CC.5000908@livinglogic.de>

Delaney, Timothy C (Timothy) wrote:
> Tim Peters wrote:
> 
> 
>>[Delaney, Timothy C (Timothy)]
>>
>>>OK - the problem as I see it is that a given module that exists, but
>>>raises ImportError, only raises ImportError *once*, whereas it really
>>>should raise ImportError every time
>>
>>Jim's after something different, while the problem you're thinking
>>about is more general than just ImportError.  When an import of an
>>existing module fails for *any* reason, subsequent attempts to import
>>the same module succeed.  For example,
> 
> Good point. This is the same as not wanting modules in an insane state
> in sys.modules.

This simplest solution for this problem would be to put the module
in sys.modules only after executing the module's code finishes.
Unfortunately this breaks circular imports, because it would lead
to infinite recursion.

This looks like we can't have both: Having a module available only
after importing it works and having circular imports where one
module uses another module that is only half initialized.

Bye,
    Walter D?rwald

From walter at livinglogic.de  Wed Jul 28 10:36:38 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Jul 28 10:36:41 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4106C5D4.1020200@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com>
	<4106C5D4.1020200@v.loewis.de>
Message-ID: <41076596.6070106@livinglogic.de>

Martin v. L?wis wrote:

> M.-A. Lemburg wrote:
> 
>> I like the idea, but don't think the implementation is
>> the right way to do it. Instead, I'd suggest using a new
>> error handling strategy "break" ( = break processing as
>> soon as errors are found).
> 
> Can you demonstrate this approach in a patch? I think it
> is unimplementable: the codec cannot communicate to the
> error callback that it ran out of data.

We would need a special attribute in the exception for
that, but the problem IMHO is a different one. This makes
it impossible to use other error handling schemes than
"break" in stateful decoders.

Bye,
    Walter D?rwald


From mal at egenix.com  Wed Jul 28 11:16:53 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Jul 28 11:16:55 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <41076596.6070106@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>
	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>
	<41076596.6070106@livinglogic.de>
Message-ID: <41076F05.7020300@egenix.com>

Walter D?rwald wrote:
> Martin v. L?wis wrote:
> 
>> M.-A. Lemburg wrote:
>>
>>> I like the idea, but don't think the implementation is
>>> the right way to do it. Instead, I'd suggest using a new
>>> error handling strategy "break" ( = break processing as
>>> soon as errors are found).
>>
>>
>> Can you demonstrate this approach in a patch? I think it
>> is unimplementable: the codec cannot communicate to the
>> error callback that it ran out of data.
> 
> 
> We would need a special attribute in the exception for
> that, but the problem IMHO is a different one. This makes
> it impossible to use other error handling schemes than
> "break" in stateful decoders.

I don't understand... are you referring to some extra
attribute for storing arbitrary state ? If so, why would
adding such an attribute make it impossible to use
other error handling schemes ?

The problem with your patch is that you are adding a whole
new set of decoders to the core which duplicate much of what
the already existing decoders implement. I don't like that
duplication and would like to find a way to only have *one*
implementation per decode operation. Of course, encoders
would have to provide the same interfaces for symmetry
reasons.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 28 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From walter at livinglogic.de  Wed Jul 28 11:19:09 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Jul 28 11:19:12 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4106CC80.4070401@egenix.com>
References: <4106BD91.4060009@livinglogic.de> <4106C24F.3070203@egenix.com>
	<4106C5D4.1020200@v.loewis.de> <4106CC80.4070401@egenix.com>
Message-ID: <41076F8D.3030507@livinglogic.de>

M.-A. Lemburg wrote:

> Martin v. L?wis wrote:
> 
>> M.-A. Lemburg wrote:
>>
>>> I like the idea, but don't think the implementation is
>>> the right way to do it. Instead, I'd suggest using a new
>>> error handling strategy "break" ( = break processing as
>>> soon as errors are found).
>>
>> Can you demonstrate this approach in a patch? I think it
>> is unimplementable: the codec cannot communicate to the
>> error callback that it ran out of data.
> 
> The codec can: the callback gets all the necessary information
> and can even manipulate the objects being worked on.
> 
> But you have a point: the current implementations of the
> various encode/decode functions don't provide interfaces to
> report back the number of bytes read at C level - the codecs
> module wrappers add these numbers assuming that all bytes
> were read.

This is the correct thing to do for the stateless decoders:
any incomplete byte sequence at the end of the input is an
error. But then it doesn't make sense to return the number
of bytes decoded for the stateless decoder, because this is
always the size of the input. For the stateful decoder this
is just some sort of state common to all decoders: the decoder
keeps the incomplete byte sequence to be used in the next call.
But then this state should be internal to the decoder and not
part of the public API. This would make the decode() method
more usable: When you want to implement an XML parser that
supports the xml.sax.xmlreader.IncrementalParser interface,
you have an API mismatch. The parser has to use the stateful
decoding API (i.e. read()), which means the input is in the
form of a byte stream, but this interface expects it's input
as byte chunks passed to multiple calls to the feed() method.
If StreamReader.decode() simply returned the decoded unicode
object and keep the remaining undecoded bytes as an internal
state then StreamReader.decode() would be directly usable.

But this isn't really a "StreamReader" any more, so the best
solution would probably be to have a three level API:
* A stateless decoding function (what codecs.getdecoder
   returns now);
* A stateful "feed reader", which keeps internal state
   (including undecoded byte sequences) and gets passed byte
   chunks (should this feed reader have a error attribute or
   should this be an argument to the feed method?);
* A stateful stream reader that reads its input from a
   byte stream. The functionality for the stream reader could
   be implemented once using the underlying functionality of
   the feed reader (in fact we could implement something similar
   to sio's stacking streams: the stream reader would use
   the feed reader to wrap the byte input stream and add
   only a read() method. The line reading methods (readline(),
   readlines() and __iter__() could be added by another stream
   filter)

> The error callbacks could, however, raise an exception which
> includes all the needed information, including any state that
> may be needed in order to continue with coding operation.

This makes error callbacks effectively unusable with stateful
decoders.

> We may then need to allow additional keyword arguments on the
> encode/decode functions in order to preset a start state.

As those decoding functions are private to the decoder that's
probably OK. I wouldn't want to see additional keyword arguments
on str.decode (which uses the stateless API anyway). BTW, that's
exactly what I did for codecs.utf_7_decode_stateful, but I'm not
really comfortable with the internal state of the UTF-7 decoder
being exposed on the Python level. It would be better to encapsulate
the state in a feed reader implemented in C, so that the state is
inaccessible from the Python level.

Bye,
    Walter D?rwald


From walter at livinglogic.de  Wed Jul 28 11:24:04 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Jul 28 11:24:07 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <AF2FDF21-E03C-11D8-838F-000A95686CD8@redivi.com>
References: <4106BD91.4060009@livinglogic.de>	<20040727181432.576B.JCARLSON@uci.edu>
	<AF2FDF21-E03C-11D8-838F-000A95686CD8@redivi.com>
Message-ID: <410770B4.5090000@livinglogic.de>

Bob Ippolito wrote:

>> [...]
>> Having the option of readline() and readlines() being ambiguous anywhere
>> sounds like a misfeature.  Furthermore, since all other readline and
>> readlines calls that do not inherit from StreamReader use the
>> unambiguous "always include the line ending", changing StreamReader to
>> be different is obviously the wrong thing to do.

It isn't: the default is still keepends=True. (In fact changing it
breaks the PEP 263 functionality).

> While this reasoning makes sense for readline(), it most definitely does 
> not for readlines() or __iter__().  unicode objects have a splitlines()

That's exactly where I got the idea from. And it frees the user from
having to deal with the various possible line endings (\r, \n, \r\n,
\u2028). But note that the patch doesn't implement this yet, it only
breaks lines at \n.

> function with this feature, which is probably what he's using in his 
> implementation (I used it in mine), so it's pretty natural to expose 
> that option to the external interface.

Bye,
    Walter D?rwald


From walter at livinglogic.de  Wed Jul 28 11:38:16 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Jul 28 11:38:21 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4f0b69dc04072720514dd3ab92@mail.gmail.com>
References: <4106BD91.4060009@livinglogic.de>
	<4f0b69dc04072720514dd3ab92@mail.gmail.com>
Message-ID: <41077408.2060000@livinglogic.de>

Hye-Shik Chang wrote:

> On Tue, 27 Jul 2004 22:39:45 +0200, Walter D?rwald
> <walter@livinglogic.de> wrote:
> 
>>Pythons unicode machinery currently has problems when decoding
>>incomplete input.
>>
>>When codecs.StreamReader.read() encounters a decoding error it
>>reads more bytes from the input stream and retries decoding.
>>This is broken for two reasons:
>>1) The error might be due to a malformed byte sequence in the input,
>>    a problem that can't be fixed by reading more bytes.
>>2) There may be no more bytes available at this time. Once more
>>    data is available decoding can't continue because bytes from
>>    the input stream have already been read and thrown away.
>>(sio.DecodingInputFilter has the same problems)
> 
> StreamReaders and -Writers from CJK codecs are not suffering from
> this problems because they have internal buffer for keeping states
> and incomplete bytes of a sequence. In fact, CJK codecs has its
> own implementation for UTF-8 and UTF-16 on base of its multibytecodec
> system.  It provides a "working" StreamReader/Writer already. :)

Seems you had the same problems with the builtin stream readers! ;)

BTW, how do you solve the problem that incomplete byte sequences
are retained in the middle of a stream, but should generate errors
at the end?

>>I've uploaded a patch that fixes these problems to SF:
>>http://www.python.org/sf/998993
>>
>>The patch implements a few additional features:
>>- read() has an additional argument chars that can be used to
>>   specify the number of characters that should be returned.
>>- readline() is supported on all readers derived from
>>   codecs.StreamReader().
> 
> I have no comment for these, yet.
> 
>>- readline() and readlines() have an additional option
>>   for dropping the u"\n".
> 
> +1
> 
> I wonder whether we need to add optional argument for writelines()
> to add newline characters for each lines, then.

This would probably be a nice convenient additional feature,
but of course you could always pass a GE to writelines():
stream.writelines(line+u"\n" for line in lines)

Bye,
    Walter D?rwald


From mal at egenix.com  Wed Jul 28 11:42:52 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Jul 28 11:42:54 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <41076F8D.3030507@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>
	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>
	<4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de>
Message-ID: <4107751C.2020207@egenix.com>

Walter D?rwald wrote:
> This is the correct thing to do for the stateless decoders:
> any incomplete byte sequence at the end of the input is an
> error. But then it doesn't make sense to return the number
> of bytes decoded for the stateless decoder, because this is
> always the size of the input. 

The reason why stateless encode and decode APIs return the
number of input items consumed is to accomodate for error
handling situations like these where you want to stop
coding and leave the remaining work to another step.

The C implementation currently doesn't make use of this
feature.

> For the stateful decoder this
> is just some sort of state common to all decoders: the decoder
> keeps the incomplete byte sequence to be used in the next call.
> But then this state should be internal to the decoder and not
> part of the public API. This would make the decode() method
> more usable: When you want to implement an XML parser that
> supports the xml.sax.xmlreader.IncrementalParser interface,
> you have an API mismatch. The parser has to use the stateful
> decoding API (i.e. read()), which means the input is in the
> form of a byte stream, but this interface expects it's input
> as byte chunks passed to multiple calls to the feed() method.
> If StreamReader.decode() simply returned the decoded unicode
> object and keep the remaining undecoded bytes as an internal
> state then StreamReader.decode() would be directly usable.

Please don't mix "StreamReader" with "decoder". The codecs
module returns 4 different objects if you ask it for
a codec set: two stateless APIs for encoding and decoding
and two factory functions for creating possibly stateful
objects which expose a stream interface.

Your "stateful decoder" is actually part of a StreamReader
implementation and doesn't have anything to do with the
stateless decoder.

I see two possibilities here:

1. you write a custom StreamReader/Writer implementation
    for each of the codecs which takes care of keeping
    state and encoding/decoding as much as possible

2. you extend the existing stateless codec implementations
    to allow communicating state on input and output; the
    stateless operation would then be a special case

> But this isn't really a "StreamReader" any more, so the best
> solution would probably be to have a three level API:
> * A stateless decoding function (what codecs.getdecoder
>   returns now);
> * A stateful "feed reader", which keeps internal state
>   (including undecoded byte sequences) and gets passed byte
>   chunks (should this feed reader have a error attribute or
>   should this be an argument to the feed method?);
> * A stateful stream reader that reads its input from a
>   byte stream. The functionality for the stream reader could
>   be implemented once using the underlying functionality of
>   the feed reader (in fact we could implement something similar
>   to sio's stacking streams: the stream reader would use
>   the feed reader to wrap the byte input stream and add
>   only a read() method. The line reading methods (readline(),
>   readlines() and __iter__() could be added by another stream
>   filter)

Why make things more complicated ?

If you absolutely need a feed interface, you can feed
your data to a StringIO instance which is then read from
by StreamReader.

>> The error callbacks could, however, raise an exception which
>> includes all the needed information, including any state that
>> may be needed in order to continue with coding operation.
> 
> This makes error callbacks effectively unusable with stateful
> decoders.

Could you explain ?

>> We may then need to allow additional keyword arguments on the
>> encode/decode functions in order to preset a start state.
> 
> As those decoding functions are private to the decoder that's
> probably OK. I wouldn't want to see additional keyword arguments
> on str.decode (which uses the stateless API anyway). BTW, that's
> exactly what I did for codecs.utf_7_decode_stateful, but I'm not
> really comfortable with the internal state of the UTF-7 decoder
> being exposed on the Python level. It would be better to encapsulate
> the state in a feed reader implemented in C, so that the state is
> inaccessible from the Python level.

See above: possibility 1 would be the way to go then.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 28 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From walter at livinglogic.de  Wed Jul 28 11:47:06 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Jul 28 11:47:09 2004
Subject: [Python-Dev] Re: [Python-checkins]
 python/dist/src/Modules/cjkcodecs	_codecs_unicode.c, 1.1, NONE
In-Reply-To: <E1Bpkrn-0002ZG-AL@sc8-pr-cvs1.sourceforge.net>
References: <E1Bpkrn-0002ZG-AL@sc8-pr-cvs1.sourceforge.net>
Message-ID: <4107761A.3010005@livinglogic.de>

perky@users.sourceforge.net wrote:
> Update of /cvsroot/python/python/dist/src/Modules/cjkcodecs
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9601
> 
> Removed Files:
> 	_codecs_unicode.c 
> Log Message:
> Remove CJKCodecs implementation of UTF-7 and UTF-8 codec which
> aren't intended to be part of Python distributiuon.  This was
> accidently imported on mass converting from standalone version of
> CJKCodecs.

If the builtin UTF-7 and UTF-8 stateful decoders are broken and
the ones from the CJKCodecs aren't, why don't we move the your
into the core and drop the builtins?

Bye,
    Walter D?rwald

From walter at livinglogic.de  Wed Jul 28 12:07:43 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Jul 28 12:07:45 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <41076F05.7020300@egenix.com>
References: <4106BD91.4060009@livinglogic.de>
	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>
	<41076596.6070106@livinglogic.de> <41076F05.7020300@egenix.com>
Message-ID: <41077AEF.4060500@livinglogic.de>

M.-A. Lemburg wrote:

> Walter D?rwald wrote:
> 
>> Martin v. L?wis wrote:
>>
>>> M.-A. Lemburg wrote:
>>>
>>>> I like the idea, but don't think the implementation is
>>>> the right way to do it. Instead, I'd suggest using a new
>>>> error handling strategy "break" ( = break processing as
>>>> soon as errors are found).
>>>
>>> Can you demonstrate this approach in a patch? I think it
>>> is unimplementable: the codec cannot communicate to the
>>> error callback that it ran out of data.
>>
>> We would need a special attribute in the exception for
>> that, but the problem IMHO is a different one. This makes
>> it impossible to use other error handling schemes than
>> "break" in stateful decoders.
> 
> I don't understand... are you referring to some extra
> attribute for storing arbitrary state ?

The position of the error is not sufficient to determine
whether it is a truncated data error or a real one:
both r"a\xf".decode("unicode-escape") and
r"a\xfx".decode("unicode-escape") raise a UnicodeDecodeException
with exc.end == len(exc.object), i.e. the error is at
the end of the input. But in the first case the error will
go away once more data is available, but in the second case
it won't.

> If so, why would
> adding such an attribute make it impossible to use
> other error handling schemes ?

It doesn't, but it would make it possible for the callback
to distinguish transient errors from real ones.

> The problem with your patch is that you are adding a whole
> new set of decoders to the core which duplicate much of what
> the already existing decoders implement. I don't like that
> duplication and would like to find a way to only have *one*
> implementation per decode operation.

I don't like the duplication either. In fact we might need
decoders that pass state, but do complain about truncated data
at the end of the stream. I think it's possible to find other
solutions. I would prefer stateful decoders implemented in C.

But I hope you agree that this is a problem that should be fixed.

> Of course, encoders
> would have to provide the same interfaces for symmetry
> reasons.

There are no encoders that have to keep state, except for
UTF-16.

Bye,
    Walter D?rwald


From jim at zope.com  Wed Jul 28 12:56:44 2004
From: jim at zope.com (Jim Fulton)
Date: Wed Jul 28 12:56:50 2004
Subject: [Python-Dev] Can we limit the effects of module execution to
 sys.modules? (was Fix import errors to have data)
In-Reply-To: <1f7befae040727085715ce2536@mail.gmail.com>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
Message-ID: <4107866C.7060609@zope.com>

Tim Peters wrote:
> [Jim Fulton]
> ...
> 
...

> Then you're proposing a way for a highly knowledgable user to
> anticipate, and partially worm around, that Python leaves behind
> insane module objects in sys.modules.  Wouldn't it be better to change
> Python to stop leaving insane module objects in sys.modules to begin
> with?  That's harder, but seems to me it would do a lot more good for
> a lot more people.

I sympathize with your frustration with this problem, but I think that
the problem is bigger that just sys.modules.  For better or worse, importing
a module may have side effects that extend beyond sys.modules. For example,
In some applications, objects get registered into registries that exist in
already-imported modules.  Perhaps we want to declare this to be a
poor style.  If a module has an impact beyond new modules added to
sys.modules, then removing all modules imported into sys.modules as
a result of attempting the import would produce bugs even more subtle
than what we have now.

Do you think it's practical to limit the effects of module import to
sys.modules, even by convention?  Could we say that it is a bug for
code executed during module import to mutate other modules, including
mutating objects contained in those other modules?  I would support this
myself. If it is possible to limit the effects of import (even by convention),
then I think it would be practical to roll-back changes to sys.modules.
If it's not practical to limit the effects of  module import then I think
the problem is effectively unsolveable, short of making Python transactional.

Personably I'm inclined to consider errors that occur while executing a
module to be pretty much fatal.  If a module has begun executing, I really
don't know what state it's in or what state it might have left other modules
in.  I'd rather report the error and get some human to fix it.  OTOH, I'm
happy to recover from the inability to find a module as long as not module
code has been executed.

FWIW, In Zope, we generally generally limit non-transactional state
changes to program startup. For that reason, we make little or no attempt to
survive startup errors.

Jim


-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From jim at zope.com  Wed Jul 28 12:59:24 2004
From: jim at zope.com (Jim Fulton)
Date: Wed Jul 28 12:59:26 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200407272050.i6RKoEY25164@guido.python.org>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>	<1f7befae040727085715ce2536@mail.gmail.com>	<4106853A.8080109@zope.com>	<20040727164913.GS7708@epoch.metaslash.com>
	<41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org>
Message-ID: <4107870C.1070404@zope.com>

Guido van Rossum wrote:
>>An even simpler aproach would be to have a ModuleNotFound error
>>that extends ImportError and gets raised if the module can't
>>be found.
> 
> 
> That wouldn't solve anything, because if foo brokenly tries to import
> bar and that raises ModuleNotFound, so will "import foo".

Doh. Right.

A possible way to make this work is to have an exception that is raised
of a module can't be found *and* if no module code has executed. <shrug>

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From hyeshik at gmail.com  Wed Jul 28 13:56:01 2004
From: hyeshik at gmail.com (Hye-Shik Chang)
Date: Wed Jul 28 13:56:04 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Modules/cjkcodecs _codecs_unicode.c, 1.1, NONE
In-Reply-To: <4107761A.3010005@livinglogic.de>
References: <E1Bpkrn-0002ZG-AL@sc8-pr-cvs1.sourceforge.net>
	<4107761A.3010005@livinglogic.de>
Message-ID: <4f0b69dc04072804561fb7c70@mail.gmail.com>

On Wed, 28 Jul 2004 11:47:06 +0200, Walter D?rwald
<walter@livinglogic.de> wrote:
> perky@users.sourceforge.net wrote:
> > Update of /cvsroot/python/python/dist/src/Modules/cjkcodecs
> > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9601
> >
> > Removed Files:
> >       _codecs_unicode.c
> > Log Message:
> > Remove CJKCodecs implementation of UTF-7 and UTF-8 codec which
> > aren't intended to be part of Python distributiuon.  This was
> > accidently imported on mass converting from standalone version of
> > CJKCodecs.
> 
> If the builtin UTF-7 and UTF-8 stateful decoders are broken and
> the ones from the CJKCodecs aren't, why don't we move the your
> into the core and drop the builtins?
> 

It could be a solution.  But builtin UTF codecs are far faster than
CJKCodecs' because of CJKCodecs' layered structure.  Almost all
usecases in the real world for codecs are simple .encode and .decode,
CJKCodecs' slowness would be too critical to consider it as a
replacement of builtin UTF codecs.

Hye-Shik
From hyeshik at gmail.com  Wed Jul 28 14:46:47 2004
From: hyeshik at gmail.com (Hye-Shik Chang)
Date: Wed Jul 28 14:46:50 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <41077408.2060000@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>
	<4f0b69dc04072720514dd3ab92@mail.gmail.com>
	<41077408.2060000@livinglogic.de>
Message-ID: <4f0b69dc040728054638b1d4a0@mail.gmail.com>

On Wed, 28 Jul 2004 11:38:16 +0200, Walter D?rwald
<walter@livinglogic.de> wrote:
> Hye-Shik Chang wrote:
> 
> > On Tue, 27 Jul 2004 22:39:45 +0200, Walter D?rwald
> > <walter@livinglogic.de> wrote:
> >
> >>Pythons unicode machinery currently has problems when decoding
> >>incomplete input.
> >>
> >>When codecs.StreamReader.read() encounters a decoding error it
> >>reads more bytes from the input stream and retries decoding.
> >>This is broken for two reasons:
> >>1) The error might be due to a malformed byte sequence in the input,
> >>    a problem that can't be fixed by reading more bytes.
> >>2) There may be no more bytes available at this time. Once more
> >>    data is available decoding can't continue because bytes from
> >>    the input stream have already been read and thrown away.
> >>(sio.DecodingInputFilter has the same problems)
> >
> > StreamReaders and -Writers from CJK codecs are not suffering from
> > this problems because they have internal buffer for keeping states
> > and incomplete bytes of a sequence. In fact, CJK codecs has its
> > own implementation for UTF-8 and UTF-16 on base of its multibytecodec
> > system.  It provides a "working" StreamReader/Writer already. :)
> 
> Seems you had the same problems with the builtin stream readers! ;)
> 
> BTW, how do you solve the problem that incomplete byte sequences
> are retained in the middle of a stream, but should generate errors
> at the end?
> 

Rough pseudo code here: (it's written in C in CJKCodecs)

class StreamReader:

    pending = '' # incomplete 

    def read(self, size=-1):
        while True:
            r = fp.read(size)
            if self.pending:
                r = self.pending + r
                self.pending = ''

            if r:
                try:
                    outputbuffer = r.decode('utf-8')
                except MBERR_TOOFEW: # incomplete multibyte sequence
                    pass
                except MBERR_ILLSEQ: # illegal sequence
                    raise UnicodeDecodeError, "illseq"

            if not r or size == -1: # end of the stream
                if r have not consumed up for the output:
                    raise UnicodeDecodeError, "toofew"

            if r have not consumed up for the output:
                self.pending = remainders of r

            if (size == -1 or               # one time read up
                len(outputbuffer) > 0 or    # output buffer isn't empty
                original length of r == 0): # the end of the stream
                    break

            size = 1 # read 1 byte in next try

        return outputbuffer


CJKcodecs' multibytecodec structure has distinguished internal error
codes for "illegal sequence" and "incomplete sequence".  And each
internal codecs receive a flag that indicates if immediate flush
is needed at time.  (for the end of streams and simple decode functions)

Regards,
Hye-Shik
From jlenton at gmail.com  Wed Jul 28 15:04:38 2004
From: jlenton at gmail.com (John Lenton)
Date: Wed Jul 28 15:04:41 2004
Subject: [Python-Dev] PEP 262 - active?
In-Reply-To: <200407230826.21976.heikowu@ceosg.de>
References: <200407230826.21976.heikowu@ceosg.de>
Message-ID: <2d9e37f9040728060493c32d1@mail.gmail.com>

On Fri, 23 Jul 2004 08:26:21 +0200, Heiko Wundram <heikowu@ceosg.de> wrote:
> Hey all!
> 
> Is anybody actively working on PEP 262? I'm currently in the process of
> implementing something like it for a package I'm writing, and I'd be glad to
> sponsor the code once it's complete.
> 
> I've tried to check out what's in the Python CVS sandbox, but can't get it at
> the moment because of network connectivity issues, and the tracker entries on
> the bugtracker aren't very helpful either (besides stating that the PEP was
> abandoned, and later reopened).
> 
> If there's interest, I'll write up a short text describing what my package can
> do in addition to what PEP 262 describes, and how it's integrated with
> distutils.

I just read the pep, and one thing that jumps at me is that most
(linux) distributions already have most of the information; at most a
(python) distribution -> package name map is needed; everything else
can be done by querying the ldistribution's package database. Of
course that is ldistribution-specific, but have you given this any
thought?

One of the pains of using CPAN on a system that already does package
management is that it is usually impossible (or at least very
difficult) to keep both in sync; the proposed "package manager
installs files" is good, but maybe it can be improved...?

-- 
John Lenton (jlenton@gmail.com) -- Random fortune:
bash: fortune: command not found
From walter at livinglogic.de  Wed Jul 28 17:13:17 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Jul 28 17:13:20 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4f0b69dc040728054638b1d4a0@mail.gmail.com>
References: <4106BD91.4060009@livinglogic.de>
	<4f0b69dc04072720514dd3ab92@mail.gmail.com>
	<41077408.2060000@livinglogic.de>
	<4f0b69dc040728054638b1d4a0@mail.gmail.com>
Message-ID: <4107C28D.9030602@livinglogic.de>

Hye-Shik Chang wrote:

> [...]
>>BTW, how do you solve the problem that incomplete byte sequences
>>are retained in the middle of a stream, but should generate errors
>>at the end?
> 
> Rough pseudo code here: (it's written in C in CJKCodecs)
> 
> class StreamReader:
> 
>     pending = '' # incomplete 
> 
>     def read(self, size=-1):
>         while True:
>             r = fp.read(size)
>             if self.pending:
>                 r = self.pending + r
>                 self.pending = ''
> 
>             if r:
>                 try:
>                     outputbuffer = r.decode('utf-8')
>                 except MBERR_TOOFEW: # incomplete multibyte sequence
>                     pass
>                 except MBERR_ILLSEQ: # illegal sequence
>                     raise UnicodeDecodeError, "illseq"
> 
>             if not r or size == -1: # end of the stream
>                 if r have not consumed up for the output:
>                     raise UnicodeDecodeError, "toofew"

Here's the problem: I'd like the streamreader to be able
to continue even when there is no input available *now*.
Perhaps there should be an additional argument to read()
named final? If final is true, the stream reader makes
sure that all pending bytes have been used up.

>             if r have not consumed up for the output:
>                 self.pending = remainders of r
> 
>             if (size == -1 or               # one time read up
>                 len(outputbuffer) > 0 or    # output buffer isn't empty
>                 original length of r == 0): # the end of the stream
>                     break
> 
>             size = 1 # read 1 byte in next try
> 
>         return outputbuffer
> 
> 
> CJKcodecs' multibytecodec structure has distinguished internal error
> codes for "illegal sequence" and "incomplete sequence".  And each
> internal codecs receive a flag that indicates if immediate flush
> is needed at time.  (for the end of streams and simple decode functions)

Bye,
    Walter D?rwald


From barry at python.org  Wed Jul 28 17:52:35 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Jul 28 17:52:15 2004
Subject: [Python-Dev] Can we limit the effects of module execution to
	sys.modules? (was Fix import errors to have data)
In-Reply-To: <4107866C.7060609@zope.com>
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com> <4107866C.7060609@zope.com>
Message-ID: <1091029954.8666.64.camel@localhost>

On Wed, 2004-07-28 at 06:56, Jim Fulton wrote:

> Do you think it's practical to limit the effects of module import to
> sys.modules, even by convention?  Could we say that it is a bug for
> code executed during module import to mutate other modules, including
> mutating objects contained in those other modules?  

It's fairly common practice to execute code at module scope, as a
side-effect of importation.  I've always been uncomfortable with that
idiom for many of the reasons you point out, but it is a darn convenient
place to do such things.

I wonder if we can't define a protocol for executing code at module
scope, but only if the module is successfully imported.  The idea is
that we'd define something like an __init__() for modules that would
only get executed after the module was imported, and if there were any
failures in importing, it wouldn't get called.

We'd have to handle chained imports though.  I haven't thought through
all the implications of that but I can imagine it will get messy.  You
probably don't want to execute any __init__()'s until all imports are
finished, you probably want to execute them in the order you saw them,
and you don't want any exceptions raised in __init__()'s to be deemed an
ImportError worthy of sys.modules deletion.

Maybe this isn't feasible, but I throw it out there as a thought.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040728/886a3a5a/attachment-0001.pgp
From guido at python.org  Wed Jul 28 18:53:14 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Jul 28 18:53:21 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Wed, 28 Jul 2004 06:59:24 EDT."
	<4107870C.1070404@zope.com> 
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
	<41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org> 
	<4107870C.1070404@zope.com> 
Message-ID: <200407281653.i6SGrEe05247@guido.python.org>

[Jim]
> A possible way to make this work is to have an exception that is raised
> of a module can't be found *and* if no module code has executed. <shrug>

Huh?  That would only make a difference if the *first* import in a
module failed.

> From: Barry Warsaw <barry@python.org>
> On Wed, 2004-07-28 at 06:56, Jim Fulton wrote:
> 
> > Do you think it's practical to limit the effects of module import to
> > sys.modules, even by convention?  Could we say that it is a bug for
> > code executed during module import to mutate other modules, including
> > mutating objects contained in those other modules?

I'm uncomfortable with that; while it's indeed conventional for
modules to mostly define classes and functions (and constants and
global variables and ...), there are many other useful things one
could do at module level, and I hesitate to declare those in any way,
shape or sense frowned upon.  You can have stricter rules within a
project, of course, but I don't want to make this part of the general
Python culture; your concerns are mostly relevant only for very large
projects.

[Barry again]
> It's fairly common practice to execute code at module scope, as a
> side-effect of importation.  I've always been uncomfortable with that
> idiom for many of the reasons you point out, but it is a darn convenient
> place to do such things.

Um, importing a module is executing code.  Defining a class or method
is executing code.  Defining a module-global (or class) constant is
executing code.  Initializing a module-global variable.  Etc.  Where
do you draw the line?

> I wonder if we can't define a protocol for executing code at module
> scope, but only if the module is successfully imported.  The idea is
> that we'd define something like an __init__() for modules that would
> only get executed after the module was imported, and if there were any
> failures in importing, it wouldn't get called.

I don't think this can reasonably done.  It would have to contain
everything that's currently at module scope, which would defeat the
purpose.

> We'd have to handle chained imports though.  I haven't thought through
> all the implications of that but I can imagine it will get messy.  You
> probably don't want to execute any __init__()'s until all imports are
> finished, you probably want to execute them in the order you saw them,
> and you don't want any exceptions raised in __init__()'s to be deemed an
> ImportError worthy of sys.modules deletion.
> 
> Maybe this isn't feasible, but I throw it out there as a thought.

Let me throw it out again.  You'll agree after you've had your
coffee. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From walter at livinglogic.de  Wed Jul 28 18:55:10 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Jul 28 18:55:14 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4107751C.2020207@egenix.com>
References: <4106BD91.4060009@livinglogic.de>
	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>
	<4106CC80.4070401@egenix.com> <41076F8D.3030507@livinglogic.de>
	<4107751C.2020207@egenix.com>
Message-ID: <4107DA6E.30500@livinglogic.de>

M.-A. Lemburg wrote:

> Walter D?rwald wrote:
> 
>> This is the correct thing to do for the stateless decoders:
>> any incomplete byte sequence at the end of the input is an
>> error. But then it doesn't make sense to return the number
>> of bytes decoded for the stateless decoder, because this is
>> always the size of the input. 
> 
> The reason why stateless encode and decode APIs return the
> number of input items consumed is to accomodate for error
> handling situations like these where you want to stop
> coding and leave the remaining work to another step.

Which in most cases is the read method.

> The C implementation currently doesn't make use of this
> feature.
> 
>> For the stateful decoder this
>> is just some sort of state common to all decoders: the decoder
>> keeps the incomplete byte sequence to be used in the next call.
>> But then this state should be internal to the decoder and not
>> part of the public API. This would make the decode() method
>> more usable: When you want to implement an XML parser that
>> supports the xml.sax.xmlreader.IncrementalParser interface,
>> you have an API mismatch. The parser has to use the stateful
>> decoding API (i.e. read()), which means the input is in the
>> form of a byte stream, but this interface expects it's input
>> as byte chunks passed to multiple calls to the feed() method.
>> If StreamReader.decode() simply returned the decoded unicode
>> object and keep the remaining undecoded bytes as an internal
>> state then StreamReader.decode() would be directly usable.
> 
> 
> Please don't mix "StreamReader" with "decoder". The codecs
> module returns 4 different objects if you ask it for
> a codec set: two stateless APIs for encoding and decoding
> and two factory functions for creating possibly stateful
> objects which expose a stream interface.
> 
> Your "stateful decoder" is actually part of a StreamReader
> implementation and doesn't have anything to do with the
> stateless decoder.

I know. I'd just like to have a stateful decoder that
doesn't use a stream interface. The stream interface
could be built on top of that without any knowlegde
of the encoding.

I wonder whether the decode method is part of the public
API for StreamReader.

> I see two possibilities here:
> 
> 1. you write a custom StreamReader/Writer implementation
>    for each of the codecs which takes care of keeping
>    state and encoding/decoding as much as possible

But I'd like to reuse at least some of the functionality
from PyUnicode_DecodeUTF8() etc.

Would a version of PyUnicode_DecodeUTF8() with an additional
PyUTF_DecoderState * be OK?

> 2. you extend the existing stateless codec implementations
>    to allow communicating state on input and output; the
>    stateless operation would then be a special case
> 
>> But this isn't really a "StreamReader" any more, so the best
>> solution would probably be to have a three level API:
>> * A stateless decoding function (what codecs.getdecoder
>>   returns now);
>> * A stateful "feed reader", which keeps internal state
>>   (including undecoded byte sequences) and gets passed byte
>>   chunks (should this feed reader have a error attribute or
>>   should this be an argument to the feed method?);
>> * A stateful stream reader that reads its input from a
>>   byte stream. The functionality for the stream reader could
>>   be implemented once using the underlying functionality of
>>   the feed reader (in fact we could implement something similar
>>   to sio's stacking streams: the stream reader would use
>>   the feed reader to wrap the byte input stream and add
>>   only a read() method. The line reading methods (readline(),
>>   readlines() and __iter__() could be added by another stream
>>   filter)
> 
> Why make things more complicated ?
> 
> If you absolutely need a feed interface, you can feed
> your data to a StringIO instance which is then read from
> by StreamReader.

This doesn't work, because a StringIO has only one file position:
 >>> import cStringIO
 >>> s = cStringIO.StringIO()
 >>> s.write("x")
 >>> s.read()
''

But something like the Queue class from the tests in the patch
might work.

>>> The error callbacks could, however, raise an exception which
>>> includes all the needed information, including any state that
>>> may be needed in order to continue with coding operation.
>>
>> This makes error callbacks effectively unusable with stateful
>> decoders.
> 
> Could you explain ?

If you have to call the decode function with errors='break',
you will only get the break error handling and nothing else.

>>> We may then need to allow additional keyword arguments on the
>>> encode/decode functions in order to preset a start state.
>>
>> As those decoding functions are private to the decoder that's
>> probably OK. I wouldn't want to see additional keyword arguments
>> on str.decode (which uses the stateless API anyway). BTW, that's
>> exactly what I did for codecs.utf_7_decode_stateful, but I'm not
>> really comfortable with the internal state of the UTF-7 decoder
>> being exposed on the Python level. It would be better to encapsulate
>> the state in a feed reader implemented in C, so that the state is
>> inaccessible from the Python level.
> 
> See above: possibility 1 would be the way to go then.

I might give this a try.

Bye,
    Walter D?rwald


From barry at python.org  Wed Jul 28 19:26:41 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Jul 28 19:26:19 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200407281653.i6SGrEe05247@guido.python.org>
References: <41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com> <4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com> <41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org>
	<4107870C.1070404@zope.com>
	<200407281653.i6SGrEe05247@guido.python.org>
Message-ID: <1091035582.8666.87.camel@localhost>

On Wed, 2004-07-28 at 12:53, Guido van Rossum wrote:

> Um, importing a module is executing code.  

Oh sure!  (I didn't mean it in that sense ;).

> Defining a class or method
> is executing code.  Defining a module-global (or class) constant is
> executing code.  Initializing a module-global variable.  Etc.  Where
> do you draw the line?

Actually, I wouldn't draw the line implicitly, but instead explicitly,
but...

> I don't think this can reasonably done.  It would have to contain
> everything that's currently at module scope, which would defeat the
> purpose.

yah, you're probably right.  Bleh.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040728/400970c8/attachment.pgp
From theller at python.net  Wed Jul 28 19:32:01 2004
From: theller at python.net (Thomas Heller)
Date: Wed Jul 28 19:34:04 2004
Subject: [Python-Dev] Static builds on Windows
Message-ID: <oem0597y.fsf@python.net>

I'm trying to build a static Python library on Windows.

Obviously this can be done by compiling the sources defining the
Py_NO_ENABLE_SHARED preprocessor symbol.

The resulting executable crashes when trying to load external extension
modules, because it tries to load another Python instance (from
python23.dll).

Much better, of course, would be if the executable wouild refuse to load
extension modules, which can be forced by commenting out the definition
of HAVE_DYNAMIC_LOADING in PC/pyconfig.h.

Now, shouldn't the definition of this symbol be protected by #ifndef
Py_NO_ENABLE_SHARED, or is there already a different mechanism (without
the need to change the source files)?

Thomas

Index: pyconfig.h
===================================================================
RCS file: /cvsroot/python/python/dist/src/PC/pyconfig.h,v
retrieving revision 1.26
diff -c -r1.26 pyconfig.h
*** pyconfig.h	10 Feb 2004 16:50:19 -0000	1.26
--- pyconfig.h	28 Jul 2004 17:31:20 -0000
***************
*** 399,406 ****
--- 399,408 ----
  /* Define if you have clock.  */
  /* #define HAVE_CLOCK */
  
+ #ifndef Py_NO_ENABLE_SHARED
  /* Define when any dynamic module loading is enabled */
  #define HAVE_DYNAMIC_LOADING
+ #endif
  
  /* Define if you have ftime.  */
  #define HAVE_FTIME

From pje at telecommunity.com  Wed Jul 28 19:40:03 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Jul 28 19:36:12 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200407281653.i6SGrEe05247@guido.python.org>
References: <Your message of "Wed,
	28 Jul 2004 06:59:24 EDT." <4107870C.1070404@zope.com>
	<41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
	<41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org>
	<4107870C.1070404@zope.com>
Message-ID: <5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>

At 09:53 AM 7/28/04 -0700, Guido van Rossum wrote:
> > On Wed, 2004-07-28 at 06:56, Jim Fulton wrote:
> > > Do you think it's practical to limit the effects of module import to
> > > sys.modules, even by convention?  Could we say that it is a bug for
> > > code executed during module import to mutate other modules, including
> > > mutating objects contained in those other modules?
>
>I'm uncomfortable with that; while it's indeed conventional for
>modules to mostly define classes and functions (and constants and
>global variables and ...), there are many other useful things one
>could do at module level, and I hesitate to declare those in any way,
>shape or sense frowned upon.  You can have stricter rules within a
>project, of course, but I don't want to make this part of the general
>Python culture; your concerns are mostly relevant only for very large
>projects.

And for those very large projects, I'm wondering if what is *really* wanted 
is actually to be able to isolate plug-ins or extensions into their own 
"module space", similar to the way the Java "classloader" hierarchy 
works.  In effect, Java can have multiple 'sys' modules, arranged in a 
hierarchy.  Each classloader has its own sys.path equivalent, and can also 
delegate imports to its parent classloader(s), in order to have some 
modules shared between child classloaders.  Classloaders are typically used 
in application servers or other systems that wrap around multiple 
third-party application modules, so that each application unit appears to 
have the entire VM to itself, even though side-by-side there may be other 
applications seeing the same or different versions of various 
modules.  This effect is achieved by creating a classloader for each 
segregated application unit, and then using it to load that application's 
main class.

If a mechanism like this were available for Python, then large systems like 
Zope could simply (well, maybe *simply* is the wrong word!) load extensions 
in a separate "module space".  If there's a problem, just throw the whole 
thing away, with no side effects in the current module space such as 
"insane" half-empty modules.

As a side benefit, individual extensions can depend on/use different 
versions of common packages, as long as they get loaded into separate 
module spaces.  Heck, you can give extensions their own private copy of the 
'logging' module, or anything else that has module-level configuration and 
doesn't "play nice" when multiple extensions want it configured differently.

I don't know for certain that such a thing is possible using just the 
standard import hooks, but it seems to me it might be.  If anybody is 
interested in such an animal, have a look at:

    http://www.eby-sarna.com/pipermail/peak/2004-July/001652.html

and drop me a line.  I presume this is offtopic for further discussion on 
python-dev unless it's proposed as a stdlib addition, and I don't think it 
should be so proposed without first having some field experience in at 
least a couple of those "very large projects".  My personal interest in 
this is mainly aimed at integrating Python with Eclipse, but it seems to me 
that systems like Zope and Chandler that want to support large numbers of 
third-party plugins, probably want to be able to do things like this.  It 
would be nice to have a common implementation, or failing that, at least 
understand what prevents a common implementation from being practical.

From mal at egenix.com  Wed Jul 28 20:31:04 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Jul 28 20:31:08 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4107DA6E.30500@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>
	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>
	<4107DA6E.30500@livinglogic.de>
Message-ID: <4107F0E8.3010005@egenix.com>

Walter D?rwald wrote:
> M.-A. Lemburg wrote:
> 
>> Walter D?rwald wrote:
>>
>>> This is the correct thing to do for the stateless decoders:
>>> any incomplete byte sequence at the end of the input is an
>>> error. But then it doesn't make sense to return the number
>>> of bytes decoded for the stateless decoder, because this is
>>> always the size of the input. 
>>
>>
>> The reason why stateless encode and decode APIs return the
>> number of input items consumed is to accomodate for error
>> handling situations like these where you want to stop
>> coding and leave the remaining work to another step.
> 
> Which in most cases is the read method.

The read method only happens to use the stateless
encode and decode methods. There nothing in the design
spec that mandates this, though.

>> The C implementation currently doesn't make use of this
>> feature.
>>
>>> For the stateful decoder this
>>> is just some sort of state common to all decoders: the decoder
>>> keeps the incomplete byte sequence to be used in the next call.
>>> But then this state should be internal to the decoder and not
>>> part of the public API. This would make the decode() method
>>> more usable: When you want to implement an XML parser that
>>> supports the xml.sax.xmlreader.IncrementalParser interface,
>>> you have an API mismatch. The parser has to use the stateful
>>> decoding API (i.e. read()), which means the input is in the
>>> form of a byte stream, but this interface expects it's input
>>> as byte chunks passed to multiple calls to the feed() method.
>>> If StreamReader.decode() simply returned the decoded unicode
>>> object and keep the remaining undecoded bytes as an internal
>>> state then StreamReader.decode() would be directly usable.
>>
>>
>>
>> Please don't mix "StreamReader" with "decoder". The codecs
>> module returns 4 different objects if you ask it for
>> a codec set: two stateless APIs for encoding and decoding
>> and two factory functions for creating possibly stateful
>> objects which expose a stream interface.
>>
>> Your "stateful decoder" is actually part of a StreamReader
>> implementation and doesn't have anything to do with the
>> stateless decoder.
> 
> I know. I'd just like to have a stateful decoder that
> doesn't use a stream interface. The stream interface
> could be built on top of that without any knowlegde
> of the encoding.
> 
> I wonder whether the decode method is part of the public
> API for StreamReader.

It is: StreamReader/Writer are "sub-classes" of the Codec
class.

However, there's nothing stating that .read() or .write()
*must* use these methods to do their work and that's
intentional.

>> I see two possibilities here:
>>
>> 1. you write a custom StreamReader/Writer implementation
>>    for each of the codecs which takes care of keeping
>>    state and encoding/decoding as much as possible
> 
> 
> But I'd like to reuse at least some of the functionality
> from PyUnicode_DecodeUTF8() etc.
> 
> Would a version of PyUnicode_DecodeUTF8() with an additional
> PyUTF_DecoderState * be OK?

Before you start putting more work into this, let's first
find a good workable approach.

>> 2. you extend the existing stateless codec implementations
>>    to allow communicating state on input and output; the
>>    stateless operation would then be a special case
>>
>>> But this isn't really a "StreamReader" any more, so the best
>>> solution would probably be to have a three level API:
>>> * A stateless decoding function (what codecs.getdecoder
>>>   returns now);
>>> * A stateful "feed reader", which keeps internal state
>>>   (including undecoded byte sequences) and gets passed byte
>>>   chunks (should this feed reader have a error attribute or
>>>   should this be an argument to the feed method?);
>>> * A stateful stream reader that reads its input from a
>>>   byte stream. The functionality for the stream reader could
>>>   be implemented once using the underlying functionality of
>>>   the feed reader (in fact we could implement something similar
>>>   to sio's stacking streams: the stream reader would use
>>>   the feed reader to wrap the byte input stream and add
>>>   only a read() method. The line reading methods (readline(),
>>>   readlines() and __iter__() could be added by another stream
>>>   filter)
>>
>>
>> Why make things more complicated ?
>>
>> If you absolutely need a feed interface, you can feed
>> your data to a StringIO instance which is then read from
>> by StreamReader.
> 
> 
> This doesn't work, because a StringIO has only one file position:
>  >>> import cStringIO
>  >>> s = cStringIO.StringIO()
>  >>> s.write("x")
>  >>> s.read()
> ''

Ah, you wanted to do both feeding and reading at the same
time ?!

> But something like the Queue class from the tests in the patch
> might work.

Right... I don't think that we need a third approach to
codecs just to implement feed based parsers.

>>>> The error callbacks could, however, raise an exception which
>>>> includes all the needed information, including any state that
>>>> may be needed in order to continue with coding operation.
>>>
>>>
>>> This makes error callbacks effectively unusable with stateful
>>> decoders.
>>
>>
>> Could you explain ?
> 
> 
> If you have to call the decode function with errors='break',
> you will only get the break error handling and nothing else.

Yes and ... ? What else do you want it to do ?

>>>> We may then need to allow additional keyword arguments on the
>>>> encode/decode functions in order to preset a start state.
>>>
>>>
>>> As those decoding functions are private to the decoder that's
>>> probably OK. I wouldn't want to see additional keyword arguments
>>> on str.decode (which uses the stateless API anyway). BTW, that's
>>> exactly what I did for codecs.utf_7_decode_stateful, but I'm not
>>> really comfortable with the internal state of the UTF-7 decoder
>>> being exposed on the Python level. It would be better to encapsulate
>>> the state in a feed reader implemented in C, so that the state is
>>> inaccessible from the Python level.
>>
>>
>> See above: possibility 1 would be the way to go then.
> 
> I might give this a try.

Again, please wait until we have found a good solution
to this.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 28 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From guido at python.org  Wed Jul 28 20:41:50 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Jul 28 20:41:57 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Tue, 27 Jul 2004 22:40:45 EDT."
	<1f7befae04072719401df511c3@mail.gmail.com> 
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com>
	<1f7befae04072719401df511c3@mail.gmail.com> 
Message-ID: <200407281841.i6SIfo205648@guido.python.org>

[Tim]
> When an import of an
> existing module fails for *any* reason, subsequent attempts to import
> the same module succeed.  For example,
> 
> C:\Code>type a.py
> 1/0
> 
> C:\Code>\python23\python.exe
> Python 2.3.4 (#53, May 25 2004, 21:17:02) [MSC v.1200 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import a
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "a.py", line 1, in ?
>     1/0
> ZeroDivisionError: integer division or modulo by zero
> >>> import a
> >>>
> 
> This is Bad, because an uncaught exception in module initialization
> means the module probably can't fulfill its contract, yet subsequent
> importers get no clue that the module is in a damaged state (until the
> module fails to do its job, which may or may not be obvious when it
> occurs).  A module failing to import because it suffers an ImportError
> itself is once instance of this, but the same applies to a module
> failing to import for any other reason:  in all these cases,
> subsequent imports don't get the exception the initial importer saw,
> they get a module object in an arbitrarily screwed-up state.

So let's try to devise new semantics to cover this and other cases.
One requirement is that mutual imports must work.  Another is that
when "import a" fails once, it must fail again if it is retried after
catching the first failure.

Perhaps it is as simple as deleting the module from sys.modules when
the code in import.c executing the module's body gets any kind of
exception from the execution?  This would seem to be a relatively
small change to PyImport_ExecCodeModuleEx(): unify all the error exits
and there delete the module from sys.modules.  What should it do if
the module already existed (e.g. when used by reload())?  Strawman
answer: leave it there -- the reload() semantics and common use cases
are best served by that.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From amk at amk.ca  Wed Jul 28 21:00:25 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Wed Jul 28 21:02:02 2004
Subject: [Python-Dev] PEP 262 - active?
In-Reply-To: <2d9e37f9040728060493c32d1@mail.gmail.com>
References: <200407230826.21976.heikowu@ceosg.de>
	<2d9e37f9040728060493c32d1@mail.gmail.com>
Message-ID: <20040728190025.GA13981@rogue.amk.ca>

On Wed, Jul 28, 2004 at 10:04:38AM -0300, John Lenton wrote:
> difficult) to keep both in sync; the proposed "package manager
> installs files" is good, but maybe it can be improved...?

The "install-a-file" scheme was written that way at the request of
some Red Hat people who were at PyCon, precisely so that RPMs could
also keep the package database up to date.

--amk

From guido at python.org  Wed Jul 28 21:03:40 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Jul 28 21:03:48 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Wed, 28 Jul 2004 11:41:50 PDT."
	<200407281841.i6SIfo205648@guido.python.org> 
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com>
	<1f7befae04072719401df511c3@mail.gmail.com> 
	<200407281841.i6SIfo205648@guido.python.org> 
Message-ID: <200407281903.i6SJ3ea06175@guido.python.org>

I quickly whipped up this (procastinating on my OSCON keynote slides :-):

Index: import.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/import.c,v
retrieving revision 2.233
diff -c -u -r2.233 import.c
--- import.c	27 Jun 2004 16:51:46 -0000	2.233
+++ import.c	28 Jul 2004 18:57:15 -0000
@@ -572,17 +572,23 @@
 {
 	PyObject *modules = PyImport_GetModuleDict();
 	PyObject *m, *d, *v;
+	int pre_existing = 0;
 
-	m = PyImport_AddModule(name);
-	if (m == NULL)
-		return NULL;
+	if ((m = PyDict_GetItemString(modules, name)) != NULL &&
+	    PyModule_Check(m))
+		pre_existing = 1;
+	else {
+		m = PyImport_AddModule(name);
+		if (m == NULL)
+			return NULL;
+	}
 	/* If the module is being reloaded, we get the old module back
 	   and re-use its dict to exec the new code. */
 	d = PyModule_GetDict(m);
 	if (PyDict_GetItemString(d, "__builtins__") == NULL) {
 		if (PyDict_SetItemString(d, "__builtins__",
 					 PyEval_GetBuiltins()) != 0)
-			return NULL;
+			goto error;
 	}
 	/* Remember the filename as the __file__ attribute */
 	v = NULL;
@@ -601,7 +607,7 @@
 
 	v = PyEval_EvalCode((PyCodeObject *)co, d, d);
 	if (v == NULL)
-		return NULL;
+		goto error;
 	Py_DECREF(v);
 
 	if ((m = PyDict_GetItemString(modules, name)) == NULL) {
@@ -614,6 +620,11 @@
 	Py_INCREF(m);
 
 	return m;
+
+  error:
+	if (!pre_existing)
+		PyDict_DelItemString(modules, name); /* If it fails, too bad */
+	return NULL;
 }
 
 
Superficial testing suggests it works fine, except for one problem in
test_pkgimport which is testing specifically for the broken behavior.

BTW, building CVS Python gives these warnings for me:

/home/guido/projects/python/dist/src/Modules/cjkcodecs/_codecs_iso2022.c:120: array size missing in `designations'
/home/guido/projects/python/dist/src/Modules/cjkcodecs/_codecs_iso2022.c: In function `iso2022processesc':
/home/guido/projects/python/dist/src/Modules/cjkcodecs/_codecs_iso2022.c:306: warning: `esclen' might be used uninitialized in this function
/home/guido/projects/python/dist/src/Modules/cjkcodecs/_codecs_iso2022.c: At top level:
/home/guido/projects/python/dist/src/Modules/cjkcodecs/_codecs_iso2022.c:1053: warning: excess elements in array initializer
/home/guido/projects/python/dist/src/Modules/cjkcodecs/_codecs_iso2022.c:1053: warning: (near initialization for `iso2022_kr_config.designations')
/home/guido/projects/python/dist/src/Modules/cjkcodecs/_codecs_iso2022.c:1058: warning: excess elements in array initializer
[and many more similar]

Red Hat 7.3 using gcc 2.96.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From martin at v.loewis.de  Wed Jul 28 21:04:19 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Jul 28 21:04:18 2004
Subject: [Python-Dev] Static builds on Windows
In-Reply-To: <oem0597y.fsf@python.net>
References: <oem0597y.fsf@python.net>
Message-ID: <4107F8B3.6080206@v.loewis.de>

Thomas Heller wrote:
> Now, shouldn't the definition of this symbol be protected by #ifndef
> Py_NO_ENABLE_SHARED, or is there already a different mechanism (without
> the need to change the source files)?

Perhaps. However, my understanding is that it is, in principle, possible
to provide dynamic loading even if the interpreter is all in a single
static python.exe. IOW, people sometimes claim that you can link against
an .exe, in which case you could have extension modules build for your
static interpreter, which could then be loaded dynamically.

I don't know whether anybody has tried that recently, or whether it is
worth supporting. If not, I would have no problems claiming that dynamic
loading is unavailable for a static build.

As for dynamic detection of the issue: I thought there was a mechanism
somewhere which tries to find out whether an extension module is linked
against the "wrong" .dll. Perhaps that mechanism could be extended to
support static libraries: Find out what the module name is of the module
containing importdl.obj, and refuse loading if the extension module
is linked with a different PE module.

Regards,
Martin
From theller at python.net  Wed Jul 28 22:00:32 2004
From: theller at python.net (Thomas Heller)
Date: Wed Jul 28 22:04:16 2004
Subject: [Python-Dev] Static builds on Windows
In-Reply-To: <4107F8B3.6080206@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Wed,
	28 Jul 2004 21:04:19 +0200")
References: <oem0597y.fsf@python.net> <4107F8B3.6080206@v.loewis.de>
Message-ID: <8yd352cf.fsf@python.net>

"Martin v. L?wis" <martin@v.loewis.de> writes:

> Thomas Heller wrote:
>> Now, shouldn't the definition of this symbol be protected by #ifndef
>> Py_NO_ENABLE_SHARED, or is there already a different mechanism (without
>> the need to change the source files)?
>
> Perhaps. However, my understanding is that it is, in principle, possible
> to provide dynamic loading even if the interpreter is all in a single
> static python.exe. IOW, people sometimes claim that you can link against
> an .exe, in which case you could have extension modules build for your
> static interpreter, which could then be loaded dynamically.
>
> I don't know whether anybody has tried that recently, or whether it is
> worth supporting.

I have never tried that, although I've heard that it wouldn't work.

> If not, I would have no problems claiming that dynamic
> loading is unavailable for a static build.

Currently, it does *not* work, so I think it would be good to add the
above protection in the header file. It can be removed (or extended), as
soon as someone makes it work.

> As for dynamic detection of the issue: I thought there was a mechanism
> somewhere which tries to find out whether an extension module is linked
> against the "wrong" .dll. Perhaps that mechanism could be extended to
> support static libraries: Find out what the module name is of the module
> containing importdl.obj, and refuse loading if the extension module
> is linked with a different PE module.

Yes there is, in Python/dynload_win.c.  But IIRC it works by hardcoding
the name of the python dll.  And it is wasteful to try this all the time
if it is known in advance that it cannot succeed.
Maybe it would be best to replace this module for the static build by
another one.

Ok, three options:
- make the patch I suggested to the header file.
- patch dynload_win.c in the way you suggested.
- ignore the whole issue, and replace dynload_win.c for static builds
  by another file.
- investigate if back linking from extension modules to the exe file
  would work

---

Related to this issue: Would it be a good idea to add a project to the
MSVC workspace which builds a static library?  This could either be used
directly, or as a staring point for customization.

Thomas

From guido at python.org  Wed Jul 28 22:44:28 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Jul 28 22:44:35 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Wed, 28 Jul 2004 13:40:03 EDT."
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com> 
References: <Your message of "Wed,
	28 Jul 2004 06:59:24 EDT." <4107870C.1070404@zope.com>
	<41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
	<41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org>
	<4107870C.1070404@zope.com> 
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com> 
Message-ID: <200407282044.i6SKiTk06705@guido.python.org>

> And for those very large projects, I'm wondering if what is *really*
> wanted is actually to be able to isolate plug-ins or extensions into
> their own "module space", similar to the way the Java "classloader"
> hierarchy works.  [...]

You can certainly build all that using import hooks.  But perhaps
that's too low a level, and a higher-level infrastructure would be
helpful?  Import hooks have been around for ages and many improvements
have been proposed but few of those have actually found much use (or
even found to be real improvements).

So perhaps some interested parties (count me out for lack of time :-(
) should get together and design something, learning  from Java?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From pje at telecommunity.com  Wed Jul 28 23:14:57 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Jul 28 23:11:06 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200407282044.i6SKiTk06705@guido.python.org>
References: <Your message of "Wed, 28 Jul 2004 13:40:03 EDT."
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<Your message of "Wed,
	28 Jul 2004 06:59:24 EDT." <4107870C.1070404@zope.com>
	<41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
	<41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org>
	<4107870C.1070404@zope.com>
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com>

At 01:44 PM 7/28/04 -0700, Guido van Rossum wrote:
> > And for those very large projects, I'm wondering if what is *really*
> > wanted is actually to be able to isolate plug-ins or extensions into
> > their own "module space", similar to the way the Java "classloader"
> > hierarchy works.  [...]
>
>You can certainly build all that using import hooks.  But perhaps
>that's too low a level, and a higher-level infrastructure would be
>helpful?  Import hooks have been around for ages and many improvements
>have been proposed but few of those have actually found much use (or
>even found to be real improvements).

Sorry, I'm confused.  By "higher-level infrastructure", are you meaning a 
way to co-ordinate import hooks, or something higher-level than the "module 
spaces" concept?

As far as I can tell, a properly implemented module space should allow 
arbitrary import hooks to be used within that module space, so long as the 
hooks obtain their copy of 'sys' using the previous value of 
'__import__'.  For example, if you create a hook using 'import ihooks' 
within a given module space, then that space's copy of 'ihooks' will see 
the space's copy of 'sys', so it will thus do all its magic using the right 
'sys'.

The weak spot is 'imp', and any other C-coded modules that access the 
"real" 'sys' module directly (e.g. '__builtin__').  These modules would 
have to be replaced by "safe" versions.  E.g., a 'safe_imp' in Python that 
explicitly passed in a path to the "real" imp functions, and so 
on.  Finding all of the modules that do this and providing "safe" 
replacements (while making sure that truly global "sys" attributes are 
shared) would probably be the biggest task in creating a Java-like import 
framework.

(Oh...  and another infrastructure-level issue: Python classes know their 
__module__ name, but in a system using module spaces, the module name alone 
is not sufficient to identify the module where the class originated.  In 
Java, a class is uniquely identified by the combination of its fully 
qualified name, and the classloader used to load it.  This is not an issue 
for Python functions (which reference the module globals) or for modules 
themselves, which both indirectly refer to the loader used to load the 
module.  But it could be an issue for classes.)

From martin at v.loewis.de  Wed Jul 28 23:39:02 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Jul 28 23:38:58 2004
Subject: [Python-Dev] Static builds on Windows
In-Reply-To: <8yd352cf.fsf@python.net>
References: <oem0597y.fsf@python.net> <4107F8B3.6080206@v.loewis.de>
	<8yd352cf.fsf@python.net>
Message-ID: <41081CF6.7050609@v.loewis.de>

Thomas Heller wrote:
> Related to this issue: Would it be a good idea to add a project to the
> MSVC workspace which builds a static library?  This could either be used
> directly, or as a staring point for customization.

Depends on whether it is going to be maintained. If it is, it might be a
good idea. If it is once created, and then only updated every two years
when somebody actually needs it, it might be pointless - it would be
just as much work to recreate it from scratch every time.

Regards,
Martin
From ncoghlan at iinet.net.au  Thu Jul 29 00:10:03 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu Jul 29 00:10:43 2004
Subject: [Python-Dev] Static builds on Windows
In-Reply-To: <41081CF6.7050609@v.loewis.de>
References: <oem0597y.fsf@python.net>
	<4107F8B3.6080206@v.loewis.de>	<8yd352cf.fsf@python.net>
	<41081CF6.7050609@v.loewis.de>
Message-ID: <4108243B.4040706@iinet.net.au>

Martin v. L?wis wrote:

> Thomas Heller wrote:
> 
>> Related to this issue: Would it be a good idea to add a project to the
>> MSVC workspace which builds a static library?  This could either be used
>> directly, or as a staring point for customization.
> 
> 
> Depends on whether it is going to be maintained. If it is, it might be a
> good idea. If it is once created, and then only updated every two years
> when somebody actually needs it, it might be pointless - it would be
> just as much work to recreate it from scratch every time.

If there was going to be both the ability to create a static lib and a 
dynamic lib, it would seem to make more sense to have the project for 
the dynamic lib project simply incorporate the static lib, and export 
the relevant symbols.

We do something similar at work (VC6, rather than 7), although we're 
combining a dozen or so static libs to produce the DLL. All the real 
source code is in the libraries, and the DLL project consists mainly of 
a list of include directives, and enough symbol references to get 
everything exported correctly.

Regards,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From guido at python.org  Thu Jul 29 00:13:32 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul 29 00:13:38 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Wed, 28 Jul 2004 17:14:57 EDT."
	<5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com> 
References: <Your message of "Wed, 28 Jul 2004 13:40:03 EDT."
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<Your message of "Wed,
	28 Jul 2004 06:59:24 EDT." <4107870C.1070404@zope.com>
	<41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
	<41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org>
	<4107870C.1070404@zope.com>
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com> 
	<5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com> 
Message-ID: <200407282213.i6SMDWD06878@guido.python.org>

[Guido]
> >You can certainly build all that using import hooks.  But perhaps
> >that's too low a level, and a higher-level infrastructure would be
> >helpful?  Import hooks have been around for ages and many improvements
> >have been proposed but few of those have actually found much use (or
> >even found to be real improvements).

[Phillip]
> Sorry, I'm confused.  By "higher-level infrastructure", are you
> meaning a way to co-ordinate import hooks, or something higher-level
> than the "module spaces" concept?

I meant whatever you want.  I am not familiar with the need for this
functionality, all I know is import hooks and you were clear that
those are too low-level.

> As far as I can tell, a properly implemented module space should
> allow arbitrary import hooks to be used within that module space, so
> long as the hooks obtain their copy of 'sys' using the previous
> value of '__import__'.  For example, if you create a hook using
> 'import ihooks' within a given module space, then that space's copy
> of 'ihooks' will see the space's copy of 'sys', so it will thus do
> all its magic using the right 'sys'.

Today's import hooks aren't written with recursive import hooks in
mind.

> The weak spot is 'imp', and any other C-coded modules that access
> the "real" 'sys' module directly (e.g. '__builtin__').  These
> modules would have to be replaced by "safe" versions.  E.g., a
> 'safe_imp' in Python that explicitly passed in a path to the "real"
> imp functions, and so on.  Finding all of the modules that do this
> and providing "safe" replacements (while making sure that truly
> global "sys" attributes are shared) would probably be the biggest
> task in creating a Java-like import framework.

Sure.  Feel free to build it and propose it as a standard library
addition, if you think it will be useful for others.  I might even
find a use for it -- but right now I'm +0 because I really don't have
a need for this stuff in anything I'm doing.

> (Oh...  and another infrastructure-level issue: Python classes know
> their __module__ name, but in a system using module spaces, the
> module name alone is not sufficient to identify the module where the
> class originated.  In Java, a class is uniquely identified by the
> combination of its fully qualified name, and the classloader used to
> load it.  This is not an issue for Python functions (which reference
> the module globals) or for modules themselves, which both indirectly
> refer to the loader used to load the module.  But it could be an
> issue for classes.)

But who ever uses __module__ for anything else than printing it?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Jul 29 00:20:15 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul 29 00:20:28 2004
Subject: [Python-Dev] test_sort.py failure
Message-ID: <200407282220.i6SMKFZ06964@guido.python.org>

I had test_sort.py fail once on line 143 (in test_bug453523).  Later
it wouldn't fail.  The code in question uses a random generator.
Could I have been unlucky enough that the particular random sequence
it used this time didn't create the intended failure condition?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.one at comcast.net  Thu Jul 29 00:36:38 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu Jul 29 00:36:46 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <200407282220.i6SMKFZ06964@guido.python.org>
Message-ID: <20040728223645.3B1E61E4002@bag.python.org>

[Guido van Rossum]
> I had test_sort.py fail once on line 143 (in test_bug453523).

How did it fail?  By not raising any exception, or by raising an exception
other than ValueError?

> Later it wouldn't fail.  The code in question uses a random generator.
> Could I have been unlucky enough that the particular random sequence it
> used this time didn't create the intended failure condition?

No:  every call to __lt__ mutates the list, the code checking for list
mutation should be impossible to fool, the code checking for list mutation
raises ValueError at the end of list.sort() iff mutation occurred during the
sort, and since the list has more than 1 element __lt__ will be called at
least once.  It's possible that the code checking for list mutation became
less than bulletproof after list object internals were reworked, though --
don't know about that.  While it was bulletproof before (according to both
Armin and me), it was delicate (as evidence, it took both Armin and me to
come up with it <wink>).


From pje at telecommunity.com  Thu Jul 29 00:42:19 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Jul 29 00:38:29 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200407282213.i6SMDWD06878@guido.python.org>
References: <Your message of "Wed, 28 Jul 2004 17:14:57 EDT."
	<5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com>
	<Your message of "Wed, 28 Jul 2004 13:40:03 EDT."
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<Your message of "Wed,
	28 Jul 2004 06:59:24 EDT." <4107870C.1070404@zope.com>
	<41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
	<41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org>
	<4107870C.1070404@zope.com>
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040728183934.029464d0@mail.telecommunity.com>

At 03:13 PM 7/28/04 -0700, Guido van Rossum wrote:

>But who ever uses __module__ for anything else than printing it?

Well, I've done some rather esoteric things with it in the past, but 
nothing that would be needed here.  I was just thinking that applications 
like Zope, Chandler et al would sometimes want to know, "what plugin is 
this class/object from?"  But I suppose that they could just as easily 
build that concept explicitly into their APIs.  So never mind that bit.  :)

From bob at redivi.com  Thu Jul 29 00:40:34 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Jul 29 00:40:40 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200407282213.i6SMDWD06878@guido.python.org>
References: <Your message of "Wed, 28 Jul 2004 13:40:03 EDT."
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<Your message of "Wed,
	28 Jul 2004 06:59:24 EDT." <4107870C.1070404@zope.com>
	<41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
	<41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org>
	<4107870C.1070404@zope.com>
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com>
	<200407282213.i6SMDWD06878@guido.python.org>
Message-ID: <23EA8D50-E0E7-11D8-80F1-000A95686CD8@redivi.com>

On Jul 28, 2004, at 6:13 PM, Guido van Rossum wrote:

>> (Oh...  and another infrastructure-level issue: Python classes know
>> their __module__ name, but in a system using module spaces, the
>> module name alone is not sufficient to identify the module where the
>> class originated.  In Java, a class is uniquely identified by the
>> combination of its fully qualified name, and the classloader used to
>> load it.  This is not an issue for Python functions (which reference
>> the module globals) or for modules themselves, which both indirectly
>> refer to the loader used to load the module.  But it could be an
>> issue for classes.)
>
> But who ever uses __module__ for anything else than printing it?

I've used sys.modules[__module__] to get a reference to the current 
module object.  Mostly for running doctests I guess, but I've probably 
used it in other places.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3589 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040728/d372842c/smime.bin
From nbastin at opnet.com  Thu Jul 29 00:58:02 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Thu Jul 29 00:58:21 2004
Subject: [Python-Dev] test_sort.py failure
In-Reply-To: <200407282220.i6SMKFZ06964@guido.python.org>
References: <200407282220.i6SMKFZ06964@guido.python.org>
Message-ID: <9439D600-E0E9-11D8-897C-000D932927FE@opnet.com>


On Jul 28, 2004, at 6:20 PM, Guido van Rossum wrote:

> I had test_sort.py fail once on line 143 (in test_bug453523).  Later
> it wouldn't fail.  The code in question uses a random generator.
> Could I have been unlucky enough that the particular random sequence
> it used this time didn't create the intended failure condition?

I saw the same problem once a few weeks ago on MacOS X, but I couldn't 
reproduce it.

--
Nick

From mhammond at skippinet.com.au  Thu Jul 29 01:23:44 2004
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Thu Jul 29 01:23:33 2004
Subject: [Python-Dev] MSI Documentation
In-Reply-To: <4102E0D8.5020105@v.loewis.de>
Message-ID: <02a901c474f9$ed6c1a20$0200a8c0@eden>

> I've added a bit of documentation on using the 2.4 MSI package, at
>
> http://www.python.org/2.4/msi.html
>
> If you think something is incorrect, or should be added, please let me
> know (or change it yourself if you can).

That looks nice (even though I misread your email and thought I would be
looking at documentation on *building* the MSI.)  Your document on *using*
the MSI has a much wider audience.

Out of interest, why can't we "advertise"?

Mark.

From guido at python.org  Thu Jul 29 02:00:59 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul 29 02:01:06 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: Your message of "Wed, 28 Jul 2004 18:36:38 EDT."
	<20040728223645.3B1E61E4002@bag.python.org> 
References: <20040728223645.3B1E61E4002@bag.python.org> 
Message-ID: <200407290000.i6T00xE07145@guido.python.org>

> [Guido van Rossum]
> > I had test_sort.py fail once on line 143 (in test_bug453523).
> 
> How did it fail?  By not raising any exception, or by raising an
> exception other than ValueError?

I deleted the Emacs hsell buffer where it happened, but I believe it
complained that ValueError wasn't raised.

> > Later it wouldn't fail.  The code in question uses a random
> > generator.  Could I have been unlucky enough that the particular
> > random sequence it used this time didn't create the intended
> > failure condition?
> 
> No: every call to __lt__ mutates the list, the code checking for
> list mutation should be impossible to fool, the code checking for
> list mutation raises ValueError at the end of list.sort() iff
> mutation occurred during the sort, and since the list has more than
> 1 element __lt__ will be called at least once.  It's possible that
> the code checking for list mutation became less than bulletproof
> after list object internals were reworked, though -- don't know
> about that.  While it was bulletproof before (according to both
> Armin and me), it was delicate (as evidence, it took both Armin and
> me to come up with it <wink>).

Looking at the code, it does roughly this:

	saved_ob_size = self->ob_size;
	saved_ob_item = self->ob_item;
	saved_allocated = self->allocated;
	self->ob_size = 0;
	self->ob_item = empty_ob_item = PyMem_NEW(PyObject *, 0);
	self->allocated = 0;

	...sort the saved_ob_item array...

	if (self->ob_item != empty_ob_item || self->ob_size) {
		/* The user mucked with the list during the sort. */
		...cause an exception...
	}
	...else restore self->ob_* from saved_*...

Mucking with the list causes realloc() calls to the memory block
initially pointed to by empty_ob_item.  This is likely to move the
block.  But it *could* be that there's enough space following the
block that it *doesn't* have to be moved, and if the array also
happens to be empty after the sort (there's a certain probability that
this is so), the test will pass despite the mucking.

I guess this is unavoidable given how it is coded; I guess all I can
ask for is a comment in test_sort.py explaining that there's a small
probability that it gives a false positive.  Rerunning the test
repeatedly should satisfy one's confidence that sort isn't broken.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From arigo at tunes.org  Thu Jul 29 02:20:55 2004
From: arigo at tunes.org (Armin Rigo)
Date: Thu Jul 29 02:25:40 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <200407290000.i6T00xE07145@guido.python.org>
References: <20040728223645.3B1E61E4002@bag.python.org>
	<200407290000.i6T00xE07145@guido.python.org>
Message-ID: <20040729002055.GA31229@vicky.ecs.soton.ac.uk>

Hello,

On Wed, Jul 28, 2004 at 05:00:59PM -0700, Guido van Rossum wrote:
> Mucking with the list causes realloc() calls to the memory block
> initially pointed to by empty_ob_item.  This is likely to move the
> block.  But it *could* be that there's enough space following the
> block that it *doesn't* have to be moved, and if the array also
> happens to be empty after the sort (there's a certain probability that
> this is so), the test will pass despite the mucking.

I think Tim is right: the code used to be correct, but the list internals
reorganization might have broken that.  As far as I remember, the reason it
used to be fine is that lists with zero elements were always deallocated, so
that empty lists never had an allocated ob_items array.  If user code
populated and then emptied again the list during the sort, it couldn't end up
again in the special state list_sort() puts it in, which is ob_size == 0 but
ob_items allocated.  Apparently, this can occur now.

Looks like another review of list_sort() is needed.  (Some time ago, after
staring at it for some time, I wondered and couldn't figure out if there was a
memory leak or not, too)


Armin
From tim.one at comcast.net  Thu Jul 29 03:19:35 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu Jul 29 03:19:43 2004
Subject: [Python-Dev] test_sort.py failure
In-Reply-To: <1091055370.1496.8.camel@localhost>
Message-ID: <20040729011941.8CA891E4002@bag.python.org>

[Mark Russell]
> I had exactly the same thing happen today as well (unrepeatable failure
> in test_bug453523).  Would it defeat the purpose of the test to use a
> fixed starting point for the random numbers so the results are
> repeatable?

Yes, it would defeat the purpose.  Randomization is there deliberately to
minimize dubious assumptions.  And this paid off!  list.sort() is broken now
in a way it wasn't broken in 2.3, and a fixed test case probably would not
have revealed this.


From tim.peters at gmail.com  Thu Jul 29 04:53:55 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Jul 29 04:53:58 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <20040729002055.GA31229@vicky.ecs.soton.ac.uk>
References: <20040728223645.3B1E61E4002@bag.python.org>
	<200407290000.i6T00xE07145@guido.python.org>
	<20040729002055.GA31229@vicky.ecs.soton.ac.uk>
Message-ID: <1f7befae0407281953799472ad@mail.gmail.com>

[Armin Rigo]
> I think Tim is right: the code used to be correct, but the list internals
> reorganization might have broken that.  As far as I remember, the reason it
> used to be fine is that lists with zero elements were always deallocated, so
> that empty lists never had an allocated ob_items array.  If user code
> populated and then emptied again the list during the sort, it couldn't end up
> again in the special state list_sort() puts it in, which is ob_size == 0 but
> ob_items allocated.

Exactly right.

>  Apparently, this can occur now.

Ditto.  List objects grew another member, to record the number of list
slots allocated, which may be larger than the number currently in use.
 And the new (since 2.3) internal list_resize() never sets ob_item to
NULL on a list shrink anymore.  Indeed, even if ob_item is NULL on
entry to list_resize, and the new size requested is 0, ob_item is made
non-NULL (unless malloc(3*sizeof(PyObject*)) returns NULL, in which
case ob_item remains NULL but MemoryError is raised).

So it looks like a new delicate scheme for list_sort would be to set
ob_item to NULL at the start.  As soon as mutation adds an element,
ob_item will become non-NULL (or raise MemoryError), and it looks like
it will remain non-NULL forever after.

Alas, test_sort never fails on my box now, so I can't test this.  I
bet it would fail a lot more often on boxes where it does fail if the
test case used a 3-element list instead of a 50-element list.

> Looks like another review of list_sort() is needed.  (Some time ago, after
> staring at it for some time, I wondered and couldn't figure out if there was a
> memory leak or not, too)

Don't spread FUD <wink>.
From fdrake at acm.org  Thu Jul 29 05:44:23 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Jul 29 05:44:37 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <23EA8D50-E0E7-11D8-80F1-000A95686CD8@redivi.com>
References: <Your message of "Wed, 28 Jul 2004 13:40:03 EDT."
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<200407282213.i6SMDWD06878@guido.python.org>
	<23EA8D50-E0E7-11D8-80F1-000A95686CD8@redivi.com>
Message-ID: <200407282344.23804.fdrake@acm.org>

On Wednesday 28 July 2004 06:40 pm, Bob Ippolito wrote:
 > I've used sys.modules[__module__] to get a reference to the current
 > module object.  Mostly for running doctests I guess, but I've probably
 > used it in other places.

Did you mean __name__?

__module__ is an attribute of classes that provides the __name__ of the 
defining module.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>

From tim.peters at gmail.com  Thu Jul 29 05:55:50 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Jul 29 05:55:52 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <1f7befae0407281953799472ad@mail.gmail.com>
References: <20040728223645.3B1E61E4002@bag.python.org>
	<200407290000.i6T00xE07145@guido.python.org>
	<20040729002055.GA31229@vicky.ecs.soton.ac.uk>
	<1f7befae0407281953799472ad@mail.gmail.com>
Message-ID: <1f7befae04072820551bfa2a87@mail.gmail.com>

I think I have a fix for this, and will check it in when the tests finish.

However, it doesn't fail on my box -- MS is *always* moving the memory
around.  So I hope that, before someone rebuilds with the checkin,
they temporarily fiddle the test on their box so it fails more often. 
I expect that setting the list size to 3 instead of 50 will do that. 
If you set it to 1, the test will never fail.
From marcus.cf at bol.com.br  Thu Jul 29 04:04:54 2004
From: marcus.cf at bol.com.br (marcus)
Date: Thu Jul 29 06:00:46 2004
Subject: [Python-Dev] unnamed defs
Message-ID: <loom.20040729T031054-511@post.gmane.org>

Being unsatisfied with the way lambda works, i wondered what would the python
guys think of this:

class X:
____my_static_method = staticmethod(def (x, y): {
________some statements
____})

I hope decorators will be better for this case, but it's only an example. The
word 'lambda' could be used instead of 'def', i don't know what would be better.

The motivation of this suggestion is to improve consistency of the language and
correcting the one-expression-lambda we currently have. Look how consistent Lua
is in this regard. Passing functions to functions is considered useful by many
people (essential, sometimes), but 'lambda' cannot be used even when all we need
is a function that has an 'if' or an assignment, for example. 'lambda' seems to
be a hack.

Since unnamed functions can to be used anywhere, i chose braces as delimiters,
but indentation would work normally inside them. Also, the syntax is easy to
recognize and is not ambiguous.

If Python (3.0? 2.5?) had this feature, i would be very happy!

I suspect that this subject has already been discussed, but it's easier to find
lambda tricks than a good rationale of its limitations.

From tdelaney at avaya.com  Thu Jul 29 06:06:08 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Thu Jul 29 06:06:15 2004
Subject: [Python-Dev] RE: test_sort.py failure
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A78D@au3010avexu1.global.avaya.com>

Tim Peters wrote:

> I think I have a fix for this, and will check it in when the tests
> finish. 
> 
> However, it doesn't fail on my box -- MS is *always* moving the memory
> around.  So I hope that, before someone rebuilds with the checkin,
> they temporarily fiddle the test on their box so it fails more often.
> I expect that setting the list size to 3 instead of 50 will do that.
> If you set it to 1, the test will never fail.

Might it be worthwhile trying to capture *one* starting point that
provokes this behaviour consistently on some machines and include that
as an additional specific test case?

I guess it would depend on how consistently it could be provoked.

Tim Delaney
From tim.peters at gmail.com  Thu Jul 29 06:11:31 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Jul 29 06:11:34 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <1f7befae04072820551bfa2a87@mail.gmail.com>
References: <20040728223645.3B1E61E4002@bag.python.org>
	<200407290000.i6T00xE07145@guido.python.org>
	<20040729002055.GA31229@vicky.ecs.soton.ac.uk>
	<1f7befae0407281953799472ad@mail.gmail.com>
	<1f7befae04072820551bfa2a87@mail.gmail.com>
Message-ID: <1f7befae04072821116b415e07@mail.gmail.com>

[Tim Peters]
> I think I have a fix for this, and will check it in when the tests finish.

It's checked in now.

> However, it doesn't fail on my box -- MS is *always* moving the memory
> around.  So I hope that, before someone rebuilds with the checkin,
> they temporarily fiddle the test on their box so it fails more often.
> I expect that setting the list size to 3 instead of 50 will do that.
> If you set it to 1, the test will never fail.

Ack, I meant that if you set it to 1, the test will always fail, with
or without this patch (__lt__ is never called then, so the list never
mutates then).
From greg at cosc.canterbury.ac.nz  Thu Jul 29 05:53:55 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Jul 29 06:25:35 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A3CA@au3010avexu1.global.avaya.com>
Message-ID: <200407290353.i6T3rtL8019361@cosc353.cosc.canterbury.ac.nz>

> OTOH, ensuring that if *any* exception is thrown when importing a
> module, it will not appear in sys.modules (and the exception is raised
> each time you try to import) is IMO a very useful property, and I
> think would solve what I think is Jim's issue at the root.

It's going to be difficult to ensure that nothing ever gets a
reference to a broken module, because of circular imports.  Suppose A
imports B, which imports A. If A's initialisation subsequently fails,
then even if A is removed from sys.modules, B still contains a
reference to the broken A.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Thu Jul 29 05:57:57 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Jul 29 06:25:43 2004
Subject: [Python-Dev] Re: Rationale behind removing kwdict from keyword.py?
In-Reply-To: <ce78tt$ukh$1@sea.gmane.org>
Message-ID: <200407290357.i6T3vvBe019369@cosc353.cosc.canterbury.ac.nz>

> Relying on help(), while I acknowledge may have been mistaken, is not
> IMHO a completely farfetched error.

Maybe help() should only show info for things mentioned in __all__,
if the module has one, unless told otherwise?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From tim.peters at gmail.com  Thu Jul 29 06:54:41 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Jul 29 06:54:44 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A78D@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A78D@au3010avexu1.global.avaya.com>
Message-ID: <1f7befae04072821541e90d983@mail.gmail.com>

[Delaney, Timothy C (Timothy)]
> Might it be worthwhile trying to capture *one* starting point that
> provokes this behaviour consistently on some machines and include that
> as an additional specific test case?

Fine by me if someone else can do that (it never failed on my box no
matter how I fiddled with it -- but it certainly *could* have failed
on my box before "in theory").

> I guess it would depend on how consistently it could be provoked.

That's the rub -- it depends on whether the system realloc() manages
(in the pre-patch code) to extend the initial empty_ob_item thingie
in-place, and do all subsequent extends in-place too.  The overall
odds should be better the shorter the initial list (> 1), but it still
depends on the system realloc(), and almost certainly depends too on
the state of the C malloc heap at the time the test begins.

The initial malloc request was for 1 byte, and I think There's A
Reason the MS realloc() can never extend its result for that in-place
when a non-silly number of bytes is requested (as an actual allocation
requires).  The same would be true if we had been using PyMalloc to
allocate empty_ob_item (i.e., PyMalloc could not extend it in-place
either).
From kbk at shore.net  Thu Jul 29 07:45:57 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Thu Jul 29 07:46:02 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200407290545.i6T5jvpJ015791@h006008a7bda6.ne.client2.attbi.com>


Patch / Bug Summary
___________________

Patches :  279 open ( +4) /  2483 closed (+13) /  2762 total (+17)
Bugs    :  762 open (+12) /  4303 closed (+22) /  5065 total (+34)
RFE     :  145 open ( +0) /   130 closed ( +0) /   275 total ( +0)

New / Reopened Patches
______________________

tarfile.py fix for bug #949052  (2004-07-21)
       http://python.org/sf/995126  opened by  Lars Gust?bel

tarfile.py fix for bug #990325  (2004-07-21)
CLOSED http://python.org/sf/995225  opened by  Lars Gust?bel

fix for title case bug: #995422  (2004-07-22)
       http://python.org/sf/995740  opened by  Baris Metin

Keyword argument support in cPickle  (2004-07-22)
CLOSED http://python.org/sf/995766  opened by  Dima Dorfman

Skipped test expectations for freebsd5  (2004-07-22)
CLOSED http://python.org/sf/995782  opened by  Dima Dorfman

Avoid calling tp_compare with different types  (2004-07-22)
       http://python.org/sf/995939  opened by  Dima Dorfman

hp-roman codec  (2004-07-22)
CLOSED http://python.org/sf/996067  opened by  Richard Brodie

correct urllib2 header capitalization (fix for #994101)  (2004-07-22)
CLOSED http://python.org/sf/996159  opened by  Johannes Gijsbers

New patch for Python 2.3 spec file.  (2004-07-23)
       http://python.org/sf/996316  opened by  Sean Reifschneider

Allow pydoc to work with XP Themes (.manifest file)  (2004-07-24)
       http://python.org/sf/997284  opened by  Greg Chapman

httplib.HTTPConnection.request() bug  (2004-07-25)
       http://python.org/sf/997626  opened by  John J Lee

crypt module documentation extended  (2004-07-25)
CLOSED http://python.org/sf/997668  opened by  Fernando Altomare Serboncini

imaplib deleteacl and myrights  (2004-07-26)
CLOSED http://python.org/sf/998149  opened by  Arnaud MAZIN

Decoding incomplete unicode  (2004-07-27)
       http://python.org/sf/998993  opened by  Walter D?rwald

Update kwargs in pickle docs to match implementations  (2004-07-28)
       http://python.org/sf/999280  opened by  Dima Dorfman

Patches Closed
______________

tarfile.py fix for bug #990325  (2004-07-21)
       http://python.org/sf/995225  closed by  tim_one

faster os.walk  (2004-07-19)
       http://python.org/sf/994057  closed by  loewis

Unicode in distutils.core.setup()  (2004-07-19)
       http://python.org/sf/993943  closed by  theller

Keyword argument support in cPickle  (2004-07-22)
       http://python.org/sf/995766  closed by  loewis

Skipped test expectations for freebsd5  (2004-07-22)
       http://python.org/sf/995782  closed by  loewis

gettext improvements  (2004-06-27)
       http://python.org/sf/980500  closed by  niemeyer

correct urllib2 header capitalization (fix for #994101)  (2004-07-22)
       http://python.org/sf/996159  closed by  jlgijsbers

crypt module documentation extended  (2004-07-25)
       http://python.org/sf/997668  closed by  loewis

nntplib: group descriptions and RFC2980  (2002-09-06)
       http://python.org/sf/605370  closed by  loewis

locale.getdefaultlocale fails with empty env. variable  (2004-05-29)
       http://python.org/sf/962487  closed by  loewis

imaplib deleteacl and myrights  (2004-07-26)
       http://python.org/sf/998149  closed by  loewis

hp-roman8 codec  (2004-07-22)
       http://python.org/sf/996067  closed by  lemburg

Deferred String Addtion:  Proof-of-concept  (2004-06-20)
       http://python.org/sf/976162  closed by  gvanrossum

New / Reopened Bugs
___________________

Ensure -single_module on Mac OS X  (2004-07-21)
       http://python.org/sf/995019  opened by  Bill Bumgarner

Should build Python dylib with -single_module  (2004-07-21)
CLOSED http://python.org/sf/995023  opened by  Bob Ippolito

logging.handlers.RotatingFileHandler missing error handling  (2004-07-21)
CLOSED http://python.org/sf/995109  opened by  Simon Dahlbacka

Deprecation Warning lies when it cannot parse your encoding  (2004-07-21)
CLOSED http://python.org/sf/995206  opened by  Laura Creighton

title case bug  (2004-07-21)
CLOSED http://python.org/sf/995422  opened by  Kimo Johnson

Does not build selected SGI specific modules  (2004-07-21)
       http://python.org/sf/995458  opened by  Bruce D. Ray

PEP 263 regular expression fails  (2004-07-21)
CLOSED http://python.org/sf/995522  opened by  Sjoerd Mullender

memory leak with theads and enhancement of the timer class  (2004-07-22)
       http://python.org/sf/995907  opened by  Tobias Haar

method after() and afer_idle() are not thead save  (2004-07-22)
       http://python.org/sf/995925  opened by  Tobias Haar

TclError with intel's hypertheading  (2004-07-22)
       http://python.org/sf/995956  opened by  Tobias Haar

Operations between sets  (2004-07-22)
CLOSED http://python.org/sf/995983  opened by  Francesco Ricciardi

'set' only operations  (2004-07-22)
CLOSED http://python.org/sf/995987  opened by  Francesco Ricciardi

readline function pointer with not tty's  (2004-07-22)
       http://python.org/sf/996259  opened by  Lisandro Dalcin

Bug somewhere in email Parser or PortableUnixMailbox  (2004-07-23)
CLOSED http://python.org/sf/996359  opened by  Skip Montanaro

math and cmath docs don't specify radians  (2004-07-23)
       http://python.org/sf/996392  opened by  Tim Delaney

Unicode String formatting does not correctly handle objects  (2004-06-02)
CLOSED http://python.org/sf/964929  reopened by  lemburg

normpath symlinks doc warning (bug #990669)  (2004-07-23)
       http://python.org/sf/996626  opened by  Johannes Gijsbers

realpath: resolve symlinks before normalizing (bug #990669)  (2004-07-23)
       http://python.org/sf/996627  opened by  Johannes Gijsbers

os.environ documentation should indicate unreliability  (2004-07-23)
CLOSED http://python.org/sf/996748  opened by  Richard Tibbetts

ConfigParser behavior change  (2004-07-24)
       http://python.org/sf/997050  opened by  David Goodger

optparse.py:668 triggers FutureWarning  (2004-07-24)
       http://python.org/sf/997100  opened by  Charles

What's New in 2.4: sorted(): doc bug?  (2004-07-24)
CLOSED http://python.org/sf/997166  opened by  Michael Dyck

strftime() backwards incompatibility  (2004-07-24)
       http://python.org/sf/997368  opened by  Jp Calderone

"disjunct" should be "disjoint"  (2004-07-25)
       http://python.org/sf/997533  opened by  Tim Freeman

dircache fix; raise like os.listdir, plus optimization  (2004-07-25)
       http://python.org/sf/997726  opened by  Carl Henrik Holth Lunde

LC_CTYPE locale and strings  (2004-07-26)
       http://python.org/sf/997768  opened by  Pekka Pessi

Enclosing Scope missing from namespace in Tutorial  (2004-07-26)
       http://python.org/sf/997912  opened by  Brian vdB

bsddb has default flag of c, not r  (2004-07-26)
       http://python.org/sf/998001  opened by  kim bruning

typo in documentation  (2004-07-26)
       http://python.org/sf/998066  opened by  Eric Chen

Typo in http://docs.python.org/lib/module-locale.html  (2004-07-26)
CLOSED http://python.org/sf/998170  opened by  Laura Creighton

Popen3.poll race condition  (2004-07-26)
       http://python.org/sf/998246  opened by  Tres Seaver

replace file() with open() in libcvs.tex  (2004-07-26)
       http://python.org/sf/998307  opened by  Fedor Baart

pickle bug - recursively memoizing class?  (2004-07-27)
       http://python.org/sf/998998  opened by  Skip Montanaro

Compiler module doesn't handle global statement correctly  (2004-07-27)
       http://python.org/sf/999042  opened by  Jim Fulton

compiler module doesn't support unicode characters in laiter  (2004-07-28)
       http://python.org/sf/999444  opened by  Jim Fulton

Setup.local ignored by setup.py  (2004-07-28)
       http://python.org/sf/999767  opened by  Stephan A. Terre

zlib home page wrong in three files  (2004-07-28)
CLOSED http://python.org/sf/999776  opened by  Stephan A. Terre

Bugs Closed
___________

unknown parsing error  (2004-07-03)
       http://python.org/sf/984714  closed by  loewis

Should build Python dylib with -single_module  (2004-07-21)
       http://python.org/sf/995023  closed by  etrepum

logging.handlers.RotatingFileHandler missing error handling  (2004-07-21)
       http://python.org/sf/995109  closed by  vsajip

Deprecation Warning lies when it cannot parse your encoding  (2004-07-21)
       http://python.org/sf/995206  closed by  lemburg

testtar.tar is partially insane  (2004-07-13)
       http://python.org/sf/990325  closed by  tim_one

logging module prints unexpected message when no handler set  (2004-07-20)
       http://python.org/sf/994421  closed by  vsajip

logging module: default levels do not work as documented.  (2004-07-20)
       http://python.org/sf/994416  closed by  vsajip

Py_RETURN_NONE causes too much warnings  (2004-07-19)
       http://python.org/sf/994255  closed by  tim_one

title case bug  (2004-07-21)
       http://python.org/sf/995422  closed by  rhettinger

PEP 263 regular expression fails  (2004-07-21)
       http://python.org/sf/995522  closed by  akuchling

online documentation says "Nac" in the link, "Nav" for mod  (2004-07-13)
       http://python.org/sf/990524  closed by  akuchling

cPickle doesn't like protocol keyword argument, pickle does  (2004-02-01)
       http://python.org/sf/888594  closed by  loewis

Operations between sets  (2004-07-22)
       http://python.org/sf/995983  closed by  rhettinger

'set' only operations  (2004-07-22)
       http://python.org/sf/995987  closed by  rhettinger

Bug somewhere in email Parser or PortableUnixMailbox  (2004-07-23)
       http://python.org/sf/996359  closed by  montanaro

Unicode String formatting does not correctly handle objects  (2004-06-02)
       http://python.org/sf/964929  closed by  lemburg

Unicode String formatting does not correctly handle objects  (2004-06-02)
       http://python.org/sf/964929  closed by  lemburg

os.environ documentation should indicate unreliability  (2004-07-23)
       http://python.org/sf/996748  closed by  tim_one

What's New in 2.4: sorted(): doc bug?  (2004-07-24)
       http://python.org/sf/997166  closed by  akuchling

python24.dll installation problems  (2004-07-14)
       http://python.org/sf/990945  closed by  loewis

test_unicode_file fails on Win98SE  (2004-03-28)
       http://python.org/sf/924703  closed by  loewis

Typo in http://docs.python.org/lib/module-locale.html  (2004-07-26)
       http://python.org/sf/998170  closed by  nnorwitz

zlib home page wrong in three files  (2004-07-28)
       http://python.org/sf/999776  closed by  nnorwitz

From ncoghlan at iinet.net.au  Thu Jul 29 08:05:44 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu Jul 29 08:06:22 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <1f7befae04072821541e90d983@mail.gmail.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A78D@au3010avexu1.global.avaya.com>
	<1f7befae04072821541e90d983@mail.gmail.com>
Message-ID: <410893B8.9090405@iinet.net.au>

Tim Peters wrote:
> [Delaney, Timothy C (Timothy)]
> 
>>I guess it would depend on how consistently it could be provoked.
> 
> That's the rub -- it depends on whether the system realloc() manages
> (in the pre-patch code) to extend the initial empty_ob_item thingie
> in-place, and do all subsequent extends in-place too.  The overall
> odds should be better the shorter the initial list (> 1), but it still
> depends on the system realloc(), and almost certainly depends too on
> the state of the C malloc heap at the time the test begins.

If a shorter list would provoke this more consistently, would there be a 
benefit in having test_sort run this test twice? (Once with length 50, 
once with length 3)

Regards,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From tim.peters at gmail.com  Thu Jul 29 08:21:59 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Jul 29 08:22:05 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <410893B8.9090405@iinet.net.au>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A78D@au3010avexu1.global.avaya.com>
	<1f7befae04072821541e90d983@mail.gmail.com>
	<410893B8.9090405@iinet.net.au>
Message-ID: <1f7befae04072823213fb2013d@mail.gmail.com>

[Nick Coghlan]
> If a shorter list would provoke this more consistently, would there be a
> benefit in having test_sort run this test twice? (Once with length 50,
> once with length 3)

Since I can't provoke it at all, someone who can will have to judge that one.

The test case as-is provoked segfaults in an earlier Python release,
and in more than one way, and needed a relatively large list to do so.
 It wasn't stellar at provoking this new bug, but at least three
people did see it fail.  That's good enough for me.  (And I have to
wonder how many others saw it fail but didn't think to report it until
Guido showed them it was possible <wink>.)

If someone wants to write a test to provoke the specifc bug that was
fixed, start with a list containing 3 elements.  Give the objects a
__lt__ that appends an element the first time it's called, pops that
element the second time it's called, asserts that it's not called a
3rd time, and returns True.  Whether that fails (by failing to raise
ValueError) still depends on the platform realloc(), but maximizes the
chance that the system realloc() will reuse the same memory blob
throughout.

And with that, I retire from this thread unless another bug pops up.
From ixokai at gmail.com  Thu Jul 29 09:39:25 2004
From: ixokai at gmail.com (IxokaI)
Date: Thu Jul 29 09:39:29 2004
Subject: [Python-Dev] unnamed defs
In-Reply-To: <loom.20040729T031054-511@post.gmane.org>
References: <loom.20040729T031054-511@post.gmane.org>
Message-ID: <f8ef9ae404072900393b214bd7@mail.gmail.com>

Forgive a lurker, but...

> Being unsatisfied with the way lambda works, i wondered what would the python
> guys think of this:
> 
> class X:
> ____my_static_method = staticmethod(def (x, y): {
> ________some statements
> ____})

I can't think of any reason why you wouldn't want to do:

class X:
    def myStaticMethod(x, y):
        some statements
    myStaticMethod = staticmethod(myStaticMethod)

I know its just an example, but I just don't get the allure of
anonymous functions. There's no extra typing in this example, its not
any more clear, and in fact, it looks less clear to me. That doesn't
jump out at me saying 'function definition!'. And in situations where
it may be less typing, its still IMHO always less clear.

--Stephen
From arigo at tunes.org  Thu Jul 29 11:33:55 2004
From: arigo at tunes.org (Armin Rigo)
Date: Thu Jul 29 11:39:19 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <1f7befae04072821116b415e07@mail.gmail.com>
References: <20040728223645.3B1E61E4002@bag.python.org>
	<200407290000.i6T00xE07145@guido.python.org>
	<20040729002055.GA31229@vicky.ecs.soton.ac.uk>
	<1f7befae0407281953799472ad@mail.gmail.com>
	<1f7befae04072820551bfa2a87@mail.gmail.com>
	<1f7befae04072821116b415e07@mail.gmail.com>
Message-ID: <20040729093355.GA10255@vicky.ecs.soton.ac.uk>

Hello Tim,

On Thu, Jul 29, 2004 at 12:11:31AM -0400, Tim Peters wrote:
> It's checked in now.

Unfortunately it is not true that ob_item can never be set to NULL with normal
list operations.  list_ass_slice() and list_inplace_repeat() can both do that.  
The former is not so rare as it is used e.g. for 'del lst[:]'.

The invariants of the ob_xxx list members should be clearly specified
somewhere and we should stick to them.  E.g. it is unclear to me why we allow
list_ass_slice() to reset ob_item to NULL and ob_size to 0 without resetting
ob_allocated to 0 -- I see why it doesn't crash in a subsequent list_resize(),
but it looks messy.

Alternatively, the quickest fix for the broken listsort() would have been to
keep the empty_ob_item hack but just check that ob_allocated is still zero in
addition to ob_size.  However the current version is quite clean and can be
made bulletproof by ensuring that ob_item is never reset to NULL.


A bientot,

Armin.
From arigo at tunes.org  Thu Jul 29 12:07:28 2004
From: arigo at tunes.org (Armin Rigo)
Date: Thu Jul 29 12:12:16 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <20040729093355.GA10255@vicky.ecs.soton.ac.uk>
References: <20040728223645.3B1E61E4002@bag.python.org>
	<200407290000.i6T00xE07145@guido.python.org>
	<20040729002055.GA31229@vicky.ecs.soton.ac.uk>
	<1f7befae0407281953799472ad@mail.gmail.com>
	<1f7befae04072820551bfa2a87@mail.gmail.com>
	<1f7befae04072821116b415e07@mail.gmail.com>
	<20040729093355.GA10255@vicky.ecs.soton.ac.uk>
Message-ID: <20040729100728.GA13201@vicky.ecs.soton.ac.uk>

Hello again,

I will check the following test in when it is fixed:

    def test_undetected_mutation(self):
        # Python 2.4a1 did not always detect mutation
        memorywaster = []
        for i in range(20):
            def mutating_cmp(x, y):
                L.append(3)
                L.pop()
                return cmp(x, y)
            L = [1,2]
            self.assertRaises(ValueError, L.sort, mutating_cmp)
            def mutating_cmp(x, y):
                L.append(3)
                del L[:]
                return cmp(x, y)
            self.assertRaises(ValueError, L.sort, mutating_cmp)
            memorywaster = [memorywaster]

In the two parts, the list is emptied using two different ways: L.pop() which
uses list_resize(), and del L[:] which uses list_ass_slice().  On my Linux
machine, the first test eventually fails on yesterday's CVS head, but the
second one passes.  Conversely, today's CVS makes the first test pass but the
second one fail.


Armin
From arigo at tunes.org  Thu Jul 29 14:36:00 2004
From: arigo at tunes.org (Armin Rigo)
Date: Thu Jul 29 14:40:48 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <20040729100728.GA13201@vicky.ecs.soton.ac.uk>
References: <20040728223645.3B1E61E4002@bag.python.org>
	<200407290000.i6T00xE07145@guido.python.org>
	<20040729002055.GA31229@vicky.ecs.soton.ac.uk>
	<1f7befae0407281953799472ad@mail.gmail.com>
	<1f7befae04072820551bfa2a87@mail.gmail.com>
	<1f7befae04072821116b415e07@mail.gmail.com>
	<20040729093355.GA10255@vicky.ecs.soton.ac.uk>
	<20040729100728.GA13201@vicky.ecs.soton.ac.uk>
Message-ID: <20040729123600.GA6844@vicky.ecs.soton.ac.uk>

Hello,

The following (just checked in) changes should solve the problem:

* drop the unreasonable list invariant that ob_item should never come back
  to NULL during the lifetime of the object.

* listobject.c nevertheless did not conform to the other invariants,
  either; fixed.

* listobject.c now uses list_clear() as the obvious internal way to clear
  a list, instead of abusing list_ass_slice() for that.  It makes it easier
  to enforce the invariant about ob_item == NULL.

* listsort() sets allocated to -1 during sort; any mutation will set it
  to a value >= 0, so it is a safe way to detect mutation.  A negative
  value for allocated does not cause a problem elsewhere currently.
  test_sort.py has a new test for this fix.

* listsort() leak: if items were added to the list during the sort, AND if
  these items had a __del__ that puts still more stuff into the list,
  then this more stuff (and the PyObject** array to hold them) were
  overridden at the end of listsort() and never released.


Armin
From skip at pobox.com  Thu Jul 29 16:21:26 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Jul 29 16:21:48 2004
Subject: [Python-Dev] Re: Rationale behind removing kwdict from keyword.py?
In-Reply-To: <200407290357.i6T3vvBe019369@cosc353.cosc.canterbury.ac.nz>
References: <ce78tt$ukh$1@sea.gmane.org>
	<200407290357.i6T3vvBe019369@cosc353.cosc.canterbury.ac.nz>
Message-ID: <16649.2022.901462.705712@montanaro.dyndns.org>


    >> Relying on help(), while I acknowledge may have been mistaken, is not
    >> IMHO a completely farfetched error.

    Greg> Maybe help() should only show info for things mentioned in
    Greg> __all__, if the module has one, unless told otherwise?

It does, as a side-effect of the pydoc patch I referred to in an earlier
message.

Skip

From aahz at pythoncraft.com  Thu Jul 29 18:41:34 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Jul 29 18:41:41 2004
Subject: [Python-Dev] unnamed defs
In-Reply-To: <loom.20040729T031054-511@post.gmane.org>
References: <loom.20040729T031054-511@post.gmane.org>
Message-ID: <20040729164134.GA24187@panix.com>

On Thu, Jul 29, 2004, marcus wrote:
>
> Being unsatisfied with the way lambda works, i wondered what would the
> python guys think of this: [...]

While this isn't precisely off-topic for python-dev, it's been hashed so
many times that it's better to see if you can get some consensus on
comp.lang.python before bringing it up here.  In fact, I'd say that you
should have a full-fledged PEP before bringing it back to python-dev.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From guido at python.org  Thu Jul 29 20:11:08 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul 29 20:11:15 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: Your message of "Thu, 29 Jul 2004 13:36:00 BST."
	<20040729123600.GA6844@vicky.ecs.soton.ac.uk> 
References: <20040728223645.3B1E61E4002@bag.python.org>
	<200407290000.i6T00xE07145@guido.python.org>
	<20040729002055.GA31229@vicky.ecs.soton.ac.uk>
	<1f7befae0407281953799472ad@mail.gmail.com>
	<1f7befae04072820551bfa2a87@mail.gmail.com>
	<1f7befae04072821116b415e07@mail.gmail.com>
	<20040729093355.GA10255@vicky.ecs.soton.ac.uk>
	<20040729100728.GA13201@vicky.ecs.soton.ac.uk> 
	<20040729123600.GA6844@vicky.ecs.soton.ac.uk> 
Message-ID: <200407291811.i6TIB8w10700@guido.python.org>

> The following (just checked in) changes should solve the problem:
> 
> * drop the unreasonable list invariant that ob_item should never come back
>   to NULL during the lifetime of the object.
> 
> * listobject.c nevertheless did not conform to the other invariants,
>   either; fixed.
> 
> * listobject.c now uses list_clear() as the obvious internal way to clear
>   a list, instead of abusing list_ass_slice() for that.  It makes it easier
>   to enforce the invariant about ob_item == NULL.
> 
> * listsort() sets allocated to -1 during sort; any mutation will set it
>   to a value >= 0, so it is a safe way to detect mutation.  A negative
>   value for allocated does not cause a problem elsewhere currently.
>   test_sort.py has a new test for this fix.
> 
> * listsort() leak: if items were added to the list during the sort, AND if
>   these items had a __del__ that puts still more stuff into the list,
>   then this more stuff (and the PyObject** array to hold them) were
>   overridden at the end of listsort() and never released.

Wow.  I'm impressed.  Thanks!

BTW, I just wrote a little programing for playing Conway's Game of
Life, using Tkinter.  (Yes, I was inspired by Damien Conway's (no
relation) talk Tuesday night at OSCON.)  Anyway, it runs at about 5.35
generations/second in Python 2.3, and 6.35 generations/second in
Python 2.4.  Good job everyone!

--Guido van Rossum (home page: http://www.python.org/~guido/)
From amk at amk.ca  Thu Jul 29 20:16:35 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Thu Jul 29 20:18:14 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <200407291811.i6TIB8w10700@guido.python.org>
References: <20040728223645.3B1E61E4002@bag.python.org>
	<200407290000.i6T00xE07145@guido.python.org>
	<20040729002055.GA31229@vicky.ecs.soton.ac.uk>
	<1f7befae0407281953799472ad@mail.gmail.com>
	<1f7befae04072820551bfa2a87@mail.gmail.com>
	<1f7befae04072821116b415e07@mail.gmail.com>
	<20040729093355.GA10255@vicky.ecs.soton.ac.uk>
	<20040729100728.GA13201@vicky.ecs.soton.ac.uk>
	<20040729123600.GA6844@vicky.ecs.soton.ac.uk>
	<200407291811.i6TIB8w10700@guido.python.org>
Message-ID: <20040729181635.GA19071@rogue.amk.ca>

On Thu, Jul 29, 2004 at 11:11:08AM -0700, Guido van Rossum wrote:
> BTW, I just wrote a little programing for playing Conway's Game of
> Life, using Tkinter.  (Yes, I was inspired by Damien Conway's (no

Also see Demo/curses/life.py.

--amk
From guido at python.org  Thu Jul 29 20:20:27 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul 29 20:20:32 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Thu, 29 Jul 2004 15:53:55 +1200."
	<200407290353.i6T3rtL8019361@cosc353.cosc.canterbury.ac.nz> 
References: <200407290353.i6T3rtL8019361@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200407291820.i6TIKRi10773@guido.python.org>

> It's going to be difficult to ensure that nothing ever gets a
> reference to a broken module, because of circular imports.  Suppose A
> imports B, which imports A. If A's initialisation subsequently fails,
> then even if A is removed from sys.modules, B still contains a
> reference to the broken A.

Yeah, but I don't mind; my patch is still a big improvement in other
cases and doesn't really make things any worse in the above case (B
has a reference to a broken A, just like without my patch).

Did anybody look at my patch?  Shall I check it in?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Jul 29 20:24:32 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Jul 29 20:26:20 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: Your message of "Thu, 29 Jul 2004 14:16:35 EDT."
	<20040729181635.GA19071@rogue.amk.ca> 
References: <20040728223645.3B1E61E4002@bag.python.org>
	<200407290000.i6T00xE07145@guido.python.org>
	<20040729002055.GA31229@vicky.ecs.soton.ac.uk>
	<1f7befae0407281953799472ad@mail.gmail.com>
	<1f7befae04072820551bfa2a87@mail.gmail.com>
	<1f7befae04072821116b415e07@mail.gmail.com>
	<20040729093355.GA10255@vicky.ecs.soton.ac.uk>
	<20040729100728.GA13201@vicky.ecs.soton.ac.uk>
	<20040729123600.GA6844@vicky.ecs.soton.ac.uk>
	<200407291811.i6TIB8w10700@guido.python.org> 
	<20040729181635.GA19071@rogue.amk.ca> 
Message-ID: <200407291824.i6TIOWj10831@guido.python.org>

> On Thu, Jul 29, 2004 at 11:11:08AM -0700, Guido van Rossum wrote:
> > BTW, I just wrote a little programing for playing Conway's Game of
> > Life, using Tkinter.  (Yes, I was inspired by Damien Conway's (no
> 
> Also see Demo/curses/life.py.

Which I looked into first, but which doesn't run on Windows AFAICT.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From theller at python.net  Thu Jul 29 20:33:13 2004
From: theller at python.net (Thomas Heller)
Date: Thu Jul 29 20:33:25 2004
Subject: [Python-Dev] Static builds on Windows
In-Reply-To: <4108243B.4040706@iinet.net.au> (Nick Coghlan's message of
	"Wed, 28 Jul 2004 15:10:03 -0700")
References: <oem0597y.fsf@python.net> <4107F8B3.6080206@v.loewis.de>
	<8yd352cf.fsf@python.net> <41081CF6.7050609@v.loewis.de>
	<4108243B.4040706@iinet.net.au>
Message-ID: <3c3airyu.fsf@python.net>

Nick Coghlan <ncoghlan@iinet.net.au> writes:

> Martin v. L?wis wrote:
>
>> Thomas Heller wrote:
>>
>>> Related to this issue: Would it be a good idea to add a project to the
>>> MSVC workspace which builds a static library?  This could either be used
>>> directly, or as a staring point for customization.

>> Depends on whether it is going to be maintained. If it is, it might
>> be a good idea. If it is once created, and then only updated every
>> two years when somebody actually needs it, it might be pointless - it
>> would be just as much work to recreate it from scratch every time.
>
> If there was going to be both the ability to create a static lib and a
> dynamic lib, it would seem to make more sense to have the project for
> the dynamic lib project simply incorporate the static lib, and export
> the relevant symbols.

That is a good idea, and would solve the maintainace problem.
Unfortunately, it doesn't work with the current setup, since modules
have to be compiled with different preprocessor definitions for the
static and the shared lib.

> We do something similar at work (VC6, rather than 7), although we're
> combining a dozen or so static libs to produce the DLL. All the real
> source code is in the libraries, and the DLL project consists mainly
> of a list of include directives, and enough symbol references to get
> everything exported correctly.

I guess you need a def file as well, or another way to define the
exported functions?

Thomas

From tim.peters at gmail.com  Thu Jul 29 20:34:52 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Jul 29 20:34:56 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <20040729093355.GA10255@vicky.ecs.soton.ac.uk>
References: <20040728223645.3B1E61E4002@bag.python.org>
	<200407290000.i6T00xE07145@guido.python.org>
	<20040729002055.GA31229@vicky.ecs.soton.ac.uk>
	<1f7befae0407281953799472ad@mail.gmail.com>
	<1f7befae04072820551bfa2a87@mail.gmail.com>
	<1f7befae04072821116b415e07@mail.gmail.com>
	<20040729093355.GA10255@vicky.ecs.soton.ac.uk>
Message-ID: <1f7befae04072911343ee9833c@mail.gmail.com>

[Armin Rigo]
> Unfortunately it is not true that ob_item can never be set to NULL with normal
> list operations.  list_ass_slice() and list_inplace_repeat() can both do that.
> The former is not so rare as it is used e.g. for 'del lst[:]'.

Thank you for digging into this!  I like all the changes you made.

> The invariants of the ob_xxx list members should be clearly specified
> somewhere

None were, but as part of my stab at fixing this I documented what I
believe to be minimally sane listobject invariants, as comments in the
declaration of the listobject struct.

> and we should stick to them.

You think <wink>?  I confess I made no effort to verify that the
invariants actually hold.  My view was that the invariants were so
mild and so necessary that any violation discovered should (and would)
be treated as a bug.

>  E.g. it is unclear to me why we allow list_ass_slice() to reset ob_item to NULL
> and ob_size to 0 without resetting ob_allocated to 0

Since that violates one of the now-documented invariants, you can
guess my position on that.

> -- I see why it doesn't crash in a subsequent list_resize(), but it looks messy.

There's no adequate reason to endure convolutions like this to save a
theoretical nanosecond in a rare case -- if indeed that ever was
someone's intent.  I expect it's more likely an oversight, and largely
due to the previous lack of documenting intended invariants.

> Alternatively, the quickest fix for the broken listsort() would have been to
> keep the empty_ob_item hack but just check that ob_allocated is still zero in
> addition to ob_size.

Too ugly; I don't think either of us *liked* the empty_ob_item hack,
and if I was going to piss away more time on this I was determined to
get rid of it <wink>.

> However the current version is quite clean and can be
> made bulletproof by ensuring that ob_item is never reset to NULL.

It was indeed cleaner, and I'm all in favor of your later change to
get rid of the ad hoc "ob_item != NULL implies ob_item will never be
NULL" invariant (which I agree was far more expedient than
reasonable).
From martin at v.loewis.de  Thu Jul 29 20:40:48 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul 29 20:40:52 2004
Subject: [Python-Dev] Static builds on Windows
In-Reply-To: <4108243B.4040706@iinet.net.au>
References: <oem0597y.fsf@python.net>	<4107F8B3.6080206@v.loewis.de>	<8yd352cf.fsf@python.net>	<41081CF6.7050609@v.loewis.de>
	<4108243B.4040706@iinet.net.au>
Message-ID: <410944B0.8090104@v.loewis.de>

Nick Coghlan wrote:
> If there was going to be both the ability to create a static lib and a 
> dynamic lib, it would seem to make more sense to have the project for 
> the dynamic lib project simply incorporate the static lib, and export 
> the relevant symbols.

Does that actually work? Doesn't the compiler generate different code
depending on whether the code is going to be in a static vs. dynamic
library?

If you can provide a patch to the project files which makes it happen
for the VC7 projects, I would be very much in favour of including it
(provided it can be demonstrated that it does actually work, and does
not cause much overhead beyond the additional static library that is
going to be generated).

Also, there is the issue of what C runtime to chose: should that be
the static CRT, or a dynmaic one? Clearly, for the DLL, we need the
dynamic CRT, but for the static python.lib, we might need a static
CRT instead.

If the issue is only about repackaging the object files with an
invocation of lib.exe - we could write a Python script that
traverses the x86-temp-release directories, collecting all the
object files and building a library from them.

Regards,
Martin
From martin at v.loewis.de  Thu Jul 29 20:43:01 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul 29 20:43:07 2004
Subject: [Python-Dev] Static builds on Windows
In-Reply-To: <3c3airyu.fsf@python.net>
References: <oem0597y.fsf@python.net>
	<4107F8B3.6080206@v.loewis.de>	<8yd352cf.fsf@python.net>
	<41081CF6.7050609@v.loewis.de>	<4108243B.4040706@iinet.net.au>
	<3c3airyu.fsf@python.net>
Message-ID: <41094535.6040102@v.loewis.de>

Thomas Heller wrote:
> I guess you need a def file as well, or another way to define the
> exported functions?

I'd assume that the current mechanism of DL_EXPORT stays. Then, the
DLL project simply needs to link all object files together - either
by referencing symbols, or by explicitly linking all object files
together into a DLL. The __declspec(dllexport) declarations will
cause the proper symbols to get exported.

Of course, the same will happen if the static library is incorporated
into an executable - that will also export all symbols.

Regards,
Martin
From martin at v.loewis.de  Thu Jul 29 21:01:30 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul 29 21:01:36 2004
Subject: [Python-Dev] MSI Documentation
In-Reply-To: <02a901c474f9$ed6c1a20$0200a8c0@eden>
References: <02a901c474f9$ed6c1a20$0200a8c0@eden>
Message-ID: <4109498A.5020109@v.loewis.de>

Mark Hammond wrote:
> That looks nice (even though I misread your email and thought I would be
> looking at documentation on *building* the MSI.)  Your document on *using*
> the MSI has a much wider audience.
> 
> Out of interest, why can't we "advertise"?

To advertise, we would need the Extension table, and the Verb table for
extensions. I have tried to fill them out, and couldn't get it to work:
- The extension table associates (conceptually) a single binary with
   the extension. That binary is specified through the Component_ field
   of the Extension record, and is the keyfile of the component.
   So we would associate:
   .pyw: pythonw.exe
   .pyc: python.exe
   .py: python.exe
- The verb table then only adds *additional* command line arguments;
   the executable is always the same. Unfortunately, "Edit with IDLE"
   always invokes pythonw.exe, even for .py files.
Just not advertising "Edit with IDLE" might have been an option, but
I did not try to implement that approach. Currently, extensions are
authored directly into the Registry table.

To advertise short-cuts, there is already code in msi.py which works
mostly correctly. Unfortunately, after a fresh install, the icons are
not displayed - you have to activate the shortcut once to make the
icon show up. I *believe* this is because we use the wrong form of
icon: In the description of the Icon table, it says

"However, Icon files that are associated with shortcuts must be in the 
EXE binary format and must be named such that their extension matches 
the extension of the target."

I don't know how to generate the EXE binary format from the ICO format
that we currently have - feel free to experiment with that if you are
curious.

The other issue with advertised shortcuts is, of course, that installer
invokes a repair installation if the key file goes away - which may
or may not be a good thing.

Regards,
Martin


From martin at v.loewis.de  Thu Jul 29 21:06:07 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul 29 21:06:10 2004
Subject: [Python-Dev] MSI Documentation
In-Reply-To: <02a901c474f9$ed6c1a20$0200a8c0@eden>
References: <02a901c474f9$ed6c1a20$0200a8c0@eden>
Message-ID: <41094A9F.8060601@v.loewis.de>

Mark Hammond wrote:
> That looks nice (even though I misread your email and thought I would be
> looking at documentation on *building* the MSI.) 

I'm willing to work on this, but I need some guidance. As you might have
seen, I have now added notion of a config.py which controls how the MSI
is built. This gives independence from the source directory, and allows
for multiple check-outs of the msi directory, each for a different
Python build and version (I have currently two of them: one for 2.4,
and one for 2.4-Itanium). This config.py is documented in msi.py.

As for more documentation on the mechanisms of msi.py itself: I have
asked a colleague to go through the file and comment on parts of Python
code that might be unclear. However, I have the concern that all his
questions are likely answered in MSDN - so I'd be really curious what
documentation is lacking that is *not* part of MSDN.

Regards,
Martin
From theller at python.net  Thu Jul 29 21:10:47 2004
From: theller at python.net (Thomas Heller)
Date: Thu Jul 29 21:10:57 2004
Subject: [Python-Dev] Static builds on Windows
In-Reply-To: <41094535.6040102@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Thu,
	29 Jul 2004 20:43:01 +0200")
References: <oem0597y.fsf@python.net> <4107F8B3.6080206@v.loewis.de>
	<8yd352cf.fsf@python.net> <41081CF6.7050609@v.loewis.de>
	<4108243B.4040706@iinet.net.au> <3c3airyu.fsf@python.net>
	<41094535.6040102@v.loewis.de>
Message-ID: <wu0mhbns.fsf@python.net>

"Martin v. L?wis" <martin@v.loewis.de> writes:

> Thomas Heller wrote:
>> I guess you need a def file as well, or another way to define the
>> exported functions?
>
> I'd assume that the current mechanism of DL_EXPORT stays. Then, the
> DLL project simply needs to link all object files together - either
> by referencing symbols, or by explicitly linking all object files
> together into a DLL. The __declspec(dllexport) declarations will
> cause the proper symbols to get exported.

I'm lost in these symbols.  It seems, the DL_EXPORT symbol is already
deprecated, and public functions should be, and are, declared
PyAPI_FUNC.  OTOH, the only way to define PyAPI_FUNC as
__declspec(dllexport) is to define Py_ENABLE_SHARED and Py_BUILD_CORE.
And isn't Py_ENABLE_SHARED mutually exclusive with Py_NO_ENABLE_SHARED?

> Of course, the same will happen if the static library is incorporated
> into an executable - that will also export all symbols.

Which makes the exe slightly larger, because of the import table
section.  I have yet to figure out if extensions can link to these
functions - normally they expect the functions in the pythonxy.dll
module, and not in the (say) python_static.exe.

Thomas

From python at rcn.com  Thu Jul 29 09:21:30 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Jul 29 21:23:05 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <1f7befae04072911343ee9833c@mail.gmail.com>
Message-ID: <000e01c4753c$aba325a0$c830cb97@oemcomputer>

> I confess I made no effort to verify that the
> invariants actually hold.  My view was that the invariants were so
> mild and so necessary that any violation discovered should (and would)
> be treated as a bug.
> 
> >  E.g. it is unclear to me why we allow list_ass_slice() to reset
ob_item
> to NULL
> > and ob_size to 0 without resetting ob_allocated to 0
> 
> Since that violates one of the now-documented invariants, you can
> guess my position on that.
> 
> > -- I see why it doesn't crash in a subsequent list_resize(), but it
> looks messy.

list_resize() doesn't assume a valid value for ob_allocated unless
ob_item != NULL.  The idea was to avoid imposing new invariants on the
list structure so that the remaining code and extensions would be easier
to maintain.  Ideally, no code outside of PyList_New() and list_resize()
would need to know about the ob_allocated field.

If we want to insist on the ob_allocated invariants, then

* list_ass_slice() should add a line to maintain them,
* list_resize() can drop the test for ob_item != NULL, and
* tp_new needs a custom list_new() establishing the invariants.



> > Alternatively, the quickest fix for the broken listsort() would have
> been to
> > keep the empty_ob_item hack but just check that ob_allocated is
still
> zero in
> > addition to ob_size.
> 
> Too ugly; I don't think either of us *liked* the empty_ob_item hack,
> and if I was going to piss away more time on this I was determined to
> get rid of it <wink>.

That would be nice.  list_sort() serves as the model for how to defend C
code against list mutation.



Raymond

From martin at v.loewis.de  Thu Jul 29 22:12:39 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Jul 29 22:12:50 2004
Subject: [Python-Dev] Static builds on Windows
In-Reply-To: <wu0mhbns.fsf@python.net>
References: <oem0597y.fsf@python.net>
	<4107F8B3.6080206@v.loewis.de>	<8yd352cf.fsf@python.net>
	<41081CF6.7050609@v.loewis.de>	<4108243B.4040706@iinet.net.au>
	<3c3airyu.fsf@python.net>	<41094535.6040102@v.loewis.de>
	<wu0mhbns.fsf@python.net>
Message-ID: <41095A37.3070403@v.loewis.de>

Thomas Heller wrote:
> I'm lost in these symbols.  It seems, the DL_EXPORT symbol is already
> deprecated, and public functions should be, and are, declared
> PyAPI_FUNC.  

That might well be. My reference to DL_EXPORT was from memory, and I
was too lazy to look up the current macro usage. Looking again.

> OTOH, the only way to define PyAPI_FUNC as
> __declspec(dllexport) is to define Py_ENABLE_SHARED and Py_BUILD_CORE.
> And isn't Py_ENABLE_SHARED mutually exclusive with Py_NO_ENABLE_SHARED?

Right. Why again did you want to define Py_NO_ENABLE_SHARED? I would
think that you can define it and still build a static library.

> Which makes the exe slightly larger, because of the import table
> section.  I have yet to figure out if extensions can link to these
> functions - normally they expect the functions in the pythonxy.dll
> module, and not in the (say) python_static.exe.

Yes. You will need to provide an import library for the final
executable, and you need to call this import library python24.lib.
Then, extensions should pick it up automatically through the
#pragma comment.

Existing binary extensions are not affected - although it might
be possible to provide  a stub python24.dll which reexports all
symbols from <interpreter>.exe. This would be needed only if
extension modules are needed.

If you want to disable dynamic loading at run-time, you could offer
a patch that allows to disable dynamic loading - either at the C
API or even exposed to Python. One implementation of this function
could garble _PyImport_Filetab, replacing certain entries with
SEARCH_ERROR, e.g. through

PyImport_DisableExtension(".pyd");

Re-enabling a disabled extension is probably not needed.

Regards,
Martin
From walter at livinglogic.de  Thu Jul 29 22:30:43 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu Jul 29 22:30:47 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4107F0E8.3010005@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>
	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>
	<4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com>
Message-ID: <41095E73.5070001@livinglogic.de>

M.-A. Lemburg wrote:

> Walter D?rwald wrote:
> 
>> M.-A. Lemburg wrote:
>>
>>> Walter D?rwald wrote:
>>> [...]
>>> The reason why stateless encode and decode APIs return the
>>> number of input items consumed is to accomodate for error
>>> handling situations like these where you want to stop
>>> coding and leave the remaining work to another step.

But then this turns into a stateful decoder. What would
happen when stateless decoders suddenly started to decode
less than the complete string? Every user would
have to check whether decoder(foo)[1] == len(foo).

>> [...]
>> I wonder whether the decode method is part of the public
>> API for StreamReader.
> 
> It is: StreamReader/Writer are "sub-classes" of the Codec
> class.
> 
> However, there's nothing stating that .read() or .write()
> *must* use these methods to do their work and that's
> intentional.

Any read() method can be implemented on top of a stateful
decode() method.

>>> I see two possibilities here:
>>>
>>> 1. you write a custom StreamReader/Writer implementation
>>>    for each of the codecs which takes care of keeping
>>>    state and encoding/decoding as much as possible
>>
>> But I'd like to reuse at least some of the functionality
>> from PyUnicode_DecodeUTF8() etc.
>>
>> Would a version of PyUnicode_DecodeUTF8() with an additional
>> PyUTF_DecoderState * be OK?
> 
> Before you start putting more work into this, let's first
> find a good workable approach.

I agree that we need a proper design for this that gives us
the most convenient codec API with breaking backwards
compatibility (at least not for codec users). Breaking
compatibility for codec implementers shouldn't be an issue.
I'll see if I can come up with something over the weekend.

>>> 2. you extend the existing stateless codec implementations
>>>    to allow communicating state on input and output; the
>>>    stateless operation would then be a special case
>>>
>>>> But this isn't really a "StreamReader" any more, so the best
>>>> solution would probably be to have a three level API:
>>>> * A stateless decoding function (what codecs.getdecoder
>>>>   returns now);
>>>> * A stateful "feed reader", which keeps internal state
>>>>   (including undecoded byte sequences) and gets passed byte
>>>>   chunks (should this feed reader have a error attribute or
>>>>   should this be an argument to the feed method?);
>>>> * A stateful stream reader that reads its input from a
>>>>   byte stream. The functionality for the stream reader could
>>>>   be implemented once using the underlying functionality of
>>>>   the feed reader (in fact we could implement something similar
>>>>   to sio's stacking streams: the stream reader would use
>>>>   the feed reader to wrap the byte input stream and add
>>>>   only a read() method. The line reading methods (readline(),
>>>>   readlines() and __iter__() could be added by another stream
>>>>   filter)
>>>
>>> Why make things more complicated ?
>>>
>>> If you absolutely need a feed interface, you can feed
>>> your data to a StringIO instance which is then read from
>>> by StreamReader.
>>
>> This doesn't work, because a StringIO has only one file position:
>>  >>> import cStringIO
>>  >>> s = cStringIO.StringIO()
>>  >>> s.write("x")
>>  >>> s.read()
>> ''
> 
> Ah, you wanted to do both feeding and reading at the same
> time ?!

There is no other way. You pass the feeder byte string chunks
and it returns the chunks of decoded objects. With the StreamReader
the reader itself will read those chunks from the underlying
stream.

Implementing a stream reader interface on top of a feed interface
is trivial: Basically our current decode method *is* the feed
interface, the only problem is that the user has to keep state
(the undecoded bytes that have to be passed to the next call to
decode). Move that state into an attribute of the instance and
drop it from the return value and you have a feed interface.

>> But something like the Queue class from the tests in the patch
>> might work.
> 
> Right... I don't think that we need a third approach to
> codecs just to implement feed based parsers.

We already have most of the functionality in the decode method.

>>>>> The error callbacks could, however, raise an exception which
>>>>> includes all the needed information, including any state that
>>>>> may be needed in order to continue with coding operation.
>>>>
>>>> This makes error callbacks effectively unusable with stateful
>>>> decoders.
>>>
>>> Could you explain ?
>>
>> If you have to call the decode function with errors='break',
>> you will only get the break error handling and nothing else.
> 
> Yes and ... ? What else do you want it to do ?

The user can pass any value for the errors argument in the
StreamReader constructor. The StreamReader should always honor
this error handling strategy. Example.

import codecs, cStringIO

count = 0
def countandreplace(exc):
    global count
    if not isinstance(exc, UnicodeDecodeError):
       raise TypeError("can handle error")
    count += 1
    return (u"\ufffd", exc.end)

codecs.register_error("countandreplace", countandreplace)

s = cStringIO.StringIO("\xc3foo\xffbar\xc3")

us = codecs.getreader("utf-8")(s)

The first \xc3 and the \xff are real errors, the trailing
\xc3 might be a transient one. To handle this with the break
handler strategy the StreamReader would have to call the
decode() method with errors="break" instead of errors="countandreplace".
break would then have to decide whether it's a transient
error or a real one (presumably from some info in the exception).
If it's a real one it would have to call the original error
handler, but it doesn't have any way of knowing what the
original error handler was. If it's a transient error, it
would have to comunicate this fact to the caller, which could
be done by changing an attribute in the exception object.
But the decoding function still has to put the retained
bytes into the StreamReader so that part doesn't get any
simpler. Alltogether I find this method rather convoluted,
especially since we have most of the machinery in place.
What is missing is the implementation of real stateful
decoding functions.

>>>>> We may then need to allow additional keyword arguments on the
>>>>> encode/decode functions in order to preset a start state.
>>>>
>>>> As those decoding functions are private to the decoder that's
>>>> probably OK. I wouldn't want to see additional keyword arguments
>>>> on str.decode (which uses the stateless API anyway). BTW, that's
>>>> exactly what I did for codecs.utf_7_decode_stateful, but I'm not
>>>> really comfortable with the internal state of the UTF-7 decoder
>>>> being exposed on the Python level. It would be better to encapsulate
>>>> the state in a feed reader implemented in C, so that the state is
>>>> inaccessible from the Python level.
>>>
>>> See above: possibility 1 would be the way to go then.
>>
>> I might give this a try.
> 
> Again, please wait until we have found a good solution
> to this.

OK.

Bye,
    Walter D?rwald


From jim at zope.com  Thu Jul 29 23:02:47 2004
From: jim at zope.com (Jim Fulton)
Date: Thu Jul 29 23:03:28 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200407282213.i6SMDWD06878@guido.python.org>
References: <Your message of "Wed, 28 Jul 2004 13:40:03
	EDT."	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>	<Your
	message of "Wed, 28 Jul 2004 06:59:24 EDT."
	<4107870C.1070404@zope.com>	<41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>	<1f7befae040727085715ce2536@mail.gmail.com>	<4106853A.8080109@zope.com>	<20040727164913.GS7708@epoch.metaslash.com>	<41068C2F.40102@zope.com>	<200407272050.i6RKoEY25164@guido.python.org>	<4107870C.1070404@zope.com>	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com>
	<200407282213.i6SMDWD06878@guido.python.org>
Message-ID: <410965F7.7050605@zope.com>

Guido van Rossum wrote:
> [Guido]
> 

...

> But who ever uses __module__ for anything else than printing it?

It's crucial for pickling.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From pje at telecommunity.com  Thu Jul 29 23:26:25 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Jul 29 23:22:43 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <410965F7.7050605@zope.com>
References: <200407282213.i6SMDWD06878@guido.python.org> <Your message of "Wed,
	28 Jul 2004 13:40:03
	EDT."	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<Your message of "Wed,
	28 Jul 2004 06:59:24 EDT." <4107870C.1070404@zope.com>
	<41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
	<41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org>
	<4107870C.1070404@zope.com>
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com>
	<200407282213.i6SMDWD06878@guido.python.org>
Message-ID: <5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com>

At 05:02 PM 7/29/04 -0400, Jim Fulton wrote:
>Guido van Rossum wrote:
>>But who ever uses __module__ for anything else than printing it?
>
>It's crucial for pickling.

Interesting point.  That means that cPickle wouldn't work in a "module 
space" without modification.  The pure-Python pickle module would unpickle 
into the current module space.

From tim.peters at gmail.com  Thu Jul 29 23:50:57 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Jul 29 23:50:59 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <000e01c4753c$aba325a0$c830cb97@oemcomputer>
References: <000e01c4753c$aba325a0$c830cb97@oemcomputer>
Message-ID: <1f7befae040729145073281b0@mail.gmail.com>

[Raymond Hettinger]
...
> If we want to insist on the ob_allocated invariants, then

I want to insist on *documented* invariants, but I don't particularly
care what they are.  Despite the seemingly endless necessity for doing
so, it's actually not possible to reverse-engineer intended invariants
from staring at thousands of lines of code (not in C, and not in
Python code either).  Indeed, that's how list.sort() got broken here
to begin with:  you failed to divine the undocumented invariant it
relied on, and so you managed to break that invariant.  It's really
not surprising!

The idea that "list.ob_item == NULL implies list.allocated may be
lying" is at least as obscure as the invariant that broke, and if it's
intended but not documented it will be implicated in future bugs. 
There's no force working toward keeping knowledge of list.allocated
confined to the two routines you originally taught about it, unless we
believe listobject.c will never be changed again <wink>.

> * list_ass_slice() should add a line to maintain them,
> * list_resize() can drop the test for ob_item != NULL, and
> * tp_new needs a custom list_new() establishing the invariants.

I don't know about the first two, but I don't know why the last would
be needed:  list.allocated starts life at 0 now for the same reason
list.ob_item starts life as NULL now:  PyType_GenericNew guarantees to
zero-fill all the data space (there's nothing special about
list.ob_item in this respect -- the generic new has no idea there's a
pointer in the struct, it simply zeroes out everything).  That
establishes the currently documented invariants.
From ncoghlan at iinet.net.au  Thu Jul 29 23:50:40 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu Jul 29 23:51:16 2004
Subject: [Python-Dev] Static builds on Windows
In-Reply-To: <410944B0.8090104@v.loewis.de>
References: <oem0597y.fsf@python.net>	<4107F8B3.6080206@v.loewis.de>	<8yd352cf.fsf@python.net>	<41081CF6.7050609@v.loewis.de>	<4108243B.4040706@iinet.net.au>
	<410944B0.8090104@v.loewis.de>
Message-ID: <41097130.2020907@iinet.net.au>

Martin v. L?wis wrote:
> Nick Coghlan wrote:
> 
>> If there was going to be both the ability to create a static lib and a 
>> dynamic lib, it would seem to make more sense to have the project for 
>> the dynamic lib project simply incorporate the static lib, and export 
>> the relevant symbols.
> 
> Does that actually work? Doesn't the compiler generate different code
> depending on whether the code is going to be in a static vs. dynamic
> library?

Well, it works for us in VC6. However, I believe the way it is done is 
that the static library projects all define the relevant EXPORT_DLL 
macro. (I'd have to check to be certain, but I won't be back at work 
until October)

And it now occurs to me that we've never tried bypassing the main DLL 
completely, and incorporated the static libraries directly into an 
executable program.

(This part of the program has been stable for so long, it's been a 
couple of years since I've had to think about it!)

> If you can provide a patch to the project files which makes it happen
> for the VC7 projects, I would be very much in favour of including it
> (provided it can be demonstrated that it does actually work, and does
> not cause much overhead beyond the additional static library that is
> going to be generated).

Unfortunately, I don't have access to VC7, even at work - hence the fun 
and games trying to get the free toolkits to cooperate ;)

> Also, there is the issue of what C runtime to chose: should that be
> the static CRT, or a dynmaic one? Clearly, for the DLL, we need the
> dynamic CRT, but for the static python.lib, we might need a static
> CRT instead.

Ouch - OK, that one could be a problem. We link to the dynamic CRT for 
everything - we discovered the hard way that linking a '.lib' to the 
static CRT caused grief whenever we tried to use that library. Since our 
executable would be linking to a *different* CRT from the one the 
library was using, the only way it could work was by ensuring that any 
memory allocated by a particular library was freed by that library. And 
that's a restriction we weren't prepared to apply to the sub-libraries 
(we do apply it at DLL boundaries, though).

Regards,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From python at rcn.com  Thu Jul 29 12:06:19 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Jul 30 00:07:56 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <1f7befae040729145073281b0@mail.gmail.com>
Message-ID: <002901c47553$b22c4e80$c830cb97@oemcomputer>

> I want to insist on *documented* invariants

Absolutely.


> but I don't particularly
> care what they are. 

Right.  I just wanted to point out an alternative invariant that did not
rely on knowledge of the ob_allocated field.



> The idea that "list.ob_item == NULL implies list.allocated may be
> lying" is at least as obscure as the invariant that broke, 
 . . .
> There's no force working toward keeping knowledge of list.allocated
> confined to the two routines you originally taught about it, unless we
> believe listobject.c will never be changed again <wink>.

Okay.  Those good reasons to prefer the invariants you just documented.


> > * list_ass_slice() should add a line to maintain them,
> > * list_resize() can drop the test for ob_item != NULL, and

Will put these in.



> * tp_new needs a custom list_new() establishing the invariants.
>
> > PyType_GenericNew guarantees to
> zero-fill all the data space (there's nothing special about
> list.ob_item in this respect -- the generic new has no idea there's a
> pointer in the struct, it simply zeroes out everything).  That
> establishes the currently documented invariants.

Right.  I remembered that after I sent the last note.  Will add a
comment in list_init so it doesn't get forgotten again.



Raymond

From greg at cosc.canterbury.ac.nz  Fri Jul 30 04:40:38 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Jul 30 04:40:44 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <41065006.6020607@zope.com>
Message-ID: <200407300240.i6U2ecCT021561@cosc353.cosc.canterbury.ac.nz>

Jim Fulton <jim@zope.com>:

>    try:
>        import foo
>    except ImportError:
>        # Configure for absense of foo
>        ...
>    else:
>        # Configure for presense of foo
>        ...
> 
> Unfortunately, this is a bug trap.

Maybe there's room for another creative use of 'else' here...

  import:
    foo
  else:
    # Configure for absense of foo

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From arigo at tunes.org  Fri Jul 30 05:02:49 2004
From: arigo at tunes.org (Armin Rigo)
Date: Fri Jul 30 05:07:30 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <002901c47553$b22c4e80$c830cb97@oemcomputer>
References: <1f7befae040729145073281b0@mail.gmail.com>
	<002901c47553$b22c4e80$c830cb97@oemcomputer>
Message-ID: <20040730030249.GA9086@vicky.ecs.soton.ac.uk>

Hello Raymond,

On Thu, Jul 29, 2004 at 06:06:19AM -0400, Raymond Hettinger wrote:
> > > * list_ass_slice() should add a line to maintain them,
> > > * list_resize() can drop the test for ob_item != NULL, and
> 
> Will put these in.

I think the list_ass_slice() modification you are thinking about is obsolete
since the check-in I documented in my latest e-mail.

list_resize() dropping the test for ob_item != NULL looks like a sane plan:
the only C extensions that I can think of that could break would be the ones
that for extreme performance hacking don't use PyList_SetSlice() to clear a
list, but directly zero out ob_size and ob_item -- which is a real bad thing
to do.

About clearing lists efficiently: what about adding PyList_Clear() to the
public API? It's clearer than the current

   PyList_SetSlice(lst, 0, PyList_GET_SIZE(lst), (PyObject *)NULL)

(which BTW could fail with a MemoryError up to yesterday; I bet not all users
of this idiom were aware of that.  There was even an example of not checking
the return value in listobject.c itself!)  By contrast, PyList_Clear(), like
list_clear(), would never fail and could return void.


A bientot,

Armin.
From tim.peters at gmail.com  Fri Jul 30 07:01:13 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Jul 30 07:01:19 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <20040730030249.GA9086@vicky.ecs.soton.ac.uk>
References: <1f7befae040729145073281b0@mail.gmail.com>
	<002901c47553$b22c4e80$c830cb97@oemcomputer>
	<20040730030249.GA9086@vicky.ecs.soton.ac.uk>
Message-ID: <1f7befae040729220150f92040@mail.gmail.com>

[Raymond Hettinger] wrote:
>>>> * list_ass_slice() should add a line to maintain them,
>>>> * list_resize() can drop the test for ob_item != NULL, and

[ditto]
> > Will put these in.

[Armin Rigo]
> I think the list_ass_slice() modification you are thinking about is obsolete
> since the check-in I documented in my latest e-mail.

Raymond did his checkin, and figured that out.  listobject.c is again
bug-free <wink>.

> list_resize() dropping the test for ob_item != NULL looks like a sane plan:
> the only C extensions that I can think of that could break would be the ones
> that for extreme performance hacking don't use PyList_SetSlice() to clear a
> list, but directly zero out ob_size and ob_item -- which is a real bad thing
> to do.

I'm not worried about that, but Raymond also added an assert() to
catch it if it happens.  Anyone writing such a gonzo extension must
have learned by now to develop in a debug build.

> About clearing lists efficiently: what about adding PyList_Clear() to the
> public API?

+1 provided the signature is changed to return void.  Even inside
listobject.c, the "it always returns 0" qurik is used only once.  It's
sometimes handy to have a function that always returns an error value
(e.g., PyErr_NoMemory), but I think an API function that always
returns a success value would be unprecedented.  That's unexpected,
hence jarring.

> It's clearer than the current
>   PyList_SetSlice(lst, 0, PyList_GET_SIZE(lst), (PyObject *)NULL)

Brrrrrrr.

> (which BTW could fail with a MemoryError up to yesterday;

Yet never has, and never would have <wink>.

> I bet not all users of this idiom were aware of that.

I'd be surprised if any were!.

> There was even an example of not checking the return value in listobject.c
> itself!)

Google quickly found two more, in an older revision of
itertoolsmodule.c.  There are probably more.  Looks like this
particular case ends up in list_clear() now, so that even users of the
broken "clear-all-and-don't-check idiom" would become immune from the
catastrophic consequences of their sloth.

> By contrast, PyList_Clear(), like list_clear(), would never fail and could return
> void.

Yup!
From Paul.Moore at atosorigin.com  Fri Jul 30 10:44:10 2004
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Fri Jul 30 10:44:11 2004
Subject: [Python-Dev] MSI Documentation
Message-ID: <16E1010E4581B049ABC51D4975CEDB8803060F34@UKDCX001.uk.int.atosorigin.com>

From: "Martin v. L?wis"
> As for more documentation on the mechanisms of msi.py itself: I have
> asked a colleague to go through the file and comment on parts of Python
> code that might be unclear. However, I have the concern that all his
> questions are likely answered in MSDN - so I'd be really curious what
> documentation is lacking that is *not* part of MSDN.

I wouldn't worry too much about that. I've never found it easy to follow
the documentation on building MSIs (it is a *long* time since I tried,
maybe things have improved). A bit of duplication wouldn't harm here.

Paul.


__________________________________________________________________________
This e-mail and the documents attached are confidential and intended 
solely for the addressee; it may also be privileged. If you receive this 
e-mail in error, please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos Origin group 
liability cannot be triggered for the message content. Although the 
sender endeavours to maintain a computer virus-free network, the sender 
does not warrant that this transmission is virus-free and will not be 
liable for any damages resulting from any virus transmitted.
__________________________________________________________________________
From martin at v.loewis.de  Fri Jul 30 11:04:05 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Jul 30 11:04:11 2004
Subject: [Python-Dev] MSI Documentation
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8803060F34@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB8803060F34@UKDCX001.uk.int.atosorigin.com>
Message-ID: <410A0F05.9050604@v.loewis.de>

Moore, Paul wrote:
> However, I have the concern that all his
>>questions are likely answered in MSDN - so I'd be really curious what
>>documentation is lacking that is *not* part of MSDN.
> 
> 
> I wouldn't worry too much about that. I've never found it easy to follow
> the documentation on building MSIs (it is a *long* time since I tried,
> maybe things have improved). A bit of duplication wouldn't harm here.

My problem is that I have no clue what bits deserve duplication. To 
fully understand the thing, starting from scratch, you need atleast
50 pages of text. This would exceed the actual lines of code by a great
number, and it would be like explaining Python syntax and semantics
in comments of a Python script.

Regards,
Martin
From arigo at tunes.org  Fri Jul 30 11:57:27 2004
From: arigo at tunes.org (Armin Rigo)
Date: Fri Jul 30 12:02:21 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <1f7befae040729220150f92040@mail.gmail.com>
References: <1f7befae040729145073281b0@mail.gmail.com>
	<002901c47553$b22c4e80$c830cb97@oemcomputer>
	<20040730030249.GA9086@vicky.ecs.soton.ac.uk>
	<1f7befae040729220150f92040@mail.gmail.com>
Message-ID: <20040730095727.GA17@vicky.ecs.soton.ac.uk>

Hello Tim,

On Fri, Jul 30, 2004 at 01:01:13AM -0400, Tim Peters wrote:
> > About clearing lists efficiently: what about adding PyList_Clear() to the
> > public API?
> 
> +1 provided the signature is changed to return void.  Even inside
> listobject.c, the "it always returns 0" qurik is used only once.

Actually, inside listobject.c, the 'return 0' comes from the fact that
list_clear() is used for the type's tp_clear.

Something I didn't think about is that the PyList_Xxx() API checks the type of
its first argument.  If PyList_Clear() does it too it cannot return void.

:-(


Armin
From jim at zope.com  Fri Jul 30 14:28:18 2004
From: jim at zope.com (Jim Fulton)
Date: Fri Jul 30 14:28:59 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com>
References: <200407282213.i6SMDWD06878@guido.python.org> <Your message of "Wed,
	28 Jul 2004 13:40:03
	EDT."	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<Your message of "Wed,
	28 Jul 2004 06:59:24 EDT." <4107870C.1070404@zope.com>
	<41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
	<41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org>
	<4107870C.1070404@zope.com>
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com>
	<200407282213.i6SMDWD06878@guido.python.org>
	<5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com>
Message-ID: <410A3EE2.7010805@zope.com>

Phillip J. Eby wrote:
> At 05:02 PM 7/29/04 -0400, Jim Fulton wrote:
> 
>> Guido van Rossum wrote:
>>
>>> But who ever uses __module__ for anything else than printing it?
>>
>>
>> It's crucial for pickling.
> 
> 
> Interesting point.  That means that cPickle wouldn't work in a "module 
> space" without modification.  The pure-Python pickle module would 
> unpickle into the current module space.

And that might be the wrong module space.  If one was going to introduce a
concept of module spaces, one would need to extend the pickling machinery
as well.

I'll note that Python already has module spaces. They're called "packages". :)

The problem with Python's current package system is that it's not possible,
in general, to write packages that can be moved around within the package
system, because relative imports aren't reobust or flexible enough.
PEP 328 would fix this. PEP 328 would allow creation of packages that
worked regardless of where there were places or that worked relative
to a containing package regardless of where that was placed. You could
then have different module spaces expressed by adding a new layer of top-level
modules.  You wouldn't need any new machinery beyond PEP 328.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From pje at telecommunity.com  Fri Jul 30 17:06:20 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Jul 30 17:02:45 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <410A3EE2.7010805@zope.com>
References: <5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com>
	<200407282213.i6SMDWD06878@guido.python.org> <Your message of "Wed,
	28 Jul 2004 13:40:03
	EDT."	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<Your message of "Wed,
	28 Jul 2004 06:59:24 EDT." <4107870C.1070404@zope.com>
	<41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
	<41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org>
	<4107870C.1070404@zope.com>
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com>
	<200407282213.i6SMDWD06878@guido.python.org>
	<5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com>

At 08:28 AM 7/30/04 -0400, Jim Fulton wrote:
>The problem with Python's current package system is that it's not possible,
>in general, to write packages that can be moved around within the package
>system, because relative imports aren't reobust or flexible enough.
>PEP 328 would fix this. PEP 328 would allow creation of packages that
>worked regardless of where there were places or that worked relative
>to a containing package regardless of where that was placed. You could
>then have different module spaces expressed by adding a new layer of top-level
>modules.  You wouldn't need any new machinery beyond PEP 328.

Hm.  The concept I'm proposing would allow code to be used in a module 
space without needing to know about module spaces or be written to 
co-operate in such a fashion.  If you can make everybody conform to some 
sort of coding standard, you wouldn't even need PEP 328 to accomplish your 
goal.  ;)

As a counterexample, consider seven Zope products that depend on two 
different versions of PIL.  Using PEP 328, you can't make this work without 
either loading seven copies of PIL, or else requiring the version number to 
be part of the package name.  You actually have it somewhat better right 
now because you can at least fool the "this package or absolute" mechanism 
into doing what you want.  If I understand correctly, PEP 328 will 
ultimately remove the possibility of doing it that way.

From fumanchu at amor.org  Wed Jul 28 19:50:05 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Fri Jul 30 18:37:47 2004
Subject: [Python-Dev] Can we limit the effects of module execution
	tosys.modules? (was Fix import errors to have data)
Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022DB7@exchange.hqamor.amorhq.net>

Barry Warsaw wrote:
> I wonder if we can't define a protocol for executing code at module
> scope, but only if the module is successfully imported.  The idea is
> that we'd define something like an __init__() for modules that would
> only get executed after the module was imported, and if there were any
> failures in importing, it wouldn't get called.
> 
> We'd have to handle chained imports though.  I haven't thought through
> all the implications of that but I can imagine it will get messy.  You
> probably don't want to execute any __init__()'s until all imports are
> finished, you probably want to execute them in the order you saw them,
> and you don't want any exceptions raised in __init__()'s to 
> be deemed an ImportError worthy of sys.modules deletion.

I'll just note that many large projects do this sort of thing on their
own already, because the order of imports/execution is both critical and
highly project-specific. If such an __init__() were to be developed, the
primary benefit IMO would be reducing many of those ad-hoc schemes to
"one way to do it", which might be worth the effort even if it doesn't
"fix" the original issue.


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
From jlenton at gmail.com  Fri Jul 30 19:07:52 2004
From: jlenton at gmail.com (John Lenton)
Date: Fri Jul 30 19:07:54 2004
Subject: [Python-Dev] PEP 262 - active?
In-Reply-To: <20040728190025.GA13981@rogue.amk.ca>
References: <200407230826.21976.heikowu@ceosg.de>
	<2d9e37f9040728060493c32d1@mail.gmail.com>
	<20040728190025.GA13981@rogue.amk.ca>
Message-ID: <2d9e37f904073010075b058371@mail.gmail.com>

On Wed, 28 Jul 2004 15:00:25 -0400, A.M. Kuchling <amk@amk.ca> wrote:
> On Wed, Jul 28, 2004 at 10:04:38AM -0300, John Lenton wrote:
> > difficult) to keep both in sync; the proposed "package manager
> > installs files" is good, but maybe it can be improved...?
> 
> The "install-a-file" scheme was written that way at the request of
> some Red Hat people who were at PyCon, precisely so that RPMs could
> also keep the package database up to date.

and it's a good base to start with, I didn't mean the contrary. What I
meant was that it should be done in such a way that if Debian or
RedHat wanted to they could subclass (or implement the interface) to
attain tighter integration.

-- 
John Lenton (jlenton@gmail.com) -- Random fortune:
bash: fortune: command not found
From guido at python.org  Fri Jul 30 22:17:25 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Jul 30 22:17:31 2004
Subject: [Python-Dev] Pie thrown
Message-ID: <200407302017.i6UKHPJ14281@guido.python.org>

The pie has been thrown.

As was known ahead of time, the Parrot team conceded the contest:

http://www.mail-archive.com/perl6-all@perl.org/msg40418.html

(and followup).

At the start of OSCON's final keynote, in front of (2000?) attendees,
I received $10 and a 12-pack of beer, and Dan received a cream pie in
the face.  I believe the event was captured on video.

Thanks to Dan for being a good sport!

We'll be consuming the beer at the Vancouver Python workshop.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From mwh at python.net  Thu Jul 29 12:19:19 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Jul 30 22:56:12 2004
Subject: [Python-Dev] Re: Rationale behind removing kwdict from keyword.py?
In-Reply-To: <ce7857$tn0$1@sea.gmane.org> (Fernando Perez's message of "Tue,
	27 Jul 2004 21:56:25 -0600")
References: <ce6ve7$g4m$1@sea.gmane.org>
	<1f7befae04072719134056c647@mail.gmail.com>
	<ce7857$tn0$1@sea.gmane.org>
Message-ID: <2macxjf74o.fsf@starship.python.net>

Fernando Perez <fperez528@yahoo.com> writes:

> Tim Peters wrote:
>
>> Note that it's very common in older modules (which keyword.py is) to
>> define the newer __all__ rather than risk breaking code by massive
>> renaming of variables to stick an underscore in front of them.
>
> Fair enough.  For a long time I'd gotten used to assuming that the lack of a
> leading _ was a safe sign of 'publicness' (that's probably not a word :), and
> didn't really rely too much on __all__.  Lesson learned... (I wouldn't be
> surprised to find I'm not the only one with this bad habit)

Well, not entirely your fault, I'd say.  If kwdict wasn't part of the
public API of the module, it was arguably a bug in all previous
versions of the module that it *didn't* start with an underscore.

Fortunately, this is shallow and easy to mend, but small random
breakages like still to be avoided.

Cheers,
mwh

-- 
  well, take it from an old hand: the only reason it would be easier
  to program in C is that you can't easily express complex problems
  in C, so you don't.                   -- Erik Naggum, comp.lang.lisp
From mwh at python.net  Thu Jul 29 12:25:22 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Jul 30 22:56:15 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <20040729093355.GA10255@vicky.ecs.soton.ac.uk> (Armin Rigo's
	message of "Thu, 29 Jul 2004 10:33:55 +0100")
References: <20040728223645.3B1E61E4002@bag.python.org>
	<200407290000.i6T00xE07145@guido.python.org>
	<20040729002055.GA31229@vicky.ecs.soton.ac.uk>
	<1f7befae0407281953799472ad@mail.gmail.com>
	<1f7befae04072820551bfa2a87@mail.gmail.com>
	<1f7befae04072821116b415e07@mail.gmail.com>
	<20040729093355.GA10255@vicky.ecs.soton.ac.uk>
Message-ID: <2m6587f6ul.fsf@starship.python.net>

Armin Rigo <arigo@tunes.org> writes:

> The invariants of the ob_xxx list members should be clearly
> specified somewhere and we should stick to them.

Yes.

> E.g. it is unclear to me why we allow list_ass_slice() to reset
> ob_item to NULL and ob_size to 0 without resetting ob_allocated to 0
> -- I see why it doesn't crash in a subsequent list_resize(), but it
> looks messy.

Well, "a mistake" would be my guess <wink>.

Cheers,
mwh

-- 
  You sound surprised.  We're talking about a government department
  here - they have procedures, not intelligence.
                                            -- Ben Hutchings, cam.misc
From gward at python.net  Sat Jul 31 00:39:36 2004
From: gward at python.net (Greg Ward)
Date: Sat Jul 31 00:39:38 2004
Subject: [Python-Dev] Optik 1.5 in Python 2.4
Message-ID: <20040730223936.GA26696@cthulhu.gerg.ca>

Hope I'm not too late to get Optik 1.5 into Python 2.4.  I release Optik
1.5a1 last weekend and so far haven't heard a peep of complaint.
Sneaking it into Python 2.4a2 would help ferret out any compatibility
bugs.  (The changes are pretty small; the only reason I called it Optik
1.5 instead of 1.4.2 is because there are a bunch of small changes, and
it's over a year since 1.4.1.  ;-)

The only catch is that I've updated the Optik reStructuredText docs, and
haven't yet devised a clever way to convert them to LaTeX-for-the-
Python-manual format (talking about optparse rather than Optik).  (Code
donations welcome.  ;-)

So: I'd like to checkin Optik 1.5a1 as optparse.py right now, and worry
about the docs later.  Hopefully before Python 2.4a2 goes out the door,
but no promises.  Objections?  If I don't hear any soon, I'll do the
deed tomorrow morning.

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
Everybody is going somewhere!!  It's probably a garage sale
or a disaster Movie!!
From tim.peters at gmail.com  Sat Jul 31 02:57:41 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Jul 31 02:57:45 2004
Subject: [Python-Dev] MSI Documentation
In-Reply-To: <410A0F05.9050604@v.loewis.de>
References: <16E1010E4581B049ABC51D4975CEDB8803060F34@UKDCX001.uk.int.atosorigin.com>
	<410A0F05.9050604@v.loewis.de>
Message-ID: <1f7befae04073017577c0ee9d6@mail.gmail.com>

[Martin v. L?wis, on documenting MSI-building]
> My problem is that I have no clue what bits deserve duplication. To
> fully understand the thing, starting from scratch, you need at least
> 50 pages of text. This would exceed the actual lines of code by a great
> number, and it would be like explaining Python syntax and semantics
> in comments of a Python script.

I think you can assume basic familiarity with MSI-building.  Then,
like anything else, you comment parts you found especially difficult,
confusing, subtle, or delicate to work out -- even if they're clearly
covered somewhere in the multiple gigabytes of MSDN verbiage, and it
was just hard for you to find where.  Sometimes crucial details are
buried in footnotes.  Or, if it's like any other MS API I've ever
used, sometimes crucial details are plain missing, and "the thing that
worked" was the result of a dozen or ten poke-and-hope attempts.
From tim.peters at gmail.com  Sat Jul 31 04:09:25 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Jul 31 04:09:28 2004
Subject: [Python-Dev] RE: test_sort.py failure
In-Reply-To: <20040730095727.GA17@vicky.ecs.soton.ac.uk>
References: <1f7befae040729145073281b0@mail.gmail.com>
	<002901c47553$b22c4e80$c830cb97@oemcomputer>
	<20040730030249.GA9086@vicky.ecs.soton.ac.uk>
	<1f7befae040729220150f92040@mail.gmail.com>
	<20040730095727.GA17@vicky.ecs.soton.ac.uk>
Message-ID: <1f7befae04073019095ec163a3@mail.gmail.com>

[Armin Rigo]
> Actually, inside listobject.c, the 'return 0' comes from the fact that
> list_clear() is used for the type's tp_clear.

Eww, yes.  I missed that.

> Something I didn't think about is that the PyList_Xxx() API checks the type of
> its first argument.  If PyList_Clear() does it too it cannot return void.
> 
> :-(

You could get away with breaking the pattern for this one, provided
it's documented that behavior is undefined if an object of wrong type
is passed.  Unclear whether it's worth it.  But call it PyList_CLEAR()
and Guido wouldn't even blink <wink>.
From tim.peters at gmail.com  Sat Jul 31 06:17:15 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Jul 31 06:17:17 2004
Subject: [Python-Dev] Re: Can we limit the effects of module execution to
	sys.modules? (was Fix import errors to have data)
In-Reply-To: <4107866C.7060609@zope.com>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4107866C.7060609@zope.com>
Message-ID: <1f7befae04073021175bc48767@mail.gmail.com>

[Tim, wants to keep insane modules out of sys.modules]

[Jim Fulton]
> I sympathize with your frustration with this problem, but I think that
> the problem is bigger that just sys.modules.  For better or worse, importing
> a module may have side effects that extend beyond sys.modules. For example,
> In some applications, objects get registered into registries that exist in
> already-imported modules.  Perhaps we want to declare this to be a
> poor style.  If a module has an impact beyond new modules added to
> sys.modules, then removing all modules imported into sys.modules as
> a result of attempting the import would produce bugs even more subtle
> than what we have now.

I wouldn't want to remove all, just the modules that failed.  For example,

    A imports B
        B imports C # no problem
        B imports D # and that raises an exception not caught by B

C is fine, I only want to nuke D and B.

As to style, in my own code I strive to make modules "reload safe". 
So, for example, I wouldn't even consider doing one of these things as
a side effect of merely importing a module:

+ Create a lock file.
+ Start a thread.
+ Open a socket.
+ Register with a registry indeed.

Now that said, I've only seen imports wrapped in a try in two ways:

1.

try:
    import X
except ImportError:
    something

That's invariably trying to check for the availability of X, though,
not also trying to check for whether something X imports doesn't
exist.  If you pursue a saner way to write that, I'll always use it.

2.

try:
    import X
except:
    something

That one is almost always a mistake, as a bare "except" is almost
always a mistake in any context.  The author almost always intended
the same thing as #1, but was too lazy or inexperienced to write that.
 Bug in, bugs out.  That a later attempt to import X doesn't also fail
is a bug magnifier.

I've never seen something like

3.

try:
    import X
except ZeroDivisionError:
    something

If, as I suspect, nobody (and "almost nobody" is the same to me
<wink>) *intends* to catch an error from an import other than
ImportError, then import errors other than ImportError are fatal soon
after in practice, and then there's nothing much to worry about.

Catching ImportError still leaves insane modules around, though, and
that does cause real problems.  You've convinced me I'd rather have a
better way to spell "does X exist?" than catching an ImportError from
an attempt to import X.

> Do you think it's practical to limit the effects of module import to
> sys.modules, even by convention?

I'm sure you didn't intend that to be *so* extreme -- like surely a
module is allowed to initialize its own module-level variables.  If I
read "effects" as "effects visible outside the module", then that's
what you said <wink>.

> Could we say that it is a bug for code executed during module import to
> mutate other modules, including mutating objects contained in those other
> modules?  I would support this myself.

It's hard to spell the intent precisely.  "reload safe" covers a world
of non-local (wrt the module in question) that both are and aren't
problematic.  For example, calling random.random() during module
initialization should be fine, but it certainly mutates state, and
irrevocably so, inside the random module. Because it's hard to be
precise here, best practice is likely to remain more a matter of good
judgment than of legislation.

> If it is possible to limit the effects of import (even by convention),
> then I think it would be practical to roll-back changes to sys.modules.
> If it's not practical to limit the effects of  module import then I think
> the problem is effectively unsolveable, short of making Python transactional.

There we don't agree -- I think it's already practical, based on that
virtually no Python application *intends* to catch errors from imports
other than ImportError, so that almost all "real bugs" in module
initialization are intended to stop execution.  In turn, in the cases
where ImportErrors are intentionally caught now, they generally occur
in "import blocks" near the starts of all modules in the failing
import chain, and so none of the modules involved have yet *done* any
non-trivial initialization -- they're all still trying to import the
stuff they need to *start* doing the meat of their initialization.  If
some modules happen to import successfully along the way, fine, they
should stay in sys.modules, and then importing them again later won't
run their initialization code again.  IOW, once a module has announced
its sanity by importing successfully, I want that to "stick" no matter
what happens later.

> Personably I'm inclined to consider errors that occur while executing a
> module to be pretty much fatal.  If a module has begun executing, I really
> don't know what state it's in or what state it might have left other modules
> in.  I'd rather report the error and get some human to fix it.

I think that's widespread belief too.  Heck, if Zope doesn't violate
it, who else would be so perverse <wink>?

> OTOH, I'm happy to recover from the inability to find a module as long as no
> module code has been executed.

Having a clearer way to determine module availability/existence would
be a real help.

> FWIW, In Zope, we generally generally limit non-transactional state
> changes to program startup. For that reason, we make little or no attempt to
> survive startup errors.

I've never tried to survive a startup error myself either, nor have
any Python projects I'm aware of attached to any of my previous
employers.  Anyone else?
From gward at python.net  Sat Jul 31 18:24:48 2004
From: gward at python.net (Greg Ward)
Date: Sat Jul 31 18:24:52 2004
Subject: [Python-Dev] Optik 1.5 in Python 2.4
In-Reply-To: <20040730223936.GA26696@cthulhu.gerg.ca>
References: <20040730223936.GA26696@cthulhu.gerg.ca>
Message-ID: <20040731162447.GA29931@cthulhu.gerg.ca>

On 30 July 2004, I said:
> So: I'd like to checkin Optik 1.5a1 as optparse.py right now, and worry
> about the docs later.

OK, optparse.py and test_optparse.py are now upgraded to Optik 1.5a1.
Docs ... ummm ... later.

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
If at first you don't succeed, redefine success.
From tim.peters at gmail.com  Sat Jul 31 22:24:52 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Jul 31 22:24:55 2004
Subject: [Python-Dev] Test breakage due to optparse
Message-ID: <1f7befae040731132452604ed7@mail.gmail.com>

Heads up.  Recent changes to optparse and/or to its test are causing
several spurious test failures,  Here's a minimal incantation to get
the first new failure of this kind:

C:\Code\python\PCbuild>python  -E -tt ../lib/test/regrtest.py
test_optparse test_pep277
test_optparse
test_pep277
test test_pep277 produced unexpected output:
**********************************************************************
*** lines 2-4 of actual output doesn't appear in expected output after line 1:
+ u'\xdf-\u66e8\u66e9\u66eb'
+ ['???', '???', '??????', '????????????', '????G\xdf', 'Ge??-sa?',
'Gr\xfc\xdf-Gott', 'abc', 'ascii']
+ [u'Gr\xfc\xdf-Gott', u'abc', u'ascii',
u'\u0393\u03b5\u03b9\u03ac-\u03c3\u03b1\u03c2',
u'\u0417\u0434\u0440\u0430\u043
2\u0441\u0442\u0432\u0443\u0439\u0442\u0435',
u'\u05d4\u05e9\u05e7\u05e6\u05e5\u05e1', u'\u306b\u307d\u3093',
u'\u66e8\u
05e9\u3093\u0434\u0393\xdf', u'\u66e8\u66e9\u66eb']
**********************************************************************
1 test OK.
1 test failed:
    test_pep277

In all, with a complete test run, there are 15 spurious test failures
on Windows now:

15 tests failed:
    test_pep277 test_pkg test_popen test_popen2 test_profile
    test_pyexpat test_regex test_rgbimg test_rotor test_scope
    test_thread test_threadedtempfile test_tokenize test_winreg
    test_xreadline

No idea why.  Offhand, from the symptoms it looks like a change may be
sending stdout into a black hole.  Constructs like

          sys.stdout = StringIO()
!         self.assertRaises(self.parser.parse_args, (cmdline_args,), None,
!                           SystemExit, expected_output,
!                           self.redirected_stdout)
          sys.stdout = sys.__stdout__

in the patch certainly intensify that suspicion <wink>.
From tim.peters at gmail.com  Sat Jul 31 22:59:59 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Jul 31 23:00:02 2004
Subject: [Python-Dev] Re: Test breakage due to optparse
In-Reply-To: <1f7befae040731132452604ed7@mail.gmail.com>
References: <1f7befae040731132452604ed7@mail.gmail.com>
Message-ID: <1f7befae040731135919642b19@mail.gmail.com>

[Tim]
> Heads up.  Recent changes to optparse and/or to its test are causing
> several spurious test failures, ...

OK, this is definitely the fault of the recent optparse checkins.  For
whatever reason, after test_optparse runs, regrtest.py is convinced
that the test directory is '/foo/bar'.  After that, it can't (among
other things) find the expected-output files anymore for tests that
run after test_optparse.

I'm sure I'll have that fixed soon.  But I want to whine about this
one more time:  PLEASE don't check things in without running the whole
test suite!  Each weekend now I'm struggling like hell to give some
time to Python, and I largely piss it away fixing new test breakage in
tests that must have failed on the platform where the patch was
developed.  I have all the sympathy in the world for Windows-specific
test breakage due to patches developed on Linux, but I'm out of
sympathy for plain sloppiness.

So there.
From goodger at python.org  Sat Jul 31 23:37:28 2004
From: goodger at python.org (David Goodger)
Date: Sat Jul 31 23:40:31 2004
Subject: [Python-Dev] Re: Test breakage due to optparse
In-Reply-To: <1f7befae040731135919642b19@mail.gmail.com>
References: <1f7befae040731132452604ed7@mail.gmail.com>
	<1f7befae040731135919642b19@mail.gmail.com>
Message-ID: <410C1118.6020003@python.org>

Tim Peters wrote:
 > PLEASE don't check things in without running the whole test suite!

I for one would rather that you didn't waste your time on something
that somebody else could have and should have done instead.  Why not
adhere to the Checkin Policy?

     * Before checking in a code change, re-run the entire Python test
       suite to be sure that you haven't broken anything.

     -- http://www.python.org/dev/tools.html

Especially in a clear-cut case like this, just revert the changes and
send the ball back to the offender's court.

-- 
David Goodger <http://python.net/~goodger>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20040731/64cb3f98/signature.pgp